AARL

The relevance of vendors' usage statistics in academic library e-resource management: a New Zealand study

Nicola McDowell and G E Gorman

Abstract Vendors' usage statistics for e-resources can be a valuable tool for informing collection management decisions; accordingly, this study looked at their usefulness to New Zealand academic librarians for collection management decision-making. Data were gathered from vendors on usage statistics supplied to the University of Otago for databases and e-journals. Then librarians at academic libraries in New Zealand were surveyed to elicit how useful they found vendors' usage statistics. The study found that New Zealand academic libraries utilised vendors' usage statistics for informing collection management decisions. However, there was no significant correlation between the vendors' usage statistics currently provided and those desired by academic librarians. In contrast to previous work, this study found that the usage statistics desired by librarians at different academic libraries varied, and that customisable usage statistics would be valuable. In concurrence with previous studies it was also found that the lack of standardisation in vendors' usage statistics limited their usefulness and was a key concern for many academic librarians.

Academic libraries subscribe to an increasing number of e-resources, including e-journals and indexing, abstracting and full-text databases, [1] and experience indicates that the use of e-resources is increasing. Accordingly, librarians need to develop effective collection management principles, procedures and policies specifically for managing electronic information.

Vendors provide usage statistics for their e-resources, which can be used by librarians to justify funding applications, [2] and to inform collection development decisions, [3] deselection decisions [4] and cancellation decisions. [5] However, the usefulness of usage statistics can be affected by:

• the type of data provided and how they are supplied, [6]
• the manager's understanding of the statistics, [7] and
• the manager's skill in utilising statistical data. [8]

The purpose of this research was to determine whether academic librarians in New Zealand use vendors' usage statistics to inform their collection management decisions, and to understand how useful they find the data currently provided.

The literature on usage statistics

The now classic University of Pittsburgh study highlighted the relevance of usage statistics to collection management by combining both the cost of the material and operational costs with the usage of the resource to determine a cost per use. [9] This study concluded that, in order to maintain a cost-effective collection and to better meet user's needs with the budget available, ongoing monitoring of usage was essential.

What are vendors' usage statistics used for?

As the best indicator of future use, [10] details of past usage provided in vendors' usage statistics can direct and justify funding applications, [11] collection development decisions, [12] de-accession and storage decisions, [13] and cancellation decisions. [14] However, little is known about whether New Zealand academic libraries use vendors' usage statistics to inform their collection management decisions.

Many vendors are changing their policy of providing the electronic version free with a print subscription, so librarians have to decide which, or both, to subscribe to. Usage statistics for the electronic and print formats are an important tool for informing that decision, although other issues should also be considered, including the provider, licensing, local politics, publication structure, technological considerations and local resources. [15]

Reliability of usage statistics

Usage statistics provided by different vendors vary widely in their format, usability and definitions of terminology. [16] This makes it difficult to compare usage across vendors, and needs to be considered when usage statistics are employed as a measure of user demand for resources or to facilitate selection, retention or cancellation decisions. Furthermore, a vendor's usage statistics often serve the vendor's purpose better than the librarian's. [17] For example, because usage statistics are employed to assess the value of their resources, vendors try to make the usage data look as good as possible. [18] This contributes to variation between the definitions and formats used by vendors, and also decreases the relevance of usage statistics for collection management decisions.

Standardisation of usage statistics

There is a consensus in the literature that, to increase their versatility for informing collection management decisions, vendors' statistics should be standardised to make them easier to analyse and comparable both across vendors and from year to year. [19] In New Zealand it is not known what the level of standardisation among vendors is, or whether this is problematic for academic librarians.

Although some guidelines for standardising the collection of usage statistics are emerging (for example, Project COUNTER and the ICOLC guidelines), they are not yet widely adopted. [20] The guidelines outline what types of data should be collected, how they should be delivered and the format of the report. [21] If widely adopted, the standards would help ensure that usage statistics were more useful, comparable and reliable. While most vendors know of the guidelines, few have implemented them. [22] Work is necessary to understand why they are not being adopted so that the issue can be addressed more effectively.

While collaboration between vendors and librarians on standardisation of data is important, librarians are not totally reliant on the vendors' co-operation for improving the versatility of usage statistics for informing collection management decisions. For example, librarians could examine data monthly to spot variations in use of e-resources or in reporting that usage, examine data for individual titles, and consider the data in terms of the user population. [23] Librarians can also use locally collected usage statistics to check the reliability of vendors' data or to enable an accurate comparison across different vendors. [24] Ideally, vendors' usage statistics should be standardised, but until then it is valuable for librarians to know how to use already available data productively to assess user demand for resources accurately.

Vendors' usage statistics and the needs of academic librarians

As noted, vendors' usage statistics are used by librarians to justify funding applications, [25] and to inform collection development decisions, [26] deselection decisions [27] and cancellation decisions. [28] However, while research shows that librarians want vendors to provide information on usage of e-resources, to inform their collection management decisions, [29] little work shows what types of data are most useful for this purpose.

In contrast, much work has demonstrated the importance of standardising vendors' usage statistics, especially if usage is to be compared across e-resources or from year to year. [30] However, research is needed to see if the types of data outlined in the guidelines are the most useful to all New Zealand academic librarians, or whether their needs differ.

Methodology

The research was conducted in two sequential phases. The first phase gathered information on the type of data provided in vendors' usage statistics, their format and delivery, and vendor use of terminology. This quantitative data informed the design of the survey that was used in the second phase of the research. The second phase survey sought to obtain information from academic librarians about the use and relevance of vendors' usage statistics to collection management decisions. Qualitative questions were also included to elicit more detail to facilitate the examination and interpretation of the quantitative data collected.

For phase one the focus of the study was the University of Otago Library in Dunedin, which subscribes to a wide range of e-resources, including databases and e-journals from a variety of vendors. At the University of Otago 37 vendors provide access to e-journal products, and 38 vendors provide access to 76 online and CD-ROM database products.

The population was stratified because of the potential for differences between the vendors of databases and the vendors of e-journals. The two populations were then randomly sampled using Nth sampling [31] to select 11 database products and 11 e-journal vendors for the study (Table 1). It was assumed that the usage statistics provided by a particular vendor to the University of Otago would be the same as those provided to other New Zealand academic libraries by the same vendor.

Table 1
The eleven database products and eleven e-journal vendors

Both online usage statistics provided by vendors to the University of Otago and vendors' websites were used to gather examples of usage statistics from each vendor/e-resource. The response rate for the vendor survey was 95 per cent. Information was obtained from 21 of the 22 vendors, and for each vendor six questions were addressed:

• Are usage statistics provided by the vendor/for the resource?
• What type of data are provided?
• In what format are the usage statistics provided?
• How are usage statistics delivered?
• What standards are complied with?
• Is the term 'use' clearly defined?

A coding manual was used to code data gathered from each vendor, and some responses given in the category 'other' were post-coded. A unique identity number was used to identify each vendor, while retaining confidentiality. A frequency distribution table was used to describe the data, and because it was nominal data the measure of central tendency used was the mode. In both phases one and two of the study missing values were not replaced by a mean or other value. Instead statistical analysis was only of actual responses. Any not applicable (N/A) responses were also treated as missing data.

In phase two the survey included all 32 public tertiary institutions in New Zealand (universities, polytechnics, institutes of technology and colleges of education). The collection management librarian (or equivalent) was asked to respond on behalf of each institution, involving other staff as appropriate. If no member of staff with collection management responsibility could be identified, the library manager was selected to participate. Responses were received from all four types of tertiary institutions, and responding libraries had a range of e-resources from a variety of vendors.

The survey consisted of a series of quantitative questions complemented by qualitative questions to enable participants to enrich their answers with further detail if they chose. The survey covered the following areas:

• demographic information about the participant's library, role, e-resources subscribed to, and vendors used,
• whether they accessed usage statistics, and if so, how often,
• whether they used them to inform their collection management decisions; why or why not; if so, how,
• the interpretation of the terminology vendors use for their statistical categories,
• the format and delivery of the usage statistics,
• the type of usage data provided, and
• additional comments.

The overall response rate for the survey was 59 per cent. Thirty-two surveys were distributed and of the 20 returned, 19 had been completed.

A coding manual was used to code the survey responses, and the qualitative data was post-coded. Both ordinal and nominal data were gathered in the survey. The nominal data were described using a frequency distribution table. The ordinal data, where participants ranked their choices, were described by calculating the median, the mode, the maximum and minimum and the range. For the most part the limited size of datasets meant it was not feasible to employ chi-square or other tests of significance.

Phase one results

Question 1: Are usage statistics provided?

It was found that 80 per cent of vendors provided usage statistics to libraries subscribing to their e-resource. Of the 20 per cent of vendors who did not provide usage statistics, one was planning to introduce them in 2004.

Question 2: What types of usage statistics are provided?

The type of usage statistic most frequently provided was 'usage broken down by calendar month' (all vendors), and the least commonly provided was 'publication date of material accessed' (13 per cent). At least one type of data in their usage statistics was provided by 63 per cent of vendors. Figure 1 shows that only three types of data were provided by more than half of the vendors, 'usage by month', 'number of turnaways' and 'type of material accessed'; but ten other types of data were only provided by one vendor and were categorised as 'other'.

Figure 1
Type of usage statistics provided by vendors (N=16)

Question 3: What format is used to deliver the usage statistics?

The most common format was HTML (73 per cent), followed by comma-separated values or CSV (67 per cent), while the least frequently offered formats were ASCII (20 per cent) and Excel (13 per cent). Over half (62 per cent) of the vendors offered more than one delivery format for their usage statistics so librarians could choose the one that suited their needs and preference.

Question 4: How are the usage statistics delivered?

The most common method for the delivery of usage statistics was online (81 per cent of vendors). The only other method used by the vendors was e-mail, used by the remaining 19 per cent.

Question 5: What standards are complied with?

The most frequent response to this question (69 per cent) was that no standards were complied with. Of the 31 per cent of vendors who complied with standards, all complied with COUNTER standards, and six per cent also complied with the ICOLC standards. One vendor, who was not yet providing usage statistics, was planning to provide statistics that complied with the ICOLC standard from 2004.

Question 6: Is the term 'use' clearly defined?

A definition of use was outlined by 69 per cent of vendors when calculating the usage statistics for e-resources, but for all of these vendors the definition of use was different.

Table 2 shows the relationship between the vendors who complied with standards and those who provided a definition of 'use'. The data suggest a relationship between compliance with standards and defining 'use'.

Table 2
Vendors who define 'use' and who comply with standards (N=16)

Percentage subject to rounding

Phase two results

Question 1: Please select your job title

Surveys were most frequently completed by either the collection management/technical services librarian (47 per cent), or the systems librarian (42 per cent), although 21 per cent of the surveys were completed by multiple staff.

Question 2: Please select the type of library

Survey responses were received from all four types of academic library included in the population sample. The greatest percentage of responses was received from polytechnic libraries (42 per cent) and the smallest from college of education libraries (11 per cent).

Question 3: Please list the main electronic resources that you purchase/subscribe to

The 19 responding libraries identified 121 different electronic resources as the main ones subscribed to. Of these, the most frequently identified e-resources were Index New Zealand and Te Puna (the National Library of New Zealand bibliographic database), both identified by 63 per cent of libraries. The next most frequently identified e-resources were ProQuest 5000 (58 per cent of libraries), and Expanded Academic ASAP (37 per cent of libraries). Only one library identified 84 different e-resources (69 per cent of the total).

Question 4: Please list the vendors from which you mainly purchase/subscribe to e- resources

The 19 responding libraries identified 35 main vendors from which e-resources were purchased. The vendor identified most frequently was the National Library of New Zealand (74 per cent of libraries). The next most frequently identified were Ebsco and ProQuest, (58 per cent of libraries), followed by Knowledge Basket and Thomson Gale, which were identified by 37 per cent of libraries. Only one library identified 18 vendors (51 per cent).

Question 5: How often does your library access usage statistics?

Table 3
Frequency of accessing usage statistics by library type (N=19)

Percentages subject to rounding

Usage statistics were accessed by 53 per cent of libraries at least monthly, while 26 per cent accessed usage statistics at least annually, and eleven per cent either accessed usage statistics at least weekly or did not access them at all. Table 3 shows that the frequency of accessing usage statistics varied with library type. The data suggest a relationship between frequency of accessing usage statistics and library type.

Question 6: Please indicate how much you agree/disagree with each statement

The most frequent answer to all sections of Question 6 was 'Agree', and for all but one of the sections no libraries disagreed with the statements. (These statements appear on the axis of Figure 2.) However, the proportion that strongly agreed, agreed, disagreed or strongly disagreed varied (Figure 2).

Figure 2
Use of vendors' usage statistics in academic libraries (N=17)

Seven other ways in which usage statistics were employed in libraries were identified, the most frequently being for compiling annual reports (16 per cent), and to monitor off-site/on-site use of e-resources (16 per cent). Cross-tabulation of the uses of vendors' usage statistics by library type did not suggest any relationship between library type and the employment of usage statistics for informing collection management decisions.

Question 7: What do you consider 'use of an electronic resource' to mean?

Respondents were presented with a set of definitions and could select multiple definitions. All responding libraries understood 'use' to mean either doing a search, or accessing a table of contents, abstract or full text of an article. The majority (95 per cent) of libraries also understood 'use' to include browsing the e-resource, and 84 per cent understood 'use' to mean printing from the e-resource. Only 26 per cent of libraries understood 'use' to mean accessing the homepage of the e-resource. However, vendors defined 'use' more narrowly, and the interpretation varied across vendors (Table 4). For four of the seven categories there was a consensus among librarians that it did constitute a use of the e-resource, whereas the only consensus among vendors was that printing from the e-resource did not constitute use.

Table 4
Vendors' and librarians' understanding of 'use' of an e-resource

Percentages subject to rounding.
* A total percentage is not applicable to multiple response questions.

Question 8: Please rank each format for delivering usage statistics listed below in order of preference on a scale of 1 to 6.

The range of preferences for delivery format of usage statistics was broad. Excel, followed by Word, were the most preferred delivery formats, and the least preferred was ASCII (Figure 3).

Table 5 shows that two delivery formats used most frequently by vendors (HTML and CSV) were not the two delivery methods most preferred by librarians (Excel and Word).

Figure 3
Percentage of respondents who ranked each delivery format (N=18)

Table 5
Delivery format of usage statistics most frequently offered by vendors (N=16) and most preferred by librarians (N=18)

Percentages subject to rounding.
* Number of libraries who rank the format 1 to 3.
^ A total percentage is not applicable to multiple response questions.

Question 9: Please rank each method of delivering usage statistics listed below in order of preference on a scale of 1 to 4.

The most preferred delivery method was online, followed by e-mail and mail. The least preferred delivery method was fax. For all delivery methods the range of responses was one, with all libraries ranking online or e-mail first or second, and all ranking fax or mail third or fourth.

The two delivery formats used most frequently by vendors were also the two formats most preferred by librarians. No vendors delivered their usage statistics by post or fax, and no librarians preferred these delivery methods. While a chi-square test found no relationship between the delivery method used by vendors and that preferred by librarians, non-parametric correlation coefficients (Spearmans and Kendalls) were both significant at the P < 0.001 level (zero values included).

Question 10: Please indicate how useful you think each type of data is for informing collection management decisions.

For each type of data provided in vendors' usage statistics, at least one library responded that the data was 'very useful' for informing collection management decisions (Figure 4). The types of data that all libraries found either 'very useful' or 'useful' for informing collection management decisions were 'type of material accessed' and 'usage broken down by title'. Other types of data frequently considered 'very useful' or 'useful' for informing collection management decisions were 'usage broken down by calendar month' (95 per cent of libraries) and 'number of turnaways' (89 per cent of libraries). The type of data that libraries most frequently identified as either 'not very useful' or 'not useful at all' for informing collection management decisions was 'usage broken down by time of day' (39 per cent of libraries).

Figure 4
Usefulness of each type of data for collection management decision-making (N=18)

Table 6 shows the relationship between the type of usage statistics provided by vendors and the type of usage statistics preferred by librarians. The chi-square test found that the relationship between the two variables was not statistically significant.

Table 6
Type of usage statistics provided by vendors (N=16) and preferred by librarians (N=17)

Percentages subject to rounding.
* Number of librarians who responded that the type of usage statistics were 'very useful' or 'useful'.
^ Indicates N=18.
** A total percentage is not applicable to multiple response questions.
Chi-Square = 9.66267
df = 8

Librarians also identified a number of other types of usage statistics they considered useful (Table 7). Vendors currently provided some of these, but not others.

Table 7
Other types of data identified as useful by academic librarians

Percentages subject to rounding.

Analysis of the results

What are the commonalities in the type of usage statistics provided by vendors?

The study found that there was a lack of standardisation in the type of usage statistics provided by different vendors. While one type of usage data (usage by month) was provided by all vendors, the majority were only provided by one vendor, making it difficult to compare the usage of different e-resources. The lack of commonality and standardisation in the type of usage statistics provided also means that some vendors may not provide librarians with the type of usage statistics they need to inform collection management decisions. To address this issue, Hahn and Faulkner's e-journal metrics enable librarians to quantify the value of the e-resource to their users and to enable comparisons across e-resources. [32] However, metrics rely upon the vendors providing the required data.

Is there a relationship between the type of usage statistics provided by vendors and those desired by academic librarians?

Peters found that vendors' usage statistics often serve the vendor's purpose better than the librarians. [33] Luther concurred, finding that vendors try to make the data look as good as possible, as they know it will be used to inform collection management decisions. [34] This study supported these findings; there was no statistically significant relationship between the type of usage statistics provided by vendors and those desired by academic librarians. For example, the 'publication date of material accessed' was the least frequently provided type of data by vendors (13 per cent), but was identified as the fifth most useful out of nine types of data by librarians (72 per cent). Although the causality has not been studied, Luther found that vendors sometimes avoided collecting certain usage statistics to protect user's privacy, and Hahn and Faulkner suggested that an example of this was providing detailed information on specific articles accessed. [35]

Four types of usage statistics were identified by the majority of librarians as either 'very useful' or 'useful' for informing collection management decisions. However, only one of these was provided by the majority of vendors: 'usage by month'. The others were provided by around 50 per cent of vendors. Data suggest that there was more consensus among librarians about the key types of usage statistics than there was among the vendors. Again this suggests that many vendors are not providing all types of data that librarians would find useful.

While the study found a consensus among librarians as to the key types of usage statistics, eleven other types of usage statistic were identified as being useful by only one or two libraries. These were identified in response to a free-text question in the survey, which could not be statistically analysed. However, it does suggest that, while a few key usage statistics are useful to all libraries, librarians' needs do vary, and that the same types of usage statistics do not meet the needs of all librarians. For example, one type of data identified was consortia usage, which would only be relevant to libraries participating in a consortium to purchase the particular e-resource. Another library responded that cost per full text article was useful, but the value of this information would vary depending upon the type of e-resource, and the pricing policy.

What are the commonalities in the format and delivery of usage statistics across vendors?

The study found that the majority of vendors used HTML and CSV to deliver usage statistics, and that no vendors use PDF or Word. It was also common for vendors to provide librarians with a choice of either delivery format. All vendors provided their usage statistics electronically, either online or via e-mail, and no vendors used fax or mail. The obvious advantage of online and e-mail delivery is that they are quicker and more convenient for the vendor to send.

Is there a relationship between the format and delivery method of usage statistics provided by vendors and that desired by academic librarians?

Note that the focus two questions ago was on the type of usage statistics provided by vendors and desired by librarians; this question focuses on format and delivery method, quite different aspects of vendor statistics.

Format
This study did not find a significant relationship between the formats used by vendors and those desired by librarians (Table 5). The formats ranked in the top three most frequently by librarians were Excel, Word and CSV, but the three formats offered most frequently by vendors were HTML, CSV and ASCII.

While data identified a spread in the librarians' preferred delivery format, there was consensus in the reasoning behind their preferences. In concurrence with Shim and McClure's findings, [36] librarians' preferences were based on the ability to manipulate and customise usage statistics, download into Excel, and then use in reports. While CSV allows this, the other two of the three most frequently offered delivery formats (HTML and ASCII) do not. Although CSV appears to meet librarian's needs, it had a mode ranking of four out of six, and eleven per cent of respondents identified it as their least preferred delivery format. This suggests that some librarians were not familiar with CSV and were not aware of its versatility for downloading into different formats, including Excel.

Delivery method
This study found that librarians preferred usage statistics to be provided electronically, either online or via e-mail. These methods were preferred because the data would already be in electronic form ready to be copied or re-formatted into reports, databases and Excel spreadsheets. Electronic delivery is also faster, so the data can be more up to date, factors identified as important by several librarians. Librarians also commented that ability to manipulate and selectively print out the data was important, which would not be possible if usage statistics were mailed or faxed to the library.

One aspect of e-mail delivery that was identified as a problem in the study was when vendors compress a large file of data into a zip file for sending via e-mail. While this can be a convenient way to send a large file, many found opening the zip file difficult. A statistically significant relationship was identified between the delivery method used by vendors and the delivery method preferred by librarians.

Do vendors use the same terminology as each other for their statistics categories?

While 69 per cent of vendors did provide a definition of 'use' to aid librarians in interpreting their usage statistics, there was no common definition across vendors. Some vendors defined it as accessing the homepage of the e-resource, some as downloading the full text article, and others itemised use counts of a number of different activities including searching or accessing table of contents, abstracts or full text.

Do academic librarians understand vendors' terminology accurately when interpreting usage statistics?

There was little variation in the understanding of the term 'use' between librarians. The majority interpreted 'use' broadly and responded that it included any activity within the e-resource from browsing or searching it, to accessing table of contents/abstracts/full text or printing from the e-resource. This suggests that for most librarians, 'use' of an e-resource means active use, not just passively accessing the homepage, which was in concurrence with most of the vendors.

However, while most librarians, and a minority of vendors, understood 'use' of an e-resource to mean one of a broad range of activities within the resource, most vendors defined 'use' very specifically, and their definitions varied. This dichotomy means that many librarians are not interpreting vendors' terminology accurately. However, even if librarians are not interpreting 'use' correctly, if the vendors provide their definition of 'use' librarians should be able to interpret the data accurately. If no definition of 'use' is provided, it is difficult for librarians to compare use of different e-resources.

To what extent do academic librarians employ the usage statistics provided by vendors to inform their collection management decisions?

A number of earlier studies found that vendors' usage statistics facilitate collection management decision-making. [37] This study has confirmed their use by academic librarians in New Zealand. The data show that more than 80 per cent of libraries indicated that usage statistics were used for informing selection decisions, cancellation decisions and funding requests.

Shim and McClure found that librarians were reluctant to employ vendors' usage statistics to compare the use of different e-resources. [38] This study disagrees with their findings, since 76 per cent per of the responding libraries agreed or strongly agreed that statistics were used to compare the usage of different e-resources at their library. The response to this question highlights the importance of vendors supplying standardised usage statistics that can be used to make valid comparisons.

This study found that 65 per cent of libraries use vendors' usage statistics to aid other collection management decisions, such as comparing usage of print and electronic resources, informing selection and continuation decisions of print resources, and judging whether the appropriate number of user licences have been purchased. A number of other uses of statistics were identified, but the study was limited to the employment of usage statistics to inform collection management decisions.

Two libraries responded that they did not employ vendors' usage statistics at all, nor did they access usage statistics. The reason given by one library was that they were charged a flat rate, independent of usage. However, vendors' usage statistics can still be valuable. For example, data on how much the e-resource is used is helpful when deciding whether to renew a subscription. The other library that did not access vendors' usage statistics was planning to access them this year (2004), and intended to compare the use of different e-resources within the COUNTER framework. This highlights the importance of standards to those academic librarians trying to use vendors' usage statistics to inform their collection management decisions.

How could vendors' usage statistics more effectively inform their collection management decisions?

A review of the literature found that vendors' usage statistics are employed for informing collection management decisions. [39] This was confirmed in this study. However, the study also highlighted a number of areas in which vendors' usage statistics could be more useful to librarians.

Standards
Studies by Shim and McClure and Jaguszewski and Probst found that usage statistics provided by different vendors vary widely in their format and terminology. [40] Luther found that the emerging standards for the collection of usage statistics were not being widely adopted by vendors, [41] and several studies found that a lack of standardisation in vendors' usage statistics made it difficult to compare usage of different e-resources. [42] This study confirmed the lack of standardisation and the need for standardisation was a recurring theme in librarians' comments throughout the survey.

The percentage of vendors not complying with standards (69 per cent) makes it difficult for academic librarians to use the data effectively to compare use of different e-resources, and to interpret and use the statistics provided by different vendors effectively. To improve the usefulness of vendors' usage statistics for informing collection management decisions, vendors need to comply with an accepted standard such as COUNTER or ICOLC guidelines. The type of data provided, the layout of the report, the delivery format and the terminology used need to be standardised. The value of statistics provided by vendors who complied with a standard was enhanced by the fact that all complied with the same standard - the COUNTER standard.

Understanding librarians' needs
Peters found that vendors' usage statistics serve the vendor's purpose better than the librarian's. [43] Similarly, Luther's study found that vendors try to make the data look as good as possible since they know it will be used to inform collection management decisions. [44] This study found that there was no relationship between the type of data vendors provide and the type that librarians find useful (Table 6). However, it is not known whether this is because vendors do not understand the needs of librarians and the type of usage statistics they find useful or, if for commercial reasons, they are deliberately not providing the type of usage statistics desired.

To improve the usefulness of vendors' statistics, it is important that they understand the needs of librarians and that they provide data that meet these needs. This includes the type of usage statistics provided and the format, delivery method and the way use is counted.

Delivery format
Librarians' responses identified a clear preference for usage statistics to be delivered in a format that enables them to manipulate the data and download it into reports. Formats that would meet librarians' needs include CSV or Excel, but not HTML, which is currently the most frequently used delivery format.

Customisable usage statistics
There was consensus in the literature that standardisation of the usage statistics was important, [45] and this was confirmed by the findings of this study. However, the study also identified, both explicitly and implicitly, the need for usage statistics to be flexible or customisable because librarians' needs differed.

While there was agreement among librarians about the four most useful types of data, there were eleven types of data identified as useful by only one library (Table 7). This suggests that different libraries have different needs of their usage statistics. For example, some librarians commented on the importance of full data being provided, but others found that most of the data provided by vendors was not useful. It is therefore recommended that vendors provide customisable usage statistics so that librarians can choose the type of data they want to receive. Without customisation, usage statistics will either be providing some libraries with information they are not interested in, or failing to supply all the necessary information.

Glossary
The study found that vendors' definitions of use varied considerably, and that there was no correlation between the definition of 'use' employed by vendors and librarians' understanding of it. In concurrence with Blecic et al this study found that it was important for vendors to provide a glossary of terms employed in the usage statistics so that librarians could interpret the data accurately. [46]

Academic librarians
While standardisation of usage data by vendors is important for improving the usefulness of vendors' usage statistics for informing collection management decisions, there are a number of initiatives that can be taken by librarians:

• let vendors know how their usage statistics could better meet librarians' needs,
• examine the usage statistics monthly to spot variations in use of e-resources or in reporting that usage,
• if a glossary is provided, refer to it so that librarians' interpretation of the report is accurate,
• consider a number of factors that might affect the usage of the e-resource, and
• use locally collected usage statistics to check the reliability of vendors' data and to enable an accurate comparison of the use of different resources.

Conclusion

This study had two main research objectives as described in the problem statement:

• to determine whether academic librarians in New Zealand use vendors' usage statistics to inform their collection management decisions, and
• to determine how the usefulness of vendors' usage statistics for informing collection management decisions could be improved.

This study found that New Zealand academic librarians do use vendors' usage statistics to inform their collection management decisions. However, vendors' usage statistics do not provide academic librarians with all the data that they would find useful to inform their collection management decisions.

This study confirmed the findings of previous studies that the usefulness of vendors' usage statistics for informing collection management decisions could be improved by vendors' standardising their usage statistics by adopting either the COUNTER or ICOLC standards. Standardising the type, format and terminology of usage statistics would enable librarians to interpret and use the statistics provided by different vendors more effectively, and would also enable them to compare the use of different e-resources.

Contrary to earlier studies, this study found that different academic libraries had differing needs from their vendors' usage statistics. This was most apparent in the type of usage statistics provided, where eleven types of data were identified by only one library. Librarians should be able to customise vendors' usage statistics so that they receive the type of data most useful to them.

This study found that the only area where librarians' needs for the provision of vendors' usage statistics were being met, was in the delivery method. All vendors delivered usage statistics electronically via e-mail or online which enabled fast, convenient delivery of up-to-date data to librarians, often in a format they could manipulate for use in reports.

This paper is based on an MLIS research project undertaken by Ms McDowell as part of her degree requirement; Professor Gorman was the supervisor. The article is a joint effort, with the first draft prepared by Ms McDowell, and revised by Professor Gorman.

Notes

1. D W King et al 'Library Economics Metrics Examples of the Comparison of Electronic and Print Collections and Collection Services' Library Trends vol 51 no 3 2003 pp376-400
2. T A Peters 'What's the Use? The Value of e-Resource Usage Statistics' New Library World vol 103 no 1172/1173 2002 pp39-47
3. C Holleman 'Electronic Resources: Are Basic Criteria for the Selection of Materials Changing?' Library Trends vol 48 no 4 2000 pp694-710
4. J M Jaguszewski and L K Probst 'The Impact of Electronic Resources on Serial Cancellations and Remote Storage Decisions in Academic Research Libraries' Library Trends vol 48 no 4 2000 pp799-820
5. Peters op cit
6. Counter: Counting Online Usage of Networked Electronic Resources. http://www.projectcounter.org/about.html 2002 [10 July 2004]; International Coalition of Library Consortia (ICOLC) Guidelines for Statistical Measures of Usage of Web-Based Information Resources 2001 http://www.library.yale.edu/consortia/2001web stats.htm [10 July 2004]
7. L A Knight and K A Lyons-Mitchell 'Measure for Measure: Statistics about Statistics' Information Technology and Libraries vol 20 no 1 2001 pp34-38; J Luther White Paper on Electronic Journal Usage Statistics' 2000 http://www.clir.org [10 July 2004]; Peters Op cit
8. D G Frank, M L Madden and N R Simons 'The Use of Library Statistics by Academic Librarians: Comments on a Significant Problem and Suggestions for Improvement' Georgia Library Quarterly vol 38 no 2 2001 pp5-10; Peters op cit
9. A Kent et al Use of Library Materials: The University of Pittsburgh Study Marcel Dekker New York 1979
10. Jaguszewski and Probst op cit
11. Peters op cit
12. Holleman op cit
13. Jaguszewski and Probst op cit
14. Peters op cit
15. K Rupp-Serrano, S Robbins and D Cain 'Canceling Print Serials in Favour of Electronic: Criteria for Decision Making' Library Collections, Acquisitions and Technical Services 2002 vol 26 no 4 pp369-378
16. Jaguszewski and Probst op cit; Knight and Lyons-Mitchell op cit; W Shim and C R McClure 'Improving Database Vendors' Usage Statistics Reporting Through Collaboration Between Libraries and Vendors' College & Research Libraries 2002 vol 63 no 6 pp499-514
17. Peters op cit
18. Luther op cit
19. Knight and Lyons-Mitchell op cit; Luther op cit; Peters op cit
20. Luther op cit
21. International Coalition of Library Consortia op cit
22. D D Blecic, J B Fiscella and S E Wiberley Jr 'The Measurement of Use of Web-Based Information Resources: An Early Look at Publisher Supplied Data' College & Research Libraries 2001 vol 62 no 9 pp434-453; Knight and Lyons-Mitchell op cit
23. Blecic op cit
24. J Duy and L Vaughan 'Usage Data for Electronic Resources: A Comparison between Locally Collected and Vendor Provided Statistics' The Journal of Academic Librarianship vol 29 no 1 pp16-22
25. Peters op cit
26. Holleman op cit
27. Jaguszewski and Probst op cit
28. R Cheng, S Bischof and A J Nathanson 'Data Collection for User-Oriented Library Services: Wesleyan University Library's Experience' OCLC Systems and Services 2002 vol 18 no 4 pp195-204; Peters op cit
29. Cheng op cit
30. Knight and Lyons-Mitchell op cit; Luther op cit; Peters op cit
31. P L Alreck and R B Settle The Survey Research Handbook: Guidelines and Strategies for Conducting a Survey 2nd ed McGraw-Hill New York 1995
32. K L Hahn and L A Faulkner 'Evaluative Usage-Based Metrics for the Selection of e-Journals' College & Research Libraries vol 63 no 3 pp215-227
33. Peters op cit
34. Luther op cit
35. Blecic op cit; Luther op cit
36. Shim and McClure op cit
37. Frank, Madden and Simons op cit; Knight and Lyons-Mitchell op cit
38. Shim and McClure op cit
39. Frank, Madden and Simons op cit; Knight and Lyons-Mitchell op cit
40. Jaguszewski and Probst op cit; Shim and McClure op cit
41. Luther op cit
42. Knight and Lyons-Mitchell op cit; Luther op cit; Peters op cit
43. Peters op cit
44. Luther op cit
45. Knight and Lyons-Mitchell op cit; Luther op cit; Peters op cit
46. Blecic op cit