AARL

Digital library services: perceptions and expectations of user communities and librarians in a New Zealand academic library

Wei Xia

Abstract: This article gives an overview of research conducted at Victoria University of Wellington regarding the perceptions and expectations of user communities and librarians. It reviews the previous theory in this area and then presents the methodology for this study. This research examines the extent to which users' and librarians' perceptions of the useability of digital services differ. A range of issues and recommendations for ways to improve the digital services are discussed.

The introduction of modern information technology and information resources in a dynamic digital format has forced libraries to make changes to the way that they provide services. As more and more information sources become available online, many libraries are introducing digital services. The aim of these services is to broaden the range of information resources available and to add value to their content by making them accessible through telecommunication networks so that users can access them anytime and anywhere. Academic libraries have changed dramatically over the past decade with the aim of improving performance in their specialised functions and roles. With the aid of modern digital technology and information resources, university students and staff can access their libraries' collections remotely and conveniently to get the information they require.

This research[1] follows the user-driven and value-added theory suggested by Taylor[2] to find the extent of differences in perceptions, expectations and satisfaction relating to digital services and the reasons for these differences with the aim of delivering more satisfactory, convenient and practical services. This research reveals how well users feel their library meets their research needs, what librarians think users' information needs and preferences are, and whether users' preferences are different from librarians' perceptions of those preferences by examining the following hypotheses:

• Different user communities have different opinions on digital services. [3]
• There are relationships between future use of a service and users' satisfaction with the service, and between anticipation of future use of a service and the perceived importance of the service. [4]
• There are different perceptions and expectations of digital services between users and librarians. [5]

Users' information needs and their level of satisfaction with digital services are critical factors for the improvement of services. It is important, therefore, to determine whether users and librarians have different perspectives related to digital information services in terms of satisfaction, expectations and perceptions. The primary purpose of this research was to analyse the problems of digital services in academic libraries, and put forward recommendations for better ways to deliver them in order to increase use. Unlike previous research, this study attempted to provide a balanced perspective by investigating both users and librarians.

Literature review

In the 1980s, library and information science researchers began to pay more attention to the types of electronic services provided by libraries. Fayen,[6] Cline,[7] and Dowlin[8] discussed new ideas such as online catalogues, databases, electronic messaging systems, and community bulletin boards etc. Most of the research in the early 1980s was technology-driven or content-driven. After Taylor's user-centred and value-added theory was introduced, researchers began to pay more attention to information use. Taylor emphasised that the main input into the design of information systems must come from an analysis of information use. According to Taylor, the technology-driven model had essentially prescribed the size, shape, dynamics and even the content of information systems, while the content-driven model was derived from traditional classifications of knowledge. These two models were no longer enough to deal with the information explosion, therefore Taylor argued that research on the delivery of electronic services should be user-oriented. This point of view suggests that surveys of users and library staff can be a basic foundation for research on the performance of library services, since users are the best judges of service quality.

Other theories are also needed to evaluate digital services thoroughly. The Gap Model of Parasuraman[9] is another key theory, an approach that has been used to examine the differences between users' expectations of service quality and their perceptions of the actual quality. Five gaps are put forward in the original theory in terms of customers' expectations, management's perceptions and service quality specifications. On the basis of this Gap Model, SERVQUAL and User Satisfaction are two standardised instruments that have been widely used in a range of situations.

In the 1990s, research on and development of digital library services increased. Some of the research on academic libraries became more user-oriented. For years the professional literature had been warning academic libraries and librarians of the need to change and the danger of lagging behind users' information needs. [10] Kyrillidou[11] has provided compelling reasons for evaluating libraries based on users' feedback.

At the start of the new millennium, Fountain[12] observed that academic libraries were now turning their attention towards implementing and offering users a range of innovative ways to access information due to the numerous advantages of digital services. Choudhury et al[13] demonstrated an increasing emphasis on a user-centric perspective when evaluating digital services. According to him, decision-makers need to consider insights and perspectives regarding user preferences from a variety of sources to ensure that future digital services serve the needs of that community.

Though there is a growing amount of research on digital services, many other researchers have not understood that data gathered from the perspective of users and library staff are a necessary part of service analysis. Moreover, almost no research appears to have been done to investigate the differences between users and librarians in their understanding and assessment of digital services. As well as gaps between the perceptions and expectations of users, there may be gaps between those of the users and library staff. Ignoring users' feedback leads to a disproportionate reliance on technology-centred or content-centred theory. This lack of knowledge can make it difficult for libraries to improve their services and develop new services to meet users' needs. University libraries need to identify and understand existing problems and appreciate the potential of digital services. Although there are several good sources offering evaluation measures, such as Crawford,[14] Bertot[15] etc, few of them are entirely appropriate for evaluating digital services in academic libraries without any revision. The variety of theories and models used in previous work needs to be integrated into a unified measurement framework.

It is this lack of research into users' and librarians' opinions of digital services that attracted the researcher's attention and ultimately inspired this research. Bridging these gaps between users and librarians overcomes the drawbacks of previous research based on technology-driven or content-driven models of library services. By adopting a user-centred approach, this study aims to promote a greater understanding of the existing and potential digital services provided by academic libraries to enhance information use.

Methodology

The methodology used in this research includes both qualitative and quantitative aspects. A proportional stratified sample was used to select users in the categories of undergraduates, postgraduates, and academic staff, while simple random sampling was used to select librarians. The librarians were selected on the basis that they were involved in implementing and planning digital services or their work was affected by digital services. A questionnaire was used to survey users and librarians at Victoria University of Wellington (VUW) Library to determine any differences in their preferences for the delivery of digital services and in the information needs of the different user communities. The measures used in this research revise those measures used in previous models (eg Cameron[16]) and determine the value and importance of the features of specific digital services from the perspective of the users. The open-ended questions asked the respondents to express their views and their experiences of using digital services. The closed questions used multiple choice and Likert scales (five responses ranging from negative to positive in relation to the question statement). The questionnaires were distributed to 495 users[17] and 38 librarians by mail or personal delivery and 231[18] and 13 responses were received from users and librarians respectively. The qualitative method took the form of interviews, which allowed the researcher to get in-depth information from the librarians to investigate reasons for any differences between users' and librarians' preferences.

Findings

This study of different preferences for digital services among users and librarians highlights the importance for academic libraries to address the specific information needs and preferences of their different user communities. Some details of the survey results can be found in Tables 1-3.

Table 1

Frequency of use of current services

Different user communities have different opinions on digital services

When the responses from undergraduates, postgraduates and academic staff were compared, there were some noticeable differences.

Table 2

Assessment of current digital services (mean rating)

The results demonstrated that undergraduates, postgraduates and academic staff had different purposes for using information services based on their information needs. This finding confirms the hypothesis that different user communities have different needs for information, demonstrating that when organising some services, libraries need to consider all kinds of information needs, especially the key beneficiaries' information needs. For services that are mainly used by one community, librarians need to focus on the special information needs and behaviours of this community to avoid wasting money and other resources.

Table 3

Assessment of future digital services (mean rating)

Note: 5-strongly agree; 4-agree; 3-neutral; 2-disagree; 1-strongly disagree

There were differences in the perceptions held by different user communities of current digital services in terms of frequency (Table 1), importance, usefulness, and future use (Tables 2 and 3). Generally, more academic staff than students were aware of the digital services, which is reflected in the staff's frequent use of digital services (see Table 1). Similarly, academic staff considered the services more important than did other user communities. Some services, such as e-mail reference and online request forms, had never been used by more than 70% of undergraduates. It is important to note that few students know and use these digital services. The ratings for the importance of the features of digital services varied across different communities. Overall, 'convenient and easy access', 'easy to use' and 'useful information' were the most important features identified by users. Postgraduates and academic staff preferred advanced features of digital services, for example 'the range of resources', 'currency of information', etc, and they intended to make full use of the services. Undergraduates, however, emphasised access issues. 'Accuracy of the information' was another important feature for undergraduates. The five options given for possible improvements to digital services highlight the demand for information services from different communities. The Online Public Access Catalogue (OPAC) was the highest priority for improvement from the perspectives of all the user groups. e-mail reference was given a high priority by undergraduates, while postgraduates and the academic staff selected e-journals and electronic document delivery.

The hypothesis that different user communities are not equally satisfied with the digital services was confirmed. Those services with fewer users got the lowest satisfaction. For academic staff and undergraduates, the most satisfactory services were the OPAC and databases. Postgraduates, as key beneficiaries and frequent users of e-journals, were more satisfied with them.

The expectations of users are reflected in the ratings given to the three future services (electronic closed reserve, virtual reference, and current awareness services; see Table 3) and some of their comments on the digital services, which confirmed the hypothesis that different user communities have different expectations of future digital services. Current awareness services (CAS), which can provide broad access to information, were given high ratings by academic staff in terms of importance, usefulness and future use.

Relationships found

This study also found that strong correlations exist both between users saying that they would use a service in the future and the importance they attributed to that service, and between users saying that they would use a service in the future and their satisfaction with that service.

The more important the service is perceived, the more likely it is that users will use this service in the future. The strongest relationship occurred in online request forms and e-mail reference services (correlation coefficients were .855 and .873 respectively), and the weakest was for the OPAC and databases (correlation coefficients were .544 and .656 respectively). [19] The motivation for using digital services stems from users' information needs. As long as users can retrieve useful information through this service, they intend to use it again and consider it important. For undergraduates, this relationship was slightly weaker than that for postgraduates and academic staff. Postgraduates and academic staff preferred the usefulness of information for their work to other factors, and this influenced their decisions to use the services again in the future. Libraries can still take some remedial actions to improve the usefulness of these services and influence the users' perceptions of the services' importance for information retrieval. By doing this libraries should increase the likelihood that users will use the service again in the future.

The statistical analysis showed that there was also a strong positive relationship between satisfaction and future use of services except for the OPAC. Users' dissatisfaction with the OPAC did not affect their intention to use the service again. This anomaly probably stems from the users' dependence on the OPAC as a key resource for their learning, teaching and research needs. In fact, users have no choice but to use the OPAC, so even when they are dissatisfied with it, they still intend to use it in the future.

The relationship between satisfaction and future use of digital services among undergraduates was more obvious than for postgraduates and academic staff. The willingness of undergraduates to use the service in the future depended more on their satisfaction with the services and their current experiences of using these services. If some services are not satisfactory, undergraduates prefer not to try harder to use them, and they tend to explore alternatives. Satisfaction with the services depends on quality. In order to improve users' satisfaction, improvements can be made in the features considered important by users.

Differing perceptions and expectations between users and librarians

This study also found that differences exist between the users' and the librarians' perceptions and expectations related to the digital services.

The hypothesis was confirmed that users are not aware of the existence of digital services to the same extent as the librarians assumed. The users' awareness of services impacts on the frequency of use of these services. The OPAC and databases were used more frequently than other services. Postgraduates and academic staff used services such as e-journals and online request forms more than undergraduates. In fact, the librarians were not aware that users felt there was a lack of information about some digital services. Users' and librarians' answers showed that there was a need for more promotion of the digital services to improve awareness of the services and how to use them.

Users had different perceptions from those of librarians on the current delivery of the digital services (refer to Table 2). The digital services considered important by users differed slightly from those favoured by librarians. Users used some services infrequently or never, and therefore had not experienced the benefits of these services. This inconsistency may suggest that, although the librarians thought that provision of digital services was responding to the real needs of the users, this might not be the case. Another issue was the preference for various delivery forms of search results. Users preferred printing them out while librarians had some reservations about this, though they recognised that it was convenient for both users and librarians. When selecting and rating the five services that should be improved, both users and librarians agreed that the OPAC was the first priority. Databases were a higher priority for improvement than e-journals according to all user groups. This finding may suggest that databases are troublesome services for users, and that these difficulties are not being addressed by the librarians.

As for satisfaction level, there were noticeable variations, which confirmed the hypothesis that the percentage of users who are satisfied with the performance of the digital services was different from that of librarians. Users were not satisfied with some services such as databases and online request forms, while librarians thought these services had a satisfactory performance.

Users' expectations of the importance and usefulness of some future digital services differed from the expectations of librarians. Users and librarians had significantly different opinions regarding CAS. Users, especially academic staff, gave high ratings to its importance and usefulness, and they supported the implementation of this service. Librarians gave relatively low ratings, feeling that although CAS was helpful, it was not worthwhile for an academic library to invest time in this and it was better to direct energy into user education and patron empowerment. All the librarians agreed that virtual reference was a good service, especially for external students and for those universities with several campuses and few reference librarians. However, some overnight shiftwork is involved in providing nonstop reference services. It is a demanding service because it requires powerful machines and considerable bandwidth. One interviewee commented that it was also difficult to ensure congruence between the users' expression of their questions and their actual problems, and librarians' understanding.

Since the objective of academic libraries is to serve all user communities, university libraries should consider how to deliver satisfactory services for the range of users they serve. Some recommendations are made below for improving the services to meet the identified information needs and the preferences of different user communities.

Issues and recommendations

The main objective of this research was to make recommendations for the improvement of digital library services. A number of approaches could be taken to help bridge the gaps of preferences, perceptions and expectations between users and librarians. Before setting up any digital services, libraries should identify the needs and preferences of the user communities. There has been some discussion of the necessity to better understand and define the needs and expectations of library users in order to provide appropriate kinds and levels of service. The basic principle should be that the delivery of digital library services must satisfy users' needs.

Awareness of the services

It is important to improve the users' awareness of the services because this will determine whether users can make full use of digital services. As found in this study, unfamiliarity with digital services results in low frequency of use and dissatisfaction. The level of use is influenced heavily by users' perceived importance of the services, their satisfaction with the services, and their familiarity with the library and its resources. Those who think more highly of the digital services and are more familiar with the library are more likely to use digital services frequently. If usage is to be increased, it is important that libraries find ways to familiarise users with digital services. Promotion of these services needs to be addressed.

Porter,[20] similarly, said that the library needed to be more proactive in publicising its services to the user communities to improve users' awareness. Another recommendation made by all the interviewees was for university libraries to consider setting up tutorials for all user communities, especially new students. One interviewee recommended encouraging people to use the resources more and to stimulate their interest in learning how to use the services by distributing information packages and introducing library online information services. Liaison librarians could provide one-on-one contact services for users and maintain a frequent users list in order to introduce new services and information to those users. Another recommendation was for librarians to go to lecture sessions, tutorials or meetings to teach students and academic staff how to use online resources.

Access

Access to resources can be considered a vital aspect of evaluating satisfaction with a library's services. As the findings from the user survey show, access is one of the important factors influencing the useability of digital services. It is important for academic librarians to address the needs of their user communities by networking the library's services and resources so users can easily access them anywhere and at any time.

Easy access applies not only to the networked resource environment but also to the interface to the resources and the reliability of the relevant hardware in the library (ie, computers and servers). Ferguson and Bunge[21] suggested that libraries needed to provide direct access to their electronic resources, to ensure that resources and services can be reached quickly via a minimal number of links from the first page of the library website and they should appear under obvious (to users, not librarians) headings or categories.

Currency of Information

Responses from the survey showed that some users addressed the importance of the currency of information on the website. Libraries have a responsibility to keep their own details and the information about their resources up-to-date. Outdated information on the website makes a negative impression on users. Libraries need to make a concerted effort to keep digital information up-to-date. They should also collect users' feedback on online services and report the problems (eg broken links) to service providers. Meanwhile, students should be encouraged to assist libraries by reporting to the librarians any errors and broken links in the digital resources as soon as they find them.

OPAC

The OPAC received many negative responses from the users surveyed. According to Wakeling,[22] when OPACs are moved to a web base, libraries can add selected Internet resources to the catalogue to extend the reach of OPACs. University libraries can provide direct links from OPACs to a range of online resources. When the resources are held or subscribed to by the library, users can go directly to the website instead of copying and pasting the URL into a browser. If the resources are not held by the library, users can utilise the interlibrary loan service by clicking the relevant links. It is possible through the OPAC for university libraries to export search results directly into programs such as Endnote. This feature allows users easily to save bibliographic information into a format that will allow further manipulation.

Issues with databases

According to the comments given by the users and the libraries, databases are important and also troublesome services for users. Liaison librarians can help users choose relevant databases on the subject pages of the library website by dealing with specific areas and telling users which databases are best for them.

The survey data also show that users have difficulties in defining the terms they wish to search for or in finding useful search terms. Tutorials or similar teaching sessions could be a good way to teach information searching skills. Lecturers could be encouraged to explain to their students in lectures or tutorials how to use databases related to their assignments. It is a good idea to set up online help on the website covering frequently asked questions and some common topics for databases as it is convenient for users to refer to such aids while searching for information.

Issues of e-journals

The issues associated with e-journals were generalised into several points from the comments of users surveyed and librarians interviewed. The first important issue, which reflects the concerns of users, is the somewhat limited range of e-journals. The high cost of e-journals is one reason for this. Another reason for the limited range of e-journals is that libraries cannot select individual e-journals because most of them come within databases, or with tied subscriptions,[23] or as part of a package. Cost and demand are the main considerations for libraries in deciding to which e-journals they should subscribe. Consortium arrangements play a significant role in electronic resources, especially databases and e-journals.

The second issue mentioned by users was making sure it is easy to find which e-journals are available both on site and off site. E-journal information should be incorporated into the web-based catalogue to provide clear and direct access.

The third issue identified by the interviewees was the users' skills in searching for e-journals. Based on comments made by some of the librarians interviewed, it is important to bear in mind that users' searching skills influence their search results. Several interviewees recognised that it was important to make the users familiar with e-journals, and that library information sheets can tell users how to make full and efficient use of this resource.

Future services

Some different opinions and issues are raised in the application of several possible future services from the survey data. It is not advisable for academic libraries to implement virtual reference service. One interviewee pointed out that virtual reference entails high costs, powerful machines and librarians who are constantly available, so it may not be an efficient use of resources. He also suggested that it was much easier to telephone or e-mail users to answer their questions.

Fountain[24] stated that paper-based reserve materials are a nightmare both for the circulation staff trying to regulate use, and for the frustrated student who is unable to check out needed materials. He recommended electronic reserve software by which users could locate materials, read a full-text copy, save the item to their disk, or print out a hard copy. Libraries should give the online reservation service for closed reserve materials a prominent place in the OPAC with detailed booking information. Libraries should seek consortium involvement for the acquisition of electronic reserve software and the resolution of copyright issues to provide full-text reserve materials online.

CAS is probably useful but all the interviewees thought that it was not worthwhile for an academic library to invest time in it because this service requires too much money, time and energy. The business of an academic library, according to one interviewee, is to tell people how to find materials rather than always giving it to them. Therefore, CAS is not consistent with the purpose of information services offered by academic libraries. An alternative service a library could provide might be to notify liaison librarians when new materials arrive so they, in turn, can pass the information to their patrons working in those areas.

Conclusions

Overall, there are some positive factors influencing academic libraries to adopt digital services. A big incentive for adopting digital services and increasing their funding is that users are now expecting to be provided with digital services. The second reason for adopting digital services is that it is also more economic to provide online resources rather than to buy hardcopy materials, though this view is controversial. The main advantage of digital services is accessibility for all students and staff, especially distance students.

Another important factor is that university libraries can cooperate nationally or with universities overseas in the purchase of online resources. One incentive for this cooperation is that resources such as electronic databases are so expensive that they only become affordable by sharing costs. A second incentive is that sometimes the number of users in one university is too small to justify the subscription. Libraries can get the best resources for the least cost by joining consortia.

On the other hand, some negative factors that prevent libraries from adopting digital services are worth considering. Funding is a major problem that constrains libraries from improving current services and adopting new services, a view that was repeated by all the interviewees. It is necessary for libraries to investigate their users' information needs and to decide which online resources and services need to be provided.

Some other obstacles in implementing digital services were also mentioned by the interviewees. Whether some resources are suitable to appear online was the first issue to be considered. It is also necessary for libraries to keep a reasonable balance between paper-based resources and online resources based on users' information needs. Another issue is that libraries do not have enough qualified staff to serve users. Libraries need to employ better staff (and improve the skills and qualification levels of existing staff) in order to raise the level of professional services to their users.

The field of digital services experiences constant and rapid changes. Looking ahead, digital services will affect libraries' work processes and librarians' jobs. Libraries have a responsibility to accommodate all users' needs and should provide the necessary help to support users' information needs. Each library must also ensure that users are given opportunities to acquire information retrieval skills and to practice them throughout their years of study. Meanwhile, students need to take the necessary initiative to improve their competence in using libraries.

Acknowledgment

I would like to express my gratitude to Dr Dan Dorner, my thesis supervisor who helped me with the research and this article.

Notes

1. Wei Xia 'Digital Library Services: Perceptions and Expectations of User Communities and Librarians in a New Zealand Academic Library' unpublished MA in Library and Information Studies Victoria University of Wellington 2002
2. R S Taylor Value-added Process in Information Systems Norwood NJ Ablex 1986
3. This hypothesis can be divided into following several sub hypotheses:
   • Different user communities have different information needs.
   • Different user communities have different perceptions of the current digital services in terms of awareness, frequency of use, importance, ease of use, usefulness, accuracy and future use.
   • The different user communities are not equally satisfied with the digital services.
   • Different user communities have different expectations related to future digital services.
4. This hypothesis can be divided into these two sub hypotheses:
   • The more important the service is to the users, the more likely it is that users will use the service in the future.
   • The more satisfied with the service the users are, the more likely it is that they will use the service in the future.
5. This hypothesis comprises the following sub hypotheses:
   • The level of users' awareness of some specific digital
   • services will be lower than library staff thought.
   • Users have different perceptions from those of
   • librarians on the current digital services.
   • The percentage of users who are satisfied with the performance
   • of digital services is different from that expected by librarians.
   • Users' expectations of the importance and usefulness of
   • future digital services differ from librarians' expectations.
6. E G Fayen 'The Electronic Library in an Academic Setting' In M Koenig (ed) Managing the Electronic Library: Papers of the 1982 Conference of the Library Management Division of Special Libraries Association (pp9-13) New York Special Libraries Association 1983
7. H F Cline & L T Sinnott The Electronic Library Toronto Lexington Books 1983
8. K F Dowlin The Electronic Library New York Neal-Schuman 1984
9. A Parasuraman, V A Zeithaml & L L Berry ' SERVQUAL: A Multiple-item Scale for Measuring Consumer Perceptions of Service Quality' Journal of Retailing vol 64 no 1 1988 pp12-40
10. T Hanson 'Managing Reference and Information Services' In T Hanson & J Day (eds) Managing the Electronic Library (pp335-336) East Grinstead Bowker-Saur 1999
11. M Kyrillidou & F M Heath 'Introduction' Library Trends vol 49 no 4 2001 pp541-547
12. L M Fountain 'Trends in Web-Based Services in Academic Library' In P D Fletcher & J C Bertot (eds) World Libraries on the Information Superhighway: Preparing for the Challenges of the New Millennium (pp80-93) Hershey PA Idea Group Publishing 2000
13. S Choudhury, B Hobbs, M Lorie & N Flores 'A Framework for Evaluating Digital Library Services' D-Lib Magazine vol 8 no 7/8 2002 http://www.dlib.org/dlib/july02/choudhury.07choudhury.html
14. J Crawford Evaluation of Library and Information Services London Aslib 1996
15. J C Bertot 'Measuring Services Quality in the Networked Environment: Approaches and Considerations' Library Trends vol 49 no 4 2001 pp758-774
16. K Cameron 'Domains of Organizational Effectiveness in Colleges and Universities' Academy of Management Journal no 24 1981 pp25-27
17. Undergraduates: 207; postgraduates: 250; academic staff: 38
18. Undergraduates: 136; postgraduates: 76; academic staff: 19
19. The correlation coefficient of both relationships is significant at .01
20. S Porter 'Into the Future: Scholarly Needs, Current Provision, and Future Directions' The New Review of Academic Librarianship no 4 1998 pp190-214
21. C D Ferguson & C A Bunge 'The Shape of Services to Come: Values-Based Reference Service for the largely Digital Library' College & Research Libraries vol 58 no 3 1997 pp252-263
22. W Wakeling 'Managing Technical Services in the Electronic Library: Modernisation before Transformation' In T Hanson & J Day (eds) Managing the Electronic Library (pp445-465) East Grinstead Bowker Saur 1999
23. Some e-journals are part of a print subscription (free-with-print or tied-subscriptions). A library subscribes to a print journal and as part of the package it gets the online version free. If the library cancels the subscription to the print journals, it will lose access to the e-journals even though it has paid for the past issues.
24. L M Fountain 'Trends...'

Wei Xia, Master of Arts in Library and Information Studies, Victoria University of Wellington. gracie_xia@hotmail.com.nospam (please remove '.nospam' from address).