 home > publishing > aarl > 32.1 > full.text > article |
|||
Working Towards Best Practice in Australian University Libraries: Reflections on a National ProjectLeeanne Pitman, Isabella Trahn and Anne WilsonAbstract 'Best Practice for Australian University Libraries'was a federally funded project under the Commonwealth Department of Education, Training and Youth Affairs Evaluations and Investigations Program (EIP). The aim was to investigate best practice activities in Australian academic libraries. Reference was also made to relevant best practice activities in selected overseas countries. Best practice activities within Australian academic libraries were considered to encompass the extent of implementation of quality frameworks, the use of benchmarking and performance measurement as tools for the continuous improvement of products, processes and services, and the development of staff competencies and training required for these activities. A Council of Australian University Librarians Working Group is progressing the recommendations of the EIP project report 'Guidelines for the Application of Best Practice in Australian University Libraries: Intranational and International Benchmarks'and a parallel 'Best Practice Handbook for Australian University Libraries'has been published. EIP project reports are published in hardcopy and widely distributed and also made available on the Department of Education, Training and Youth Affairs website.[1] Members of the investigating team reflect on the lessons. The Council of Australian University Librarians (CAUL) has been concerned for some time to facilitate access by Australian university libraries to information which would assist them with the implementation of best practice initiatives. Funding from the Evaluations and Investigations Program (EIP) of the Department of Education, Training and Youth Affairs (DETYA) during 1998 provided an opportunity to fast track those aims. With the publication of the report, 'Guidelines for the Application of Best Practice in Australian University Libraries: Intranational and International Benchmarks' [2] and the accompanying 'Best Practice Handbook for Australian University Libraries',[3] and also in view of the fact that a broad overview of the project work has already been given to the profession overseas[4] it is timely to provide details of the project to the profession within Australia. There is, in addition, an obligation to the members of the Australian university library community who contributed freely to data collection and to numerous individuals who have expressed on-going interest in the outcomes. The report of the project, which had the official short title of 'Best practice for Australian university libraries', provides a snapshot of best practice activities in university and research libraries and suggests ways in which this picture could develop in the future. Much information is collated, particularly in the accompanying handbook, but the question arises as to how this work can help to provide better tools for university libraries to demonstrate accountability, quality outcomes and effective performance. GeneralitiesThere were some broad characteristics of the investigation environment which are worth noting:
ContentThe investigation encompassed an examination of the selection and implementation of quality frameworks and the extent of benchmarking and performance measurement activity. Comparisons with selected overseas institutions in the form of brief case studies were also undertaken. Through the data gathered, directly, from a series of surveys and site visits and indirectly from the literature an analysis of best practice activity in Australia and, to some extent, in comparable overseas countries was possible. Key issues included:
TerminologyInternational literature supports the difficulty of total agreement on terms related to best practice activities, but stresses the importance of finding and using an accepted/endorsed definition. It also discusses the diversion that discomfort around the use of the terminology brings in many university libraries, particularly when first encountered or initiated. Garrod and Kinnell state: The problem surrounding the identification of a definition of benchmarking highlights the whole problematic area of 'quality'terminology. Many LIS managers baulk at the use of jargon and view it as cant. It is often regarded as the way in which professed 'experts'describe what is basic common sense and good practice. Jargon tends to alienate the uninitiated by combining the elements of exclusion and exclusivity. There is therefore a need to find vocabulary to describe 'quality'methods and practices that is acceptable to the majority of the LIS profession.[7] Once terms become more widely used, however, they might become more acceptable. For example, the parameters adopted in this project were primarily based on the simple 1996 International Federation of Library Associations (IFLA) definition that performance measurement compares 'what a library is doing (performance) with what it is meant to do (mission) and wants to achieve (goals)'.[8] Evaluation should relate to appropriateness, effectiveness and efficiency. InputAs the result of looking for insight into the current level and type of activity within academic and research libraries and information services, most source material in the report and handbook, with the exception of some key resources, was focused on literature published over the last five years. A select few important historical sources were identified for the dedicated. Three surveys were developed and distributed to all Australian (38) and New Zealand (6) university libraries and the major Australian non-academic research libraries (State Libraries, National Library of Australia and CSIRO). The surveys aimed to identify best practice activities being undertaken within the sector and focused on the identification of quality, benchmarking, and performance measurement implementation and self-identified areas of best practice. On the basis of the survey findings, a series of site visits by members of the project team were undertaken to those libraries identified as exemplars of interesting quality, performance measurement implementation and best practice activity. The visits allowed for the exploration and clarification of issues raised in the survey findings, and enabled the documentation of best practice activities within these libraries which was either transferable to other sites or appropriate for inclusion in the handbook. The aim of developing the handbook was to provide busy practitioners with ready access to appropriate information. An unrelated series of visits to a number of library and university sites in the United Kingdom, Europe and the United States in 1998 by a project team member provided some direct information to supplement the published literature in the areas of the development and use of traditional and electronic library and information services performance indicators in Britain and Europe, British and European library benchmarking, and the applicability of quality frameworks to academic library management (United Kingdom and North American examples). FindingsLiterature ReviewGenerally, there is a lack of material actually published in Australia on university library experiences in quality management, benchmarking and performance measurement. This is despite evidence from the direct investigations undertaken by the project that many university libraries are actively engaged in quality management, have undertaken or intend to undertake benchmarking projects, and make great use of performance measurement as a tool for continuous improvement. Relevant Australian library conference papers form a very small subset of current professional conference programs and publications. In the university library context it appears that the trend is to do, rather than show and tell in relation to best practice. Anecdotal evidence suggests that Australian librarians look to the business oriented quality bodies within Australia, such as the Australian Quality Council,[9] for keeping up to date with the literature. Librarians have thus, to date, been forced to glean what they can from international sources on library work. The international literature has provided particularly illuminating data with regard to electronic sources as well as some useful case studies and an indication of the trends and paths being pursued internationally by bodies such as the International Standards Organization (ISO)[10] and IFLA. Whilst a number of Australian university libraries are well versed in the literature relating to work done by ISO and IFLA, there was little evidence of reference to the literature generated by the European Union sponsored projects and Standing Conference of National and University Libraries (SCONUL)[11] activity in Britain. BenchmarkingThe relevant literature, survey responses and site visits indicate that benchmarking is regarded as a useful and appropriate tool for improving products, processes and services. It also indicates that what has been largely perceived in the past as a tool for business, industry or profit oriented organisations can be used effectively in the service sector, and in particular in academic libraries. Almost half of the Australian university libraries responding to the project surveys reported involvement in formal benchmarking exercises over the past few years. One factor in the growing level of acceptance of benchmarking as a tool may be that there has always been in the libraries at least, a strong tradition of informal co-operative surveying and exchange and aggregation of data. This is demonstrated through the activities of CAUL and is captured formally through instruments such as the CAUL surveys.[12] Most Australian university librarians utilise for planning and information purposes the files of round robin surveys on topics of widespread interest. These could be as simple as 'What are you doing about?'or as complex as a comprehensive survey requiring specific data for return and standardised compilation. This type of activity supplements the Australian Academic & Research Library (AARL) statistics (Australia's version of the United States Association of Research Libraries [ARL] statistics). The AARL statistics are approaching their fiftieth year of collection and now cover a substantial time series of those standard quantitative indicators used in the past as 'benchmarks'. Access to these statistics is available through the CAUL web-site for more recent years, but there is still much developmental potential for this site.[13] The ready adoption of benchmarking within universities as a whole is reflected in the Australian Vice-Chancellors'Committee (AVCC) bench-marking project which established agreed benchmarks across the whole spectrum of university enterprise. One of a number of working parties involved in the work developed the proposals for areas relating to libraries and information services provision. Some university libraries have since begun actively reviewing and building relevant so-called McKinnon benchmarks into annual library performance reviews and some universities are looking to incorporate the benchmarks from the wider project across the whole university enterprise.[14] The report also found that a number of benchmarking methodologies have been utilised and that choice may depend on institutional goals and objectives, the organisation's size and structure and the type of process identified for benchmarking. What did the libraries benchmark? In depth process analyses with other university libraries included:
Common instruments were used in planned timetables with other university libraries to extract comparable data with the intention of discussing possible process improvement in the following areas:
The following areas have been benchmarked with organisations other than libraries either in addition to libraries or as the sole library in a benchmarking network:
Australian university libraries are also notable for seeking out distant benchmarking partners in a way that might seem incomprehensible in academic libraries in more compact nations. The first steps in international partnering have been taken through membership and participation in activities promoted by the Association of Commonwealth Universities Commonwealth Higher Education Management Service (CHEMS) Benchmarking Club[15] and the Universitas 21 global alliance.[16] International benchmarking presents particular challenges and the experience of Australian partners in this arena is probably fairly described as substantial but pioneering. Informal benchmarking exercises reported included:
Formal benchmarking for some was regarded as informal benchmarking for others. The apparent focus in the United States by the large commercial organisations on broad picture industry metrics rather than more detailed process benchmarking to improve specific practices is not as evident amongst commercial organisations in Australia. The broader Australian benchmarking scene, beyond the libraries, which currently involves many large Australian companies and former public utilities, is dominated by the Australian Quality Council (AQC). The AQC promotes national benchmarking initiatives through its benchmarking networks, and the alignment of information analysis using the national quality award framework, formerly called and still popularly known as the Australian Quality Awards.[17] This framework has a broad similarity to the European Quality Award, sponsored by the European Foundation for Quality Management (EFQM)[18] and the US (Malcolm Baldrige)[19] national quality award framework. There are some supporters of the Australian framework who claim it is both older than and superior to its big brothers. The AQC benchmarking networks go to considerable expense to provide the opportunity for fairly detailed examination of the nominated processes so that participation in a network involves both data gathering and analysis and fairly comprehensive site visits. Some small commercial companies offering data analysis and questionnaire design services have also been drawn into using the Australian Business Excellence Framework (ABEF) for their client base so that benchmarking between libraries and between libraries and commercial organisations is becoming easier with the sharing of a common approach. In the United States academic library sector, it appears that, in many large institutions there is a focus on the desire to protect the integrity of the institution and retain competitive advantage. This is far less evident in Australia. This self-contained focus gives rise to a relative lack of expressed interest in precise initiatives such as process benchmarking and a public preference for less formal activities such as the exchange of views and cooperation on new initiatives or joint lobbying for a greater good. Informal benchmarking seems to be the preferred approach also in the United Kingdom. It would also seem that it is the new universities, keen for a competitive advantage, who are the key players. This sort of pattern is not generally the case in Australia. In Europe and the United Kingdom the growth of regional links is opening up more opportunities for cooperative activities between academic and other libraries within the local area. Benchmarking is seen as an approach which may have particular relevance to the regional concept, but this regionalism is also not so typical of the Australian approach. The best practice project found that Australian benchmarking partners are usually a mixture of newer and older institutions, with some groupings including libraries as far apart as it is possible to be on the Australian continent. Willingness appears to be the key criterion. Being near neighbours in Australia is sometimes a specific criterion for not considering benchmarking. It is perhaps significant that the two national benchmarking projects undertaken and proposed for the interlibrary loan / document delivery area have come out of the United States and Australia rather than out of Europe, and that nationwide benchmarking exercises, even those restricted just to university libraries, have emerged only from the very compact European nations such as Holland and Denmark. The cultural, political and historical barriers which may impede progress in benchmarking across nations are not issues for benchmarking within Australia but are issues of great relevance in preparing Australian university libraries for the international benchmarking that new alliances are bringing. As a sovereign nation with a tendency for national uniformity and comparatively low levels of suspicion about the disclosure of information about our institutions we still have much to learn in the global context. For the four Australian university libraries involved in the Universitas 21 grouping of libraries and the other group of university libraries who have been involved in the CHEMS library benchmarking project appropriate cultural, political and historical sensitivities are being developed. Performance MeasurementPerformance measurement is universally practised as part of the management process within Australian university libraries. A wide range of performance indicators are being used for this purpose including those specifically developed by CAUL. The three published CAUL indicators are, in order of frequency of use:
Few libraries use all three indicators. A substantial minority use the materials availability indicator annually and somewhat fewer the client satisfaction indicator. As regards client satisfaction the libraries have adapted and adopted a range of well known approaches tailored to need including instruments based on the work of Van House,[20] Hernon and Altman[21] and Parasuraman.[22] CAUL has also visited the question of improvement of these indicators and national preferences for further development. Support has been given by CAUL over the past couple of years for the Working Party on Performance Measures for Reference Services, from the Victorian-based Cooperative Action for Victorian Academic Libraries (CAVAL) Reference and Information Group (CRIG) which has published a final report. This report recommends twelve indicators in three broad dimensional groupings called ASK (attributes, support, knowledge). Since reference is one area where such work is scarce the further evolution of the CRIG model will be a matter of wide interest.[23] Much effort has also been expended in university libraries around Australia on the development of in-house indicators and the modification or adaptation of other externally published indicators (ISO 11620, SERVQUAL, IFLA). The overriding motivation appears to be to develop indicators which relate closely to strategic plans and key result areas. Australia has been slower in relation to the identification and development of performance indicators for the electronic library despite some interesting speculative work in some individual institutions. Some libraries mentioned that they were following the work of McClure and Lopata[24] and Brophy and Wynne.[25] Interestingly, as part of the study, the highest rated responses to priority areas for indicator work were user satisfaction, electronic services, availability of electronic services, document delivery and quality of electronic services. The project also found that, almost universally, those university libraries actively evaluating their services were doing so as an integrated part of planning or as part of upgrading continuous improvement processes. There was no evidence of awareness of the European Union sponsored projects such as EQLIPSE (Evaluation and Quality in Library Performance: System for Europe) coordinated by the University of Central Lancashire,[26] and EQUINOX Library Performance Measurement and Quality Management System,[27] and the participation in this work of libraries such as University of Münster Libraries, Danish Business School, Dublin City University Library, and research centres such as CERLIM, the Centre for Research in Library and Information Management based within the Department of Information and Communications at Manchester Metropolitan University.[28] The Australasian members of the Universitas 21 group have and are aligning the use of the CAUL materials availability indicator and the running of client satisfaction, staff satisfaction and other surveys. Some these results are also being aligned more closely with the Australian Quality Awards framework so the data can feed into benchmarking data banks covering the wider business community. Quality/Best PracticeThere is a definite trend towards the adoption and application of quality frameworks within the Australian university library community. The influence of the Commonwealth Labour government sponsored period of Quality Audits in 1993-1995 has already been documented in a 1998 article by Williamson and Exon.[29] At that time the first Total Quality Management/Total Quality Service best practice projects were implemented in Australian university libraries. Although less than 50 percent of the respondents to the current project surveys have formally adopted and applied a framework to date, the majority indicated their intention to investigate, implement or further develop partially implemented frameworks in the next 12 months. Two university libraries and one university as a whole campus have been awarded recognition by the Australian Quality Council for the application and implementation of the Australian Quality Awards framework. This recognition has provided leadership for others. In addition other local influences have all played a part in encouraging an appreciation and use of quality principles:
The main motivating factors for implementation include improving client focus, empowering staff to be part of process improvement and demonstrating effective resource use. University libraries have clear and wide reaching concepts of what makes a successfully implemented quality framework. Their comments covered all aspects of effective management. The formal frameworks used include:
A number of universities and their libraries had implemented local adaptations based on some of the above, the Scottish Quality Management System and various frameworks for vocational education and training. ISO accreditation is particularly associated with electronic services and information technology activities. A current international trend is towards interest in adaptations of national quality award frameworks, sometimes customised for public services, education and training. United States academic libraries are able to look at the Baldrige framework tailored for educational organisations. British universities have had the choice of the use of a range of quality frameworks, a number of which have the imprimatur of the government. Examples include Chartermark[30] and some which focus on particular aspects such as the Iip (Investors in People) program.[31] Apart from a low level of Australian interest in the Iip program, these types of frameworks do not appear to travel well across the globe. Whilst a number of university libraries in Europe are currently working on adapting the Business Excellence Model framework for quality management purposes in their own libraries it would appear that about one quarter of (mainly) larger Australian libraries have, or intend to, implement aspects of the AQA framework. This has grown from the first interest in this framework five years ago in one library. Staff CompetenciesBoth the literature and the survey responses indicate that whilst many libraries have embraced quality improvement initiatives with enthusiasm, the training and competencies required for staff to work effectively with the tools and techniques has not always been addressed to the same degree. Libraries with ongoing training programs which address quality management techniques (including benchmarking and performance measurement) and related skills are rare. Yet the presence of an integrated in-house set of staff development programs which addressed the competencies required appeared to be one indicator of a really effective quality framework implementation in a large institution. Conversely, some responses acknowledged that it was a lack of appropriate training at the appropriate time which contributed to some less than satisfactory experiences. It is recognised that a significant investment of resources is required for the successful implementation of a comprehensive approach. It is more likely to be in place where the institution has dedicated staff with at least part-time responsibility for quality, staff development and training. Access to training also tended to be more formal if the programs were university initiatives and if training was conducted by an external consultant, facilitator or, again rarely, by a university training unit with the appropriate emphasis. Possible downsides to this could be the one-off approach which was not sustained or could be counterproductive. A number of university-wide training sessions in Australia have been sourced from the United States from a considerable pool of training companies which supply this type of training to local industry or to the Total Quality Management/Continuous Quality Improvement training programs implemented in some United States universities as part of an institution-wide framework or program. On the local scene the Australian Quality Council offers training programs both for quality professionals and as awareness and skills training. Any university libraries participating in either AQC self assessment, or which apply for consideration for a quality award at whatever level, receive a real measure of awareness raising and staff training and development just by the participation of staff in the processes and their follow up within the organisation. The AQC has also developed nationally endorsed quality management competencies which form a part of the Australian government approved National Training Agenda. Outcomes and RecommendationsThe recommendations of the EIP project, the survey data results and the accumulated information resource were brought to the attention of the CAUL community of university librarians. The EIP project report and the handbook are part of the DETYA publishing program, as part of the ongoing series of hard copy EIP publications and with publication also on the DETYA website. Suggestions for the future include:
A CAUL Working Group is forwarding these recommendations. There are plans to establish a website where the data gathered by the project can be updated and new information on Australasian best practice activity can be usefully displayed. This site will complement the input-oriented AARL statistics data. Professional gatherings such as the 2000 Australian Library and Information Association conference in Canberra are providing opportunities for further developments and discussion, including the further refinement of indicators. ConclusionFrom the investigations undertaken, the project team was able to:
The project identified a number of exemplars of 'best practice'in each of the areas under investigation. Libraries can learn much of value by reference to these applications. In addition, the project identified areas needing review or priority attention particularly in relation to key areas which are a challenge in terms of performance measurement and benchmarking such as electronic library services, information literacy and reference and research services. The information drawn together in the report should assist Australian university libraries in identifying models and techniques which may enable them to initiate or improve programs appropriate to their individual missions and organisational culture. Australia may be far away and down under in global terms, but, if interest and activity are an indication, it is a country which can make a solid contribution to the development of best practice in university libraries. Notes
|
|||||
|
