Conditions of Use

Performance management, measurement and reporting in a time of information-centred change

Jennifer Cram



© 1996 Jennifer Cram. Originally published in Australian Library Journal 45(3), 225-238. Edited version of "Watter Taal Praat Hy Nou?": Performance Management, Measurement and Reporting in a Time of Information-Centred Change, paper delivered at The South African Institute for Library Science Conference, Cape Town, South Africa, 1995.

ABSTRACT: In a climate of information-centred change libraries are at risk of being marginalised. In order to survive and thrive libraries and librarians must develop a competitive edge relative to competing services and demonstrate a level of competence that stresses adaptability. Managing personal and library performance for customer value involves taking a holistic view and a systems approach. Maximising customer value must flow from a library's culture, beliefs, values, management style and performance management. Methods for ensuring that library staff are facilitated to deliver quality services are discussed and the importance of taking both a behavioural and a process approach to performance management is detailed. The implementation of an integrated hierarchical performance measurement model is proposed and the benefits of moving reporting practices from an efficiency/usage focus to a value focus are discussed.


In many ways, a library service is an intangible asset, the value of which lies in the future. Its value is dictated by what its parent body is prepared to pay now in return for some expected benefit. The level of that expected benefit influences the amount of support the library receives. But the people who make the decisions, though they may benefit directly or indirectly from the services provided by the library, rarely use it directly themselves. It is therefore critical that the benefits delivered by the library are made explicit to them.

Information is no longer the province of librarians. It is the new embodiment of wealth. The central focus of commercial enterprises is now production of information, rather than goods. Without producing information, manufacturing, marketing, or achieving competitive edge is impossible. Understanding the full extent and import of this shift is critical to the capacity of libraries to survive.

The central threat is that libraries deal in reproduced information. Reproducing information costs a minute fraction of producing it. No matter how many people read it, the utility of the information is maintained, but its commercial value, measured by what people are prepared to pay to read it, very quickly drops to nothing unless the producers of the information control access to it. Advances in information technology have given producers of information the capacity to provide for-fee access. Because, once information gets into a library it becomes, to all intents and purposes, a free good, the perception of information as the new embodiment of wealth has the potential to cut off supply to libraries.

Commercially viable information is not the only type of information which has the power to change the way libraries operate. Information-centred change is the genesis of far-reaching and world-wide organisational restructuring. How individuals feel is the most accurate information they have as to whether or not they are in control of their lives. [1] Being in control is a critical issue in performance management.

All these aspects of information are critical to the successful management, measurement, and reporting of library performance. Management, measurement and reporting of library performance are critical to developing a competitive edge, and developing competitive edge is critical to a library's long-term survival. To gain and retain competitive edge requires managing libraries for value results, for quality that meets the needs of customers and returns value to all stakeholder groups. It involves filling the gaps in our service model.

Service model diagram

Diagram 1: Service Model

Filling those gaps involves improving performance management, which in turn requires improving performance measurement. Neither improvement will ensure survival on its own, so attention must be paid to the efficacy of the way we report on performance.

Performance management

Libraries have to learn to compete with each other. As Michael Porter suggests, competitive advantage arises when several local rivals pressure each other to advance.[2] When libraries were the only game in town the profession could, perhaps, risk being complacent. Libraries are no longer the only game in town.

The organisation, the staff, and how they work is the crucial, differentiating factor in achieving competitive edge. The success that results from managing people effectively is often not transparent as to its source, and therefore virtually impossible to duplicate. Good performance management has two components - behavioural and systems/processes. It requires identifying and managing the links between micro level (individual), meso level (groups), and macro level (organisational) processes. The links are the information which is extracted, communicated and acted upon. Staff are the pivotal resource.

A major hindrance to delivery of quality service of high productivity is frequently an ingrained belief of managers that human behaviour is caused by a stimulus, an event or a situation outside the behaving person, that fear is a motivator, that you can make people do what they do not want to do. Colloquially, we refer to this as the carrot-and-stick approach. Psychologists refer to it as stimulus-response. What is referred to as stimulus should more accurately be called information. Information does not make us do anything. It merely tells us what is going on.

Behaviour encompasses much more than activity - it is a combination of four separate components of which activity (or acting) is only one. The others are thinking, feeling and physiology.[3]

Managers have a prime role in helping staff to better satisfy their needs, which usually means to act and think more effectively at work. Regardless of what the employee needs to do to feel better, when he succeeds what he has done is change from one behaviour to a better one. To do that will always require that he changes how he acts and thinks, so it is on actions and thoughts that the manager must focus. We must accept feelings and physiology, and understand how important they are, but we must concentrate on what can be changed. [4]

Managing for quality requires that managers must focus on the workplace far more than on the work. [5] A manager is responsible for consistency of purpose and continuity in the organisation, for the system as a whole, and for its continuous improvement. The manager should work on the system while the staff work in the system. When libraries fail to perform up to expectation, it is the very rare manager who will identify the problem as his or her own inability to manage employees effectively. Unwise use of managerial power and poor understanding of diversity in the workforce negatively impacts on productivity and inhibits the achievement of quality work, but "incompetent" workers, under-resourcing, administration interference, excessive accountability and so on will be blamed.

Traditional management requires staff to adjust to the job as defined by the manager. Traditional managers tell rather than show how to do the job, rarely ask for suggestions, inspect the work and use some form of coercion to make staff tow the line. Professionals, however, are likely to find their work satisfying regardless of how they are treated. This obscures the gap between actual performance and what is possible. Just because traditional management is not always clearly ineffective in libraries does not mean it is effective enough to consistently produce the quality service libraries will need to deliver to survive. [6]

Conditions for quality

There are three basic conditions that have to be met to ensure library staff can produce quality work:

  • A warm and supportive work environment.
  • An explicit expectation that staff can and will do quality work.
  • A manager who guides the process of helping staff learn to continually evaluate their work, and based on this ongoing self-evaluation, improve the quality of what they do.

I am convinced that a manager's main job is to be a mentor, expediter, arbitrator and facilitator, to provide employees with the best tools and workplace as well as a friendly, noncoercive, nonadversarial atmosphere in which to work.

To manage a library in order to close those gaps in the service model, we need an understanding of what customer-value based service is in the context of our particular library, a holistic approach to performance management and a genuine willingness to take responsibility for our own performance as managers, and as staff. Work must be structured so that it satisfies the basic staff needs for belonging, power, freedom and fun at work, because it is these rather than the need to survive that leads an employee to do quality work.

Productivity

Productivity or efficiency has a critical role, though one that must always be subservient to effectiveness. It can make the difference between the survival or failure of the system over time and it marks the quality of that survival. This negative entropy enables the library to achieve surpluses that sustain its future. It really is always a matter of doing more with less. As I tell my staff, laziness is no good unless it's well carried out. Energy and resources can be redirected to produce something else.

The link between play and performance

The traditional manager erroneously assumes that the more hours worked, the more work produced. People usually hit a productivity peak at a certain point each day, or over a span of a certain number of hours worked, then their efficiency, carefulness, motivation, and creativity wane as mental and physical energy levels steadily decrease. Working beyond that effective productivity point will result in error, poor judgement, accidents and injuries, and overall uninspired thinking.

To stay alert and productive throughout an average work day, you need to top up your energy reserves before they drop so low that they affect your performance. Play is critical to maintaining abundant energy. Play allows the brain's left hemisphere to rest while the right hemisphere creates new options and possible solutions to problems. Play helps release built-up tension, can open blocked thinking and trigger creative ideas, stimulates energy, allows you to safely explore new perspectives and new interests; and creates new links between ideas.

Four five minute productivity breaks, taken at will, are written into the daily "tasks" of the staff under my control. These breaks are designated to be spent on activities which they enjoy but which can be regarded to be connected with library work, although not necessarily part of their jobs.

Important in facilitating each member of staff to work to his or her full potential is to recognise that there is no one way of working that is best for everyone. Brain dominance affects behaviour, learning and work styles, and personalities. In the traditional workplace, however, the work preferences that are mandated, or judged "good", are left-brained, and thus a preference for certainty and few variables is encouraged where organisations are operating within an increasingly changing, ambiguous environment with many variables. In managing performance and building teams we need to take brain dominance into account.

Left and right brain needs for structure

Diagram 2: Left and Right Brain Needs for Structure

Positive reinforcement and self-fulfilling prophecies

Research at the University of Washington has suggested that both absence of good words and presence of negative words are coded as a negative in communication between marriage partners. Five positives for every negative, five compliments for each non-committal answer or complaint, five expressions of affection for each outburst of anger or blame are needed to ensure the marriage survives. [7] I believe it reasonable to suggest that people need the same 5:1 ratio of positive to negative/neutral reinforcement in the workplace in order to be capable of optimum performance.

Workplace design

The physical environment can have a major impact on the quality of overall service. At least forty minutes of productive time per worker is lost each day because of poor workplace design. Research at the Buffalo Organisation for Social and Technological Innovation indicates that a better-designed environment can increase employee performance to the financial equivalent of 15% of each person's yearly salary. [8]

The quality, efficiency and effectiveness of a library's services are facilitated or lowered by a multiplicity of factors most of which can be impacted either positively or negatively by the manager.

Performance measurement

Performance measurement is a critical component of performance management. Trying to manage a library service without the systematic employment of performance measurement could be compared to the experience of Christopher Columbus: when he set sail he did not know where he was going; when he got there he did not know where he was and when he got back he did not know where he had been - and he did it all on someone else's money.

There is a plethora of published material on performance measures, but there are also a number of serious problems with this material, and with performance measurement as it is applied in most libraries

  • It is hard to see where all the bits fit together, i.e. it is not holistic
  • It is often reminiscent of economists, i.e. too much that really is important is ignored or discounted in the interest of neatness
  • The collection of data is focused on process and transactions
  • There is significant underperformance in extracting information from the data collected
  • Customer satisfaction/value data is approached as a report card rather than as a source of information about areas where improvement or further investigation is needed.

There is a Russian saying that everyone looks at the world from the belltower of his own village. Perceptions differ because our experiences differ, and because we select from among our experiences. Each of us observes different data in part because we are interested in different things. Depending on our specific perspective, our perceptions vary. [9]

But there are a number of things that should be accepted as constants.

  • Performance measurement is a management tool which delivers the concrete performance data needed to arrive at a meaningful evaluation of a library's performance. Analysis of this data gives information which while useful in decision-making, cannot replace it.
  • All data are collected through some form of observation. The investigator either watches what happens, or seeks self-observations from a user of the product or service.
  • Performance measurement is a form of mapping. The map is not the territory. Maps record difference, such as difference in altitude or vegetation. We make sense out of raw data by identifying difference. [10]
  • Performance measurement is not an end in itself. There are two purposes for performance measurement which should neither be confused nor used as a surrogate one for the other - to improve operations and ultimately services, and for reporting.

Performance measures form a hierarchy, and to be fully effective, performance measurement should follow an integrated hierarchical performance measurement model.

Hierarchy of performance measures

Diagram 3: Hierarchy of Performance Measures

Central to this hierarchy is the recognition that measures conducted at each level must feed into the next if a holistic picture of the library's performance is to be obtained, and, even more important, not only must all stakeholders be identified, but the different motivations and needs of the four broad groups of stakeholders must be understood, and recognised in the way performance data is gathered and used, and in the way the library reports on its performance.

Stakeholder Groups

Diagram 4: Stakeholder Groups

The four broad stakeholder groups are

  • Normative groups, those which provide the authority for the library to function and set general rules and regulations
  • Functional groups, those which affect many of the day-to-day activities of the library and facilitate operations and serving customers
  • Diffused groups, those which take an interest when concerned about protecting the rights of others (a grouping almost universally ignored when libraries list stakeholders as part of a strategic planning process
  • Customer groups, defined into sub-groups by their needs.

Libraries tend to measure performance in parts of the hierarchy - but usually for internal management or accountability purposes. This has allowed the relationship between the rest of the hierarchy and the ultimate measure, long-term survival of the library, to be obscured, or ignored.

It was regarded to be a great breakthrough when libraries discovered that they needed to look at output measures. Nowadays we are talking about outcomes measures, without fully understanding what these might be, as evidenced by the lack of clear-cut differentiation between processes and outputs, and by the apparent belief that outputs are an end in themselves. Outputs are what we produce to achieve stakeholder satisfaction.

Customer satisfaction information is primary and direct evaluation information. [11] This is understood by the marketing industry which regards other performance information, such as number of sales, as indirect information about customer satisfaction.  Not infrequently libraries dismiss stakeholder satisfaction as not being "objective" enough to be a valid measure, yet the identification of the concerns of all stakeholder groups is essential, so that the service can be constantly measured against them.

For a library to be successful in the long run, it must satisfy all stakeholders, not just customers groups. Conceptually, virtually all library activities, programs and policies should be evaluated in terms of their contribution to satisfying stakeholders. [12]

Satisfaction is typically measured through surveys. Surveys are direct, easy to administer and interpret, and have clarity of purpose, and face validity. Surveys of library customers tend to be occasional and conducted when a good report card needs to be presented to the parent body as a concrete support of unquantified claims that the library is a Good Thing, while indirect measures are used as the "real" measures of service. Data on satisfaction of other stakeholders is rarely, if ever, systematically collected, evaluated or even considered, and some types of libraries may be selective in the attention they pay to different customer groups. Undergraduates, for example, get very short shrift and scant attention in many University Libraries, at least in comparison to academic staff.

There are a number of problems with customer satisfaction data gained through surveys. Satisfaction measurements are generally designed to tap the underlying global or "net" satisfaction with a product or service. Hence the information being sought is aggregated, and an observed distribution of satisfaction ratings is presumed to be a reflection of "true" satisfaction. [13]

Caution should be exercised when interpreting customer satisfaction survey data or using them in decision making. In virtually all customer satisfaction surveys the distribution of responses is negatively skewed, that is, the majority of the survey participants report satisfaction.

Satisfaction normal curve

Diagram 5: Satisfaction Normal Curve

Perhaps they really are satisfied, but, perhaps satisfaction is caused by factors such as expectations, or perhaps satisfaction possesses a distribution that is different from the normal distribution of other common psychological constructs13, or the distribution may be affected by the research methodologies employed and/or the inherent intrapersonal characteristics of the surveyed customers.

The literature suggests that satisfaction data collected using different modes are not comparable, for example, oral administration of satisfaction questions, either in person or on the telephone, appears to increase satisfaction ratings by approximately 10-12 percent relative to self-administration (e.g. survey forms). [14]

While the different modes of data collection result in marketedly different reported levels of satisfaction, the distribution of those satisfaction ratings are consistently negatively skewed. The way the data is collected seems to influence the level of reported satisfaction but not the distribution shape. How you ask the question also appears to affect the level of satisfaction. A positive form (eg how satisfied are you?) appears to lead to greater reported satisfaction than a negative form (eg how dissatisfied are you?). [15]

To minimise obscuring of data by the tendency to skewing, and to ensure that we obtain useable information which can be fed back into the service design and operation loop, stakeholder satisfaction information should be sought on specific issues rather than general ones, and the questions should be framed as an information-seeking rather than a compliments-eliciting mechanism. In particular, by limiting the role of a global measure of customer satisfaction to identifying dissatisfied customers, and then acting appropriately, greater benefits are likely to accrue to a library than simply constructing a report card of customer satisfaction. [16]

Of course, if your funding authority or parent body requires report card surveys, by all means do them. But don't believe your own press. If you start believing that, because 75% or 80% of customers are satisfied, your performance is good, you will never achieve competitive edge.

Interpreting indirect measures too simplistically may also obscure critical information. For example, increased business in the user service area does not necessarily mean that we have a satisfied client. It may indicate increasing user-frustration with systems that have become increasingly complicated. [17]

Observation

Observation, in the narrow sense, is usually identified as a "qualitative" method, that is, one that produces not data but some kind of account. Observation can, however, also produce quantitative data. Observation is cheap, easy and straight-forward. Library staff do it all the time, but this rich source of data is rarely captured, accumulated or analysed. If captured systematically observations can identify problem areas which can then be further researched. Observation, then, is fundamental to our understanding of the situation in which we work, and fundamental to the process of developing methods of data collection which will provide more comprehensive data for management decisions. [18]

I was first alerted to the importance of capturing, and acting on staff observations about fifteen years ago. At the time I was managing a public library where new books were displayed on antique cedar tables easily observable from the circulation desk. Circulation staff worked the desk full-time. This enabled them to identify a phenomenon which we dubbed "Virgin Book Borrowers". A significant number of users would browse the books on the new book table, and only choose from the newest of these. A little action research revealed that these borrowers only borrowed books with few date-stamps on the slip because they liked to believe they were discerning readers of books that few could understand or appreciate. They were also predominantly some of the more influential people in the town. The value of their library use to them included confirmation of their uniqueness. We responded by regularly replacing the date-due slips on items displayed to ensure that these customers had a much larger number of items to choose from. Though it should have raised questions, in fact this had a very positive effect on the reputation of the library with its normative stakeholders.

This experience led me to develop a more directed form of observation to capitalise on the interpersonal interactions between staff and stakeholders. I call it the One-Question Survey. Staff are requested to discuss a particular topic with as many people as possible over a period of a week or two. Because the information sought is part of a conversation, rather than one question in a survey, you tap into the sub-texts that more formal surveys miss, the totality of stakeholder reactions, needs and opinion in relation to the topic, and the reasons for their responses. This sort of personal contact is a far better source of information on what you can do to improve quality of service than virtually any other form of data-gathering.

The greatest deficiency in performance measurement, however, is not the failure to collect data, but underutilisation of the data collected to evaluate performance, and a lack of a bias towards action as a result of what evaluation and interpretation is carried out.

Service delivery should be treated as exploratory or experimental. No service should be implemented in the belief that it is going to remain unchanged. It is better to implement it in the spirit of experimentation and to see if it works, checking from time to time that it is still working as intended, that is, continuing to fulfil its function.

In a way, most services in a library could be regarded to be a form of action research.

Diagrammatic representation of action research

Diagram 6: Action research

In collecting data, and evaluating performance and services, libraries are not testing scientific theories, rather policy guidance and indicators of areas of concern are the end results of our measurement regime.

Performance Reporting Collection of data and evaluation of library services serves a different purpose to reporting on performance. It is neither necessary nor advisable to report every iota of data collected. I believe that libraries routinely report too much - often pages and pages of statistics - and by so doing obscure or omit the important information.

The most frequently used measures of business success are financial ones. Therefore, the major challenge for all libraries is to make the change from reporting usage to reporting value. Despite some advances in the area of performance measurement, libraries still tend to report raw usage statistics, sometimes analysed for trends such as increase in loans. Some libraries track and report efficiency improvements. Few have moved to regular reporting of outcomes. As far as I know, only my own service has moved to reporting only value.

It is important in this sense to differentiate between the economists' definition of value and its common definition. Economists believe price is a gauge of the value one is prepared to pay. They define value as the amount paid for goods and services. In the more general sense, and the sense in which value should be reported, value equates to what the economists define as worth - what is gained or the monetary value derived. In the economic sense value and worth are not necessarily equivalent.

Reporting value, in the economic sense of worth, rather than reporting usage, enables libraries to demonstrate to parent bodies that the library delivers a "profit" on the resources invested in it.

Being able to quantify and report value, particularly in the form of return on-investment information, is critical to survival. In library terms it is demonstrating to fund providers the value of the services and information the library delivers compared with the total annual budget of the library. Because it may be difficult (though not impossible as some have claimed) and because therefore it is important to include both tangible and intangible returns, librarians may be more comfortable with calling return-on-investment, Return Value. The calculations must include the investment in the collection, in staff, and in operational costs. If you do not demonstrate that your library is a rational investment rather than a cost-centre, it will always be in danger of being downsized or outsourced. I advise that the temptation to exclude the library materials vote on the grounds of it being capital expenditure be resisted. Implementation of accrual accounting will require valuing the library collection as an asset. Exclusion of expenditure on collection from annual return-on-investment calculations could increase vulnerability to closing of libraries and realisation of their assets.

In my corporate library service we have developed a regime of in-depth holistic data gathering which delivers information about the value of library services to the organisation. It has some very simple characteristics. Our target standards for service delivery, for example, read "100% within customer defined timelines". If someone wants some information in 10 minutes or in 10 days, we meet their need.

Our reference satisfaction data is collected at point of transaction and delivers useable information on both the macro and micro levels about appropriateness of our collection, the extent to which we satisfy information requests (again defined by the customer), the amount of time we have saved the customer by doing the research (as estimated by the customer) and the way in which the information will contribute to the meeting of Departmental goals, and the monetary value of the Department having the information.

The return-on-investment realised by my library service thus does not only relate to the value of the information as it is used. It also itemises the productivity gains resulting from having librarians do the research rather than people whose main role is contributing directly to the achievement of the organisation's goals. We report accumulated benefits by adding up the customer's estimates, compare that with the time taken by library staff, and use relevant salaries to quantify the productivity gains, to which we add the estimations of dollar value of the information and appropriate narrative about the contribution to the organisational goals.

While this reporting method cannot be entirely statistical, the combination of hard monetary value benefits and narrative has enabled us to drastically reduce the volume of information reported. As we apply the benefits against the total budget, including acquisitions, and are averaging an 8:1 return on investment, it is very clear to all that corporate library services are an investment.

School and academic libraries should be demonstrating value to both the success of the academic program, and contribution to other institutional goals, such as accreditation, ranking, and capacity to attract endowments.

Public libraries represent a particular challenge because their contribution to the productivity of the local government authority may be marginal and their contribution to the community not necessarily obvious to the parent body. It is, however, entirely possible to demonstrate contribution to the economy of the area, together with some cost-reduction in delivering other Council services. A large proportion of the money spent on providing a library service goes in staff salaries. Most of those salaries are recycled into the community or returned to the community in tax-funded expenditure.

A public library allows many users to share library resources, reducing the community's expenditure on books, magazines, newspapers and other library materials, and therefore:

  • frees up personal discretionary income to be spent in other ways in the community
  • enables the council to realise savings by distributing council information through the library
  • reduces the amount of paper waste and therefore the cost of waste disposal, and
  • delivers the benefits which accrue to a community which has access to life-long learning and culture-transference opportunities.

All of these benefits can be quantified in monetary terms.

Replacing a usage and efficient-management focus in performance reporting with a return value focus requires some planning, but is not in itself difficult. The change makes reporting both simpler and more effective. Telling senior decision-makers the number or percentage increase in loans and users is neither as clear nor as powerful as telling them that the organisation obtained a return equivalent to four, eight, or ten times its investment in the library.

In reporting value care must be taken not to confuse the value of information with the value of the library. The skills and services provided by librarians are a critical contribution to the overall benefits realised.

Conclusion

Information services which compete with libraries are multiplying rapidly, yet, in the final event, it is not outside competition but unwillingness of library managers to give up control which will be the biggest threat to the future viability of libraries in the information "industry".

To achieve positive results managers must have detailed and comprehensive performance information and must loosen the reins, involving staff and other stakeholders. Empowering the stakeholders means that the library manager is less central to every decision and therefore in that narrow sense the manager is less "in control", but at the same time more in control of what counts: the results, the results that will close the gaps in the service model, and ensure continuing stakeholder satisfaction and the resulting long-term survival of the library.

References

1 Glasser, William. The Control theory manager. St Leonards, NSW, Allen & Unwin, 1994 p 60.

2 Porter, Michael, 'Don't collaborate, compete.' The Economist, June 9 1990.

3 Glasser, p 66.

4 Glasser, p 69.

5 Glasser, p xii.

6 Glasser, p 79.

7 Johnson, Catherine, `Love for life.' New Woman July 1993, pp 56-59.

8 McGee-Cooper, Ann et al. You Don't have to go home from work exhausted! Dallas: Bowen & Rogers, 1990.

9 Fisher, Roger, Elizabeth Kopelman and Andrea Kupfer Schneider. Beyond Machiavelli: tools for coping with conflict. Cambridge, Massachusetts, Harvard University Press, 1994, p 21

10 Bateson, Gregory, Steps to an ecology of mind. New York, Ballantine Books, 1972.

11 Peterson, Robert A and William R Wilson, "Measuring customer satisfaction: fact and artifact." Journal of the Academy of Marketing Science, vol 20 no 1 pp 61-71, p 61

12 Peterson, p 61.

13 Willerman, Lee, The psychology of individual and group differences. San Francisco, W H Freeman, 1979.

14 Peterson, p 65-5.

15 Peterson, p 65.

16 Peterson, p 69.

17 Cullen, Rowena `Evaluation and performance measurement in reference service,' New Zealand Libraries v 47 no 1 March 1992 pp 11-15.

18 Wilson, Tom, 'Data collection for performance evaluation' in Margaret Haines Taylor Quality assurance in libraries: the health care sector. Ottawa, Canadian Library Association, 1990 pp 49-64, p 55.