Tord Høivik: Why is quality control so hard?
A summary of the paper for oral presentation

Ten statements

1. Research projects in many countries show that the quality of reference work in public libraries is highly variable.
   
2. The level of quality is not better in Norway, Denmark or Sweden than in the United States. Norwegian librarians know the evidence.
   
3. But the research projects do not change in reference practices. How can we explain this lack of action?
   
4. Library organizations - in general - are not eager for change. Their staff members seek stability rather than change. Reference services - in particular - are resistant to change. Why?
   
5. There is no user pressure. The users are a lonely mass. They do not organize and do not complain.
   
6. There is no political pressure. Local politicians may be interested in libraries, but they are rarely interested in reference work.
   
7. There is no professional pressure. Professionals change their practices when they are committed from the inside - or monitored from the outside.
   
8. Internal motivation does not work. Librarians facing the public are too busy to safeguard reference quality.
   
9. External monitoring is impossible as long as we lack statistical data on reference work. We must measure the outcomes we want to control.
   
10. Digital reference work may improve the situation. But only if libraries use the opportunity to collect, analyze and share data on reference transactions on a routine basis.

A. Reference quality as a problem

1. In all the Scandinavian cases, the quality levels fell below the US average. And this average - the famous 55%-rule - is itself dismally low.
2. Effective libraries ought to give full and satisfying answers to 80-90% of all reference questions.

B. Is the evidence biased?

1. Nearly all studies of reference quality focus on a particular type of inquiry: factual questions with definite answers. In addition, survey questions tend to be more difficult than the average.
2. Most factual questions in public libraries are easy to answer:
   • What is the smallest state in the world?
   • Is Jerusalem a maritime city?
   • What does the girl`s name Ylva mean?
3. The 55%-rule has a bias. It does not apply to reference in general, but only to a subset of the queries.
4. As Richardson says, reference is better than we thought. Libraries have been tested under more stringent conditions than the average.
5. But the Scandinavian tests were not extreme. Professional service providers should maintain their standards under moderate pressure.

C. The lack of action

1. The research was widely reported and hotly debated at library conferences. But what happeneed in the service? Basically : very little.
2. Only a handful of libraries have introduced some forms of quality control routines. At the national level, performance measurement was not a visible concern. Reference continues as before.
3. Why is it so difficult to act on the evidence? What makes effective quality control of reference work so hard to achieve?

D. Resistance to change

1. In rapidly changing areas - the arts, media, front-edge technology - organizations tend to welcome innovation.
2. In more stable milieus - insurance, railways, libraries - tradition reigns.
3. A suitable term is inertia: An organization at rest remains at rest. An organization in motion continues in the same direction - unless it is pushed off-course by a new force.
4. People and organizations long for constancy rather than for change. We want predictable environments at work. We want mastery and control.

E. Social carriers of change

1. Resistance to change is normal. Library rhetoric is full of "challenges" and "visions". But the words are divorced from reality.
2. In organizations, inertia is the normal state of affairs. Reference work is no exception. The interesting question therefore becomes: how is institutional change at all possible?
3. The most obvious answer is technology.
4. But technology is a force that comes from the outside. It makes an impact whether we like it or not.
5. Evidence-based librarianship represents something different: a deliberate (proactive) approach to change.

If we look for agents or carriers of social change in public libraries, we could try:

1. the library clients (customers)
2. the government that pays for public library services (local government)
3. the library administrators (management)
4. the librarians (profession)

   I. What can customers do?

   1. In public libraries, the customers have very little power.
   2. Library clients do not organize themselves and demand better service quality.

   II. What can local government do?

   1. In Norway, the government has started to take a strong interest in the quality and efficiency of municipal services.
   2. When the political spotlight turns to libraries, the focus will be on loans rather than reference work.
   3. Lending statistics are easy to compile, to compare and to interpret. The reference service is much more obscure.

   III. What can managers do?

   1. Quality is a key word in modern management thinking.
   2. Industrial quality thinking is brought into the service secto: first into private, and then into public services.
   3. Official documents tend to describe organizations as rational. In rational organizations, change comes from deliberate planning.
   4. All quality models involve measurement. In order to manage reference quality, we must know the quality level.
   5. But measurement at work is seldom popular. Quality management threatens the autonomy of professional people.

IV. What can the profession do?

1. Professionals are not opposed to quality - but to measurement. They do not want to be monitored by administrators - like common employees.
2. The evidence-based approach tries to avoid this dilemma. It places the force of change within the profession itself.
3. When we say that librarianship should be evidence-based, we say that librarians should learn from the outcomes of their work.
4. Evidence-based librarianship would create a learning profession.

F. How professions learn

1. In the library sector, the relationship between practice and research is weak.
2. Medical doctors must keep in touch with research. When quality is a matter of life or death, control mechanisms are strong.
3. Reference quality is not a matter of life and death. Librarians that fail, only waste the time of their customers.
4. Single studies, however shocking, are soon forgotten. Evidence is not enough.

G. Research and practice: a missing link

1. In librarianship research and practice are only loosely related. Evidence-based librarianship as an effort to create stronger links between research results and operational practice.
2. Scientific knowledge is not an aggregation of results. Scientific knowledge is a coherent body of concepts, methods, interpretations and results.
3. Science is a collective undertaking. It proceeds by discussion, criticism and re-evaluation among researchers.
4. Scientific interaction requires conflict as well as cooperation. Sometimes we must struggle with our peers. Sometimes we can build on their work.
5. Researchers must interact deeply in order to create a collective body of knowledge.
6. Library science is an applied discipline. The purpose of quality studies is to improve quality.
7. In physics or anthropology, we cumulate knowledge. But library science is not a theoretical discipline. Librarianship advances when the practices - the state of the art - improve.
8. Quality testing has no meaning unless it leads to quality improvement. In the applied sciences, we cumulate practices.

H. Transactions: brief, private and ordinary

Traditional reference work is not very suitable for measurement. Typical reference transactions are - in general - too short, too private and too ordinary to be monitored.

I. Digital reference: a technological force

1. The new technology can change the way reference work is organized.
2. The cost of data collection is low. Questions and answers are recorded in digital form.
3. This record - the reference log - makes supervision and control feasible.
4. Librarians and assistants can share their knowledge.
5. Supervisors can check what goes on day by day.
6. Managers can analyze "inquiry trends" and "deviations from the norm"..
7. Digital reference can actually be evidence-based.

J. Summing up

1. Digital reference work may improve the situation.
2. But only if libraries use the opportunity to collect, analyze and share data on reference transactions on a routine basis.
3. Evidence-based action requires evidence. In other words: statistics.
4. EBA requires actors that act on the evidence. In other words: people that understand and argue with statistics.

Tord Høivik. Oslo University College. Full paper at: http://home.hio.no/~tord/perm/ifla/why.htm