A Research Review
of five
Online Public Access Catalog User Studies
 

by
Karen E. Medina

for LIS 380 le
Spring Semester
April 29, 1999
 



 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


    This paper is an evaluation and comparison of five research papers covering various topics in On-line Public Access Catalog (OPAC) user studies. The research papers evaluated here were purposefully chosen to cover a wide variety of OPAC studies. One was chosen simply for its publication date (1998), to represent some of the most recent research. The others were chosen for the target of the studies: children, older adults, college campuses, etc. The studies chosen for this review should be a fairly representative view of the research going on in this area.

    The papers reviewed here (in order) are:

Wiberley, Stephen E., and Robert Allen Daugherty. "User Persistence in Displaying Online Catalog Postings: LUIS." LibraryResources & Technical Services 39 (1995): 247-264.

Drabenstott, Karen M., and Marjorie S. Weller. "Failure Analysis of Subject Searches in a Test of a New Design for Subject Access to Online Catalogs." Journal of the American Society for Information Science 47 (1996): 519-537.

Borgman, Christine L., Sandra G. Hirsh, Virginia A. Walter, et al. "Children’s Searching Behavior on Browsing and Keyword Online Catalogs: The Science Library Catalog Project." Journal of the American Society for Information Science 46 (1995): 663-684.

Sit, Richard A. "Online Library Catalog Search Performance by Older Adult Users." Library & Information Science Research, 20, no. 2 (1998): 115-131.

Ferl, Terry Ellen, and Larry Millsap. "The Knuckle-Cracker’s Dilemma: A Transaction Log Study of OPAC Subject Searching." Information Technology and Libraries 15 (1996): 81-98.

    An Overview of each paper is given first. Comments on that paper come directly after each Overview. After explanations and comments of all five papers, a Comparison of the papers is made. The Overview of each paper contains some basic information about the study. The breakdown of this information follows the general format of : 1) problem or issue studied, 2) the background and prior research mentioned, 3) the researchers’ hypotheses, 4) the research methods used in the study, 5) the data and findings, 6) conclusions made, 7) future research suggested and 8) the implications of the study - all of which were gathered from each paper. The Comments section and final Comparison look at the 3 questions to ask from Chapter 17 of Jeffrey Katzer’s Evaluating Information. The questions concern 1) framework, 2) method, 3) results/conclusions. The Comparison section of the paper takes an overall look at the studies presented here and compares them.
 

An Overview of the Wiberley, Daugherty, & Danowski Paper on User Persistence (1995). - paper #1

Wiberley, Stephen E., and Robert Allen Daugherty. "User Persistence in Displaying Online Catalog Postings: LUIS." LibraryResources & Technical Services 39 (1995): 247-264.

    The 1995 Wiberley, Daughtery, and Danowski research paper looks at an OPAC user persistence study they did in 1992 at the University of Illinois at Chicago. The basic question they tried to answer was "how many items is too many items" in a retrieval set. Specifically, at what point does a searcher stop displaying items? By looking at how many postings users typically persist in displaying, these researchers hoped to find an ideal range for item retrieval sets, and suggest a point beyond which systems should provide help to the users.

    They cited many previous related research papers including names like Tagliacozo and Kochen, 1970; Kidder, 1983; Markey, 1983a, 1983b, and 1984; Tolle, 1983; Marchionini and Shneiderman, 1988; Wiberley and Daughterty, 1988 and 1990; and Prabha, 1989.

    The hypothesis behind the project was that there is an ideal retrieval set size and that it is 30 to 35 items. This was based on previous research, which indicated users normally display no more than 30 to 35 postings (Wiberley and Daugherty 1988). Their major assumption was that the goal of a user is a call number. To test their theory, Wiberley et al used the methods established by Miller and Bartz (1982) and Tolle (1983). They used a combination of human observers, written questionnaires and analysis of transaction logs. They chose a working environment study over an experimental setting. They were concerned that an experimental setting would change the motivation of the subjects, causing them to persist in their queries when normally they would have stopped.

    The experiment took place at the University of Illinois at Chicago, during the spring of 1992, with the in-place OPAC LUIS. The observations were taken morning, afternoon, and evening over a 10 week period. They looked only at in-library users (no dial-up information) at selected departmental libraries. The users who were observed through their entire interaction with the online catalog were asked to fill out questionnaires. When evaluating the transaction logs, the researchers had to do a replication of all the searches seven months later. They took into account the seven month difference when evaluating the retrieval sets.

    Persistence was measured as the number of items on an index screen that users actually displayed. 850 users were observed and were rated as to their persistence into four categories: totally persistent, partially persistent, record desplayer, and overloaded. Of the 850 observed user sessions, 490 returned questionnaires (58%), of which only 439 were usable. 26% of the users reported overload during the session. 147 postings were considered too many. The general threshold of reported overload was 100.

    These findings were compared to similar study done at the University of Illinois, Urbana with the LCS catalog in 1990, which recorded 748 observed sessions, 418 questionnaires, 11% reporting overload in the observed session, and 13 postings were found to be too many. The general threshold of reported overload in this earlier study was 15.

    Wiberley, Daugherty, and Danowski discussed the probable reason for the differences between their two studies: the LCS study at the Urbana campus in 1990 and the LUIS study at the Chicago campus in 1992. Most of the differences were thought to be based on the differences between the catalogs themselves. They claimed that LCS was less intuitive and "contained a design feature that seemed to inhibit persistence" (p. 248). LUIS was a larger database and provided subject and keyword searching which LCS did not. They indicated that LUIS is a second-generation system.

    The conclusions these researchers came to indicate that online catalog systems may want to provide assistance to users for a retrieval set of 30 or more, but need to provide assistance for retrieval sets of 200 or more items as user persistence is diminished at this point. They also conclude that the "more item-specific information a system initially displays, the greater the persistence of users in displaying additional records."

    The authors imply that further research needs to be done relating user fatigue and user persistence to system design features. They also indicate that other OPACs may find different results and that transaction logs are rich with information.

Some Comments on the Wiberley, Daugherty, & Danowski Paper - paper #1 continued.

    The framework of this article is fairly straightforward. They pose a question which their study could find an answer to: when do users give up looking at records from a retrieval set in a specific academic library online catalog? The definition of persistence and the explanation of the expected goal are reasonable. The works cited vary from studies done with card catalog subject searches, to research done on data collection, and to other OPAC user studies.

    Taking the user study out of the laboratory and observing actual users was very important for this study.  The observations were made as unobtrusive as possible.

    The actual research method is described in detail. At one point, the researchers explain a seven-month delay in the evaluation of the transaction logs. They look at the growth of the catalog during that time and have convincing numbers to indicate that the results still relate to the experimental situation.

    While the differences between the LUIS database and the LCS database of the earlier study are described, A few important considerations were missing.  The records in the LCS system for items owned before 1979 had no subject headings attached to them.  This definitely plays an important role, yet this paper never mentioned it.  The LCS system would have much higher user frustration because known items would not appear in the retrieval set.  For instance, Leigh Estabrook's PhD thesis would have title and author access, yet no subject access.  You would not be able to pull it up by looking for Libraries--United States.  User frustration at the University of Illinois Urbana Campus may be higher than at the Chigago campus.

    No mention of the user populations differences was made. Both were academic libraries, but what percentage of the users were undergraduates, graduates, staff, or faculty? Undergraduates may have different motivation in searching than a professor would. Motivation was completely over-looked, yet it plays an important part in persistence.

    It may be important to also look at how long each database had been in use. Users may actually develop a higher tolerance for large retrieval sets over time. Some of the other factors that may come into play are the fact that the records in the LCS system for items owned before 1979 had no subject headings attached to them. This plays an important role in user frustration when a known item does not appear in the retrieval set. User frustration at the University of Illinois Urbana Campus may be higher than at the Chigago campus.

    Based on their findings in this second study, the user persistence question may be entirely system dependent and not have a universal answer that applies to all OPACs. However, their conclusions do have some validity. There is a limit to the number of records a user is willing to search through and it is tied to the ease of use of a system.
 

An Overview of the Drabenstott & Weller Paper on Failure Analysis of Subject Searches (1996). - paper #2

Drabenstott, Karen M., and Marjorie S. Weller. "Failure Analysis of Subject Searches in a Test of a New Design for Subject Access to Online Catalogs." Journal of the American Society for Information Science 47 (1996): 519-537.

    The Drabenstott and Weller research project was a test of a search tree design of an experimental OPAC. They point out that subject searching on OPAC's is not meeting the needs of the user because, in general, subject searches are difficult to conduct. This project was designed to incorporate suggestions made by users into a subject search design of an online catalog.

    This paper mentions previous research done by numerous other researchers mostly concerned with OPAC design, subject heading uses, and user surveys.

    The hypothesis behind the study was that an OPAC designed to improve efficiency in subject searches will increase user perseverance and encourage browsing. The idea was that search trees would be a potential solution to subject searching problems, increasing the efficiency of subject searching by automatically switching search fields when no items are found.

    The study was conducted at two different academic libraries: the Mardigian Library of the University of Michigan - Dearborn, and the Lilly Library of Earlham College. Drabenstott was testing the design of an experimental database. Each session began with an "unobtrusive" pre-search interview and ended with post-search questions. No human monitoring of the system occurred. Two experimental databases were created, the Blue System (utilizing a prescribed search tree sequence of programmed techniques) and the Pinstripe (utilizing a limited random sequence of programmed techniques). The Blue System was being tested to see if the search tree sequence provided a more robust environment for subject searching.

    Various data were collected in this study. Precision data were taken for both systems. For the two libraries, these scores differed; one indicating that the Blue System was more precise at locating accurate hits, while the other indicated the Blue System was more precise. The Blue System was preferred or rated equal for performance.

    The conclusions that the researchers came up with are that both systems need improvement. A "good" system would help users get new ideas for subject headings. The researchers also concluded that searches failed due to lack of user perseverance (accurate hits were made, but the user failed to look at them). Query specificity and conflicting relevance assessment were also a problem.

Some Comments on the Drabenstott & Weller Paper - paper #2 continued.

   Drabenstott mentions "successful" searches, but fails to define the properties that make a search successful.  This is of vital importance.  The paper also indicates that accurate hits were made, but the user failed to look at them.  This indicates an interface problem.

   Drabenstott gives good examples and explanations of the questionnaire.

   Drabenstott fails to mention if an existing OPAC was present in the libraries she studied. This is important, as the introduction of her OPAC may have caused some confusion over which OPAC was experimental and which was the existing system.

   Other studies work with existing OPACs. Drabenstott introduced a completely new system for a short period of time. Introduction of a new OPAC usually meets with skepticism. This may have effected the outcome of her data.
 
 

An Overview of the Borgman, Hirsh, Walter, and Gallagher Paper on Children’s Searching Behavior (1995) - paper #3

Borgman, Christine L., Sandra G. Hirsh, Virginia A. Walter, et al. "Children’s Searching Behavior on Browsing and Keyword Online Catalogs: The Science Library Catalog Project." Journal of the American Society for Information Science 46 (1995): 663-684.

    The Borgman paper is a study of children’s information seeking behavior on library database systems and on-line public access catalogs; specifically looking at the effects of age, sex, and computer experience on the ability to use a catalog retrieval system. They cited research work covering a broad spectrum of topics including adult user studies; touchscreen OPACs; and various findings on abilities such as typing, spelling, alphabetizing, Boolean Logic, recognition knowledge vs. recall, and browsing capabilities of children. They also mentioned Piaget’s child psychology research.

    The overall plan of the research was to construct a catalog retrieval system, test it, and then refine the system and the theories behind the original system. The research was done in four separate studies (over 4 years) and resulted in numerous papers (10), this paper being the "first full synthesis of these experiments." The catalog retrieval system constructed for this experiment contained a graphical metaphor of a bookshelf, labeled by categories (science and technology or ‘people using science’). The system also contained a library map to show where the search items could be found. They combined this graphical interface with written choices, to allow children to recognize the words while eliminating the need for children to remember the vocabulary.

    Borgman (et al) chose a formative evaluation approach, referring to Patterson & Bloch (1987) and Vasek & Volger (1984). The subjects varied in age (9-12), sex, and ethnicity. Four different locations were studied. The subjects were all given a few minutes to explore the retrieval system before observations began. The researchers observed the performance on specific search topics provided by the researchers. The search strategies were also monitored on-line. A private interview and then a focus group interview followed the performance test. The researchers also consulted various experts (teachers and librarians) to critique the results.

    After each phase of the experiment, the retrieval system (called the Science Library Catalog or SLC) was modified. In the third and fourth phase, SLC was compared to the existing OPACs at the libraries (Orion and LePac).

The results of the experiments were analyzed individually and then summarized with charts.
 
Borgman Table. Successful searches by experiment.
Experiment
Search Topics
Success median, mean
n
Differences by
1 – SLC
6
5
4.7
34
Age; Science vs. Technology
2 – SLC
6
5
4.9
32
None
3 – SLC
4
3
2.8
33
None
3 – Orion
4
3
2.8
33
Age; Question Set
4- SLC
4
2.5
2.4
32
Science vs. Technology
4 – LePac
4
3
2.3
32
Question Set

    "The number of search topics matched was comparable across all systems and all experiments; search times were comparable, though they varied among the four SLC versions and between the two keyword online public access catalogs (OPACs). The SLC overall was robust to differences in age, sex, and computer experience. One of the keyword OPACs was subject to minor effects of age and computer experience; the other was not."

    They also found that "children prefer not to use any intermediary finding tools when they use libraries. Their first choice is to browse within a narrow range of shelves where they have successfully found books in the past; their second choice is to ask a librarian for help. Catalogs of any design run a poor third."

    "The younger children, the 10-year-olds, prefer" systems which do not rely "on correct spelling and keyboard skills." The older children who were skilled on Orion, liked its "multiple points of access" and preferred it to the "multiple screens and metaphorical design" of the Science Library Catalog.

    The researchers came to the conclusion that "the SLC approach overcomes problems with several searching features that are difficult for children in typical keyword OPAC systems: typing skills, spelling, vocabulary, and Boolean logic."

    For future research, the writers suggest similar studies to their own with a larger number of search topics, embedding keyword searching, spelling correction facilities and rank-ordered results. Unobserved studies are suggested too, to more accurately measure persistence.

    This study implies that younger children may not have the skills needed to explore OPACs typically used by adults.

Some Comments on the Borgman Paper - paper #3 continued

    The implications of the Borgman paper on children’s searching behavior also expands into the adult domain. The writers of the Borgman paper even mention "Kay’s (1991) axiom that the measure of any computer system is whether it can be used by children." The other research reviewed here implies that children are not the only ones who are having troubles using OPACs. Some of the other studies show that average OPAC users may not be able to find what they are looking for even 50% of
the time.

    The Borgman paper has some very good characteristics. It takes the time to actually define OPACs (p. 664). The other papers assume that the audience is familiar with the term. They give a good description of the catalog system they designed. The explanation of the differences between the 4 research situations was concise. They use their charts and statistics very well.

    Some of the problems with the Borgman paper include the mentioning of a "ceiling effect" in their results without explaining the term.
 

An Overview of the Sit Paper on Older Adult Users (1998) - paper #4

Sit, Richard A. "Online Library Catalog Search Performance by Older Adult Users." Library & Information Science Research, 20, no. 2 (1998): 115-131.

    Richard Sit conducted a study at the Torrance, California, Public Library. The purpose of the study was to examine the search behavior of older adult OPAC users and evaluate the errors. Sit cited many previous research projects, only one of which also studied older adults and OPACs (Mead, Sit, Rogers, Jamieson, & Rousseau, 1997).  The other works cited were studies on younger adult catalog users which discussed such topics as reducing and increasing results, Boolean logic, complex command
syntax, selection of appropriate database fields and keywords, use of multiple databases, error recovery, and information comprehension in displays, among others. Sit repeatedly refers to Borgman’s 1996 work and a "three-layer framework of knowledge needed for successful online library catalog searching."

    Sit’s hypothesis was that older adults have more troubles understanding and using computers in general than younger adults and that this would effect their ability to use on-line library catalogs. He selected 54 library patrons over the age of 50, all of whom visited the Torrance Library on a regular basis, frequently used a computer elsewhere, and were familiar with the on-line catalog at the Torrance Library (and, in some cases, other OPACs). Participants were orally interviewed (screening questions), given demographic and relevant experience questionnaires, asked to perform a set of nine on-line search tasks, and then given exit interviews. 26 of the 54 search sessions provided transaction logs. For the other 28, data was collected from their answer sheets.

    Sit breaks the findings down into overall search performance, performance of simple search functions, and performance of advanced search functions. He also analyzed the errors based on Borgman’s Categories of Online Search Errors. The highest success rate (100%) was for author, subject, or title searching (considered a "Simple Search Function"). The lowest success rate (27.8%) was for keyword searching (an advanced search task). Sit also mentions that participants in this study indicated a preference for printed instructions (61%) as a source of help when using the on-line catalog system. 59% commonly figure it out on their own, 52% seek on-line help, 44% a library staff member, and 15% a nearby patron or friend.

    From these findings, Sit comes to three conclusions. His first conclusion is that "many experienced older adult online library catalog users remain permanent novices"..."Second, older adults experienced serious online library catalog searching problems"..."Third, online library catalogs may not be accessible to a large portion of the older adult library population."

    For future research, Sit suggests more studies which look into older adults’ searching behavior. With this information, improvements of OPAC systems and training can be made to benefit all users.

Some Comments on the Sit Paper - paper #4 continued

    Sit realized the group he studied was not representative of the general population of "older adults." They were all healthy and most were "highly-educated, Caucasian, English-speaking" men. They were not even representative of the community where they lived. He brings up these issues and the population statistics on pages 119 to 120 in his paper. He goes on to say that if these 54 people have troubles searching on-line catalogs, then imagine what the average older adult would experience. Basically, he was jumping to the conclusion that his group of 54 people could be expected to perform better than the average in their age bracket. He could be wrong though. Perhaps 90-year-old Spanish speaking women would naturally grasp the concepts of on-line catalogs better than Sit’s group.

    The screening interview used in this study was intended to find subjects who met certain criteria, yet "six participants’ data were excluded from the study" after the study began. "Four were found to be novice" users, one was under 50, and one was not a "proficient English speaker." This brings up potential doubts as to whether the design of more of the questions was faulty or misunderstood by participants.

    This is the fourth paper reviewed here, and the fourth to mention technical problems which occurred during the experiment. Perhaps OPACs should be studied for the frequency and impact of technical problems.

    The layout of this paper was very helpful and clear. Compared to the other papers, Sit does a good job of sticking to the rules of academic paper structure.
 

An Overview of the Ferl & Millsap Paper on a Transaction Log Study (1996). - paper #5

Ferl, Terry Ellen, and Larry Millsap. "The Knuckle-Cracker’s Dilemma: A Transaction Log Study of OPAC Subject Searching." Information Technology and Libraries 15 (1996): 81-98.

    "The Knuckle-Cracker’s Dilemma" is a research survey and transaction analysis of OPAC subject searching performed by in-library users at the University of California, Santa Cruz for May 16-22, 1994.

    Ferl and Millsap conducted a similar study on dial-in OPAC transactions in 1991 and found that a higher proportion of undergraduates are likely to attempt subject searches than other academic library users. The failures of the dial-in subject searches in the 1991 study were the impetus for this study. Ferl and Millsap’s hypothesis was that a larger percentage of undergraduates use the library catalog in-library and that an in-library study of these users would provide more information on subject searching behavior.
 

    The study methods used for this study included the MELVYL Library OPAC which had been use since 1991. An on-line questionnaire, transaction logs, and some human observation were utilized to collect the information from 24 survey terminals located in high use areas of two libraries (the McHenry Library and the Science Library). The human observers were there only during certain times to help determine boundaries of individual user sessions.

    The findings of this study included some unexpected behavior on the part of the subjects. Many in-library users formally ended their sessions, which was not required. The questionnaire was presented to 3,407 users, completed by 667 users (approximately 20%), and a total number of 620 sessions were analyzed. 222 of those sessions were coded for subject searching. Approximately 77% of the in-library users were undergraduates (51.3% of the dial-in study had been undergraduates). The in-library sessions averaged to be twice as long as the remote user sessions.

    The searches themselves provided some useful data. "The occurrence of zero retrievals was the second most frequent phenomenon for in-library users." The common user mistakes included "illegal truncation, typographical errors, searching in inappropriate databases, and use of the wrong index term." Spelling errors had not been important in the remote user study, but in the in-library study, "misspellings were frequently the cause of search failures." The paper shares several examples of individual transaction logs and creates several tables of searcher’s errors.

    Ferl and Millsap found that many of the searchers were very persistent, trying several different ways to find pertinent items. "Over one-quarter of the users were willing to continue the search for their initial subject through ten or more attempts." Eventually, "nearly three-quarters of the users obtained useful citations after persisting" but "at least one-quarter of these searchers did not appear to retrieve useful citations." One unsuccessful attempt included a subject search combining the terms arthritis and knuckle cracking.

    Ferl and Millsap come to the conclusion that OPAC users should "be provided with more vigorous contextual assistance by the system."

Some Comments on the Ferl & Millsap Paper - paper #5 continued.

    Ferl and Millsap provide a sample of the welcome screens and the online questionnaire. They also provide the coding forms, but do not explain the various databases, which makes some of the information useless.

    Towards the end of the paper, Ferl and Millsap mention two articles: Kaske (1993) and Kurth (1993), which discuss the "complexities of conducting transaction log analysis and the limitations of findings in these studies." By mentioning studies that might prove their oun to be faulty, Ferl and Millsap were able to address teh controversy head on. The other papers avoided such controversy.
 

Summary: Looking at the Five Papers and the Questions to Ask One More Time

Framework Questions: Problem Statements, Definitions, and Literature Reviews

    The only paper which breached the expected neutrality of a good research paper was the Drabenstott paper. The writers wandered out of the objectivity needed for good research and made statements that were not necessarily part of this particular study.

    Looking at the reference bibliographies of these five papers, Christine Borgman’s papers (1986, 1989, 1990, 1991, 1992, or 1996) are mentioned in 3 of the 5 (her own included). Karen Drabenstott’s research is cited in 2 (her own included). Ferl’s research (1992, 1993, and 1996) is only mentioned in his own paper. Sit (1998) was the newcomer to OPAC studies and so could not be cited by the others. Wiberley (1988, 1990, and 1995) references his own studies and is cited by two of the other papers. Other names commonly mentioned include C. R. Hildreth, Karen Markey, and Joseph Matthews. Even though these papers varied in their emphasis, they all found certain studies to be important enough to cite.

    Most of the papers only cited studies which agreed with its own. The only noted exception was the Ferl paper that addressed the controversy over transaction log analysis. As far as the Framework questions go, this would be a strong mark against those studies who used transaction analysis as part of their methods, yet failed to address the opposing viewpoint.

Method Questions: Observations, Measurements, Control, and Generalities

    The observations made by each of these studies are indeed measuring their little part of the world. Those who studied the academic libraries were not concerned with how children use OPACs. Those who studied children’s search strategies were not interested in why Professors tend not to do subject searches. The studies were so specific that their measurements seem to measure only one small part of the OPAC world.

    The academic library OPAC studies (Drabenstott, Ferl, and Wiberley) tend to have users search for their own items. The two studies done outside of the academic realm (Borgman and Sit) had prescribed search topics that the users were asked to perform. The results may have been altered simply by this aspect of the method . For the two studies conducted outside of academic libraries, both mentioned that the users may have persisted longer in an attempt to do well on the prescribed searches.

   The academic library studies had no way to measure motivation. Any user they were monitoring could have been wasting time while their spouse was doing research. Abandoning a search under these circumstances would not be indicative of OPAC overload, yet could easily be read as such.

    All of the studies used transaction logs for their observations. Some of the studies used human observers too. Those who also used human observers mentioned that the transaction log was used for exact timing of a search session. A combination of transaction logs and human observation may be more reliable than either alone. Library searchers will often get a call number they were looking for and leave the search station to go find the item. In their absence, a waiting user may immediately sit down and begin their own search. Transaction logs would not be able to distinguish between the two users. Another problem came up in all of the papers. The transaction logs of many of the sessions were lost for various technology problems. The method of using both the transaction log and a human observer may be more reliable in the long run.

Results/Conclusions Questions: Fair Summaries, Interpretations, and Relationships

    None of the studies done at the academic libraries touched on the reliability or the representativeness of their their statistics. The older user study and the children’s searching study both indicate that their studies measured the extreme sections of the general population, but also point out that if the users they studied have troubles with subject searching, this in itself is significant. The Borgman paper studied children who were ethnically diverse. These children had trouble finding items by subject searches and Borgman points out that this is a reasonable measure of a computer system. Richard Sit claims that if well-educated older adult library users who frequent their library have troubles, then other older adult users are even more likely to fail with subject searches.

    The Drabenstott study was the only paper which viewed the lack of user perseverance as a fault of the user rather than a problem with the system. Drabenstott herself argues (in other publications) for better subject headings, fewer intervening steps, and advanced technology to overcome the problems encountered in subject searching, yet this research paper almost blames the user for not displaying the items her OPAC system found. This may be a framework violation (the language may be biased). This could also be a results/conclusion problem in that the summary is not fair.

Final Statements:

    Each of these studies assumed that its OPAC was comparable to all other OPACs, when in reality, each OPAC seems to have unique characteristics which make it impossible to truly duplicate exactly a study on any other system. Any given OPAC changes over time, items are constantly being added or dropped. OPACs are dynamic. The very fact that they are dynamic makes them a challenge to study. More importantly, OPACs are a part of librarianship that is here to stay. Our research may help the generations of OPACs to come. Research like the ones reviewed here can help OPACs evolve.

__________________________________________________________________________________________________________

1 Borgman (1995). p. 664.
2 Borgman (1995). p. 663.
3 Borgman (1995). p. 674.
4 Borgman (1995). p. 675.
5 Borgman (1995). p. 663.
6 Sit (1998). p. 118.
7 Sit (1998). p. 125.
8 Sit (1998). p. 126. "TABLE 5."
9 Sit (1998). p. 121.
10 Sit (1998). p. 129.
11 Sit (1998). p. 118.
12 Sit (1998). p. 120.
13 Ferl. (1996). p. 86.
14 Ferl. (1996). p. 85.
15 Ferl. (1996). p. 86.
16 Ferl. (1996). p. 86.
17 Ferl. (1996). p. 90.
18 Ferl. (1996). p. 90.
19 Ferl. (1996). p. 90.
20 Drabenstott. (1996). p. 535

Bibliography

Borgman, Christine L., Sandra G. Hirsh, Virginia A. Walter, et al. "Children’s Searching Behavior on Browsing and Keyword      Online Catalogs: The ScienceLibrary Catalog Project." Journal of the American Society for Information Science 46(1995): 663-684.

Drabenstott, Karen M., and Marjorie S. Weller. "Failure Analysis of Subject Searches in a Test of a New Design for Subject Access to Online Catalogs." Journal of the American Society for Information Science 47 (1996): 519-537.

Ferl, Terry Ellen, and Larry Millsap. "The Knuckle-Cracker’s Dilemma: A Transaction Log Study of OPAC Subject Searching." Information Technology and Libraries 15 (1996): 81-98.

Katzer, Jeffrey, Kenneth H. Cook, and Wayne W. Crouch, eds. Evaluating Information. New York: McGraw Hill. Chapter 17.

Sit, Richard A. "Online Library Catalog Search Performance by Older Adult Users." Library & Information Science Research 20, no. 2 (1998): 115-131.

Wiberley, Stephen E., and Robert Allen Daugherty. "User Persistence in Displaying Online Catalog Postings: LUIS." LibraryResources & Technical Services 39 (1995): 247-264.


 

Home Last modified: April 30, 1999