The Australian Library Journal

Information literacy milestones: building upon the prior knowledge of first-year students

Jenny Ellis and Fiona Salisbury

Manuscript received August 2004

This is a refereed article

Introduction

'Successful teaching is like a construction site - it is a construction site - on which students build on what they already know.' (Biggs 1999, p72)

In his influential work entitled Teaching for Quality Learning at University (1999), John Biggs repeatedly stresses the importance of planning teaching and learning activities that build upon the existing knowledge base of the student. Using a constructivist framework, Biggs explains that meaning is personal and not simply 'transmitted' from teacher to student, like dubbing an audio-tape' (1999, p13). It is a student-centred process where constructing a knowledge base is 'done not by the teacher as master-builder, but by the students using the materials supplied both by their teacher and by their experience.' (1999, p76) He argues that deep learning and cognitive growth occurs when 'new learning connects with old' or, as he also puts it, 'in the restructuring that occurs when new knowledge becomes connected with what is already known.' (1999, p73). The importance of building upon past experience and an existing knowledge base is not limited to constructivist educational theory though. It is also present in the writings of educationists in the phenomenographic camp (Prosser & Trigwell 1999, chapter 3; Ramsden 2003, p65).

The principle that teachers should assist students to build upon their prior experience applies just as much to 'teaching librarians' as it does to teachers in any other discipline.[1] However when we attempted to apply this principle to teaching information literacy in the Faculty of Arts at the University of Melbourne, we discovered that our understanding of the knowledge and prior experience of students at this university was in fact quite limited. At best it was anecdotal and impressionistic. With this poor starting point, how could we assist students to build upon what they already know? How could we provide appropriate learning activities to support student learning? Building upon earlier research (Salisbury & Ellis 2002), this paper attempts to answer some of these questions through an investigation of the prior library education, information preferences and skills of students enrolled in first year subjects in the Faculty of Arts at the University of Melbourne.

Our aim is to improve teaching practice in the area of information literacy and this includes (but is not limited to) the teaching of information skills. Recently, teaching librarian Mandy Lupton (2002) argued strongly that librarians tend to place too much emphasis on the acquisition of bibliographic skills and that this tendency perpetuates a narrow and limited view of information literacy. Traditional bibliographic instruction, she argued, is therefore inaccurately conflated with information literacy.

We are sympathetic to this view and we freely acknowledge that information-seeking skills are only one part of the broader spectrum of information literacy. Nevertheless, we still feel that the acquisition of library skills is a valid part of the lifelong learning process and critical to becoming an independent and information literate researcher. These are enabling skills that support meaningful research and deep learning.

Since we are often required to teach these skills, there is a case for saying that they should be well taught. We have no doubt that these skills need to be taught as part of a considered and sequential program where information literacy objectives are integrated into the curriculum and taught in context. However before we can design appropriate student-centred programs that support these objectives we need to have a clearer understanding of the prior experience and knowledge of the computer-literate students who are now commencing at this university.

Other researchers have collected data related to the information skills, knowledge and attitudes of students commencing post-secondary education. While these studies do shed light on the prior experience of students, they are not as useful for our purposes as might be expected. Some studies, conducted in the mid-1990s, are somewhat dated and could not be expected to reflect the ICT experience of students now entering university (Geffert & Christensen 1998, p279; Kunkel, Weaver & Cook 1996). Another relevant consideration is that the majority of similar studies were also generated in an American (as opposed to Australian) context (Dunn 2002; Carter 2002; D'Angelo 2001).

Furthermore researchers have often asked different questions of their data. Many studies emanate from an interest in outcomes assessment. These researchers investigated the knowledge base of incoming students so that they could evaluate the impact of subsequent information literacy programs on student learning (Carter 2002, p2; Dunn 2002, p26; D'Angelo 2001, p282; Churkovich & Oughtred 2002; Salisbury & Ellis 2003, p211). Hence their papers usually report upon the impact of the program, the rate of improvement in student learning, or even the effectiveness of the assessment tool. They often do not report upon the prior knowledge of the students in any detail. A huge multi-year study of students commencing in 2000 at the University of California may address this need (Dunn 2002, p29), but to our knowledge, it has not yet been published in full.[2]

Elizabeth Hartmann (2001) at the University of Ballarat, employed a qualitative approach, ascertaining student perceptions of their information literacy needs when they started university. Her team held focus groups for students as they neared the completion of first year to explore their experience of first year information needs and identify gaps in their knowledge. The qualitative findings of this reflective study complement our findings.[3]

The study

This study was conducted in Semester One, 2003 and involved students enrolled in six first year subjects in the Faculty of Arts at the University of Melbourne. The subjects were drawn from the departments of History, Geography, Linguistics and English with Cultural Studies.[4]

There were two methods of data collection:

• A questionnaire of fifteen questions was developed to obtain quantitative data regarding educational background, previous library skills training, student attitudes and preferences for information sources. This was also a test of basic library skills such as determining keywords, truncation, identifying citations and locating items in the library catalogue. The survey was brief so that it could be completed in five to six minutes at the start of a class and did not take up too much class time. It was administered to students at the start of the first information literacy session provided for that subject.
• Reflective journals. One History subject (Medieval World A) asked students to complete reflective journals related to tutorial experience. We received twenty-six examples of student reflections about the library skills workshop. These provided qualitative data as students reflected upon new learning.

This study extends related research conducted by the authors in 2002 (Salisbury & Ellis 2002). The earlier study evaluated different modes of information literacy instruction and included a pre-test of the library skills of 280 first year students. Some of the pre-test questions related to library skills were replicated in the 2003 study for purposes of comparison and validation.

Results

Educational background and previous training

We received 401 responses to the survey instrument from students who had the following educational profile. A total of 346 respondents indicated their educational level was VCE.[5] Twenty-seven already possessed a university degree and twenty-eight students did not respond to this question. The latter group may have included international students and special entry students who did not fit the normal pattern for school leavers. In this paper the VCE level group will be referred to as non-graduates. Those students who did not indicate their educational background are included in all results for the total group, but not in the tabulation of results that are specific to the graduate and non-graduate groups.

We were somewhat surprised to find that twenty-four per cent of the total group reported that they had not received any library training at all before they arrived at university. Fifty-two per cent had received some training at secondary school while the remaining twenty-two per cent had received training at university; university as well as school; or from some other source.

Information preferences

Students were asked to rate their preferred ways of searching for information. They were asked to rate the following search tools in order of preference - internet search engine, library catalogue, journal article database, local library or any other search method that they might care to nominate.

Not surprisingly, sixty-five per cent rated internet search engines as their most preferred way of searching for information whereas only twenty-four per cent rated the library catalogue as most preferred, seven per cent rated the local library as most preferred and four per cent rated the journal database as most preferred. If you combine first and second preferences, then seventy-nine per cent of the total group favoured internet search engines over the other forms of searching for information.

Preferred methods of searching for information were affected by educational background. Although students who already possessed a university degree still exhibited a strong preference for the internet search engine, they were less enthusiastic. Figure 2 indicates that graduates also endorsed a broader range of academic information sources.

Reliability of information sources

Students appear to be well aware that the information from internet search engines is not necessarily reliable. In fact, Table 1 demonstrates that the great majority only rated the reliability of internet information as 3 in a rating of 1-4. However it is interesting to note that this did not affect their preference for this method of searching for information since we have seen that sixty-five per cent of the total group rated the internet as their most preferred search tool.

Figure 1: Search preferences [6]

Figure 2: Graduate search preferences

Figure 3: Non-graduate search preferences

Table 1: Perceived reliability of internet information

The reverse situation is evident in attitudes towards the reliability of information derived from the library catalogue. We have seen that only twenty-four per cent of the total group regarded the library catalogue as their most preferred way of searching for information, yet fifty-five per cent rated this method as providing the most reliable information. In fact ninety per cent of the total group rated the library catalogue as either 1 or 2 in terms of reliability.

Table 2: Perceived reliability of library catalogue information

The situation is even more pronounced regarding attitudes to journal databases. Only four per cent of the total group rated journal databases as their most preferred method for searching for information, yet twenty-four per cent rated the journal database as providing the most reliable information.

Table 3: Perceived reliability of journal article database information

Understanding of key tasks and concepts

Students were asked to perform some of the key tasks required for conducting library research such as identifying keywords, constructing a search strategy, identifying a citation to a journal article and a chapter in a book and then finding them using the library catalogue.

Keywords

Students were asked to underline the keywords in the following essay topic:

Examine the use of suspense in Alfred Hitchcock's films: Psycho and Rear Window

The definition of keywords is subject to interpretation and a lot of latitude was allowed in the assessment of this task. Of the total group, thirty-two per cent were able to identify a suitable number of appropriate key words that would have given a sensible starting point for constructing a search strategy. However, thirty-one per cent underlined too few keywords. For example, underlining of only the words 'Examine' and 'suspense' was not considered sufficient coverage of the topic. Twenty-eight per cent displayed a tendency to underline too many key words in the topic, sometimes underlining almost every word in the sentence. Others underlined instructional words such as 'examine' and 'use of' as well as the key concepts required for searching. Eight per cent of students did not attempt this question.

The results suggest that most students have not considered that search terms may not be coterminous with the key words used for answering the essay question. This was corroborated in a reflective journal entry in which a student commented that she/he 'was not familiar with how to search, using key words, for specific information required by any particular essay question.'

Previous library skills training at school did not appear to help students to complete this task since this group was the least able to perform this task. Only twenty-four per cent of non-graduates who had received training at school were able to identify appropriate keywords. Yet thirty-three per cent of the group who reported no information skills training at all were able to identify the key words in the topic. University education was clearly an advantage here as forty-eight per cent of the group who reported that they had received library skills training at university or elsewhere were able to complete the task satisfactorily.

Boolean searching

About half of these students were unclear about the use of Boolean operators when constructing a search. Forty-nine per cent of the total group reported that they did use Boolean operators to link concepts when searching library databases and fifty-one per cent were actually able to demonstrate the correct placement of the OR operator that was required by the question. 203 students were asked the following question:

If you wanted to search the library catalogue for information about either of Hitchcock's films, Psycho and Rear Window, which connecting word (AND/OR/NOT) would you use to link the concepts?

psycho AND/OR/NOT rear window

This is higher than the result achieved by students in the 2002 study, where only twenty-six per cent of students were able to supply the required Boolean operators. The task was simplified and also re-phrased this year to include the word 'either'. Although this removed possible ambiguity, we are concerned that the automatic grammatical association of either/or may have inflated the results this year.

In the reflective journal entries, many students commented generally about learning more productive search techniques in the library skills workshop. However one student particularly commented that 'effective searching using Boolean techniques was particularly helpful as I haven't had a chance to learn it properly before.'

Truncation

Students were clearly unfamiliar with the concept of truncation. Only thirteen per cent of the total group were able to demonstrate this ability. Previous library skills training made very little difference to the outcomes in this question. For example, forteen per cent of the non-graduates who had received library skills training at school were able to truncate, while eleven per cent of the group who reported no training at all were still able to complete the task. Those who reported further training elsewhere as well as school fared much better with twenty-three per cent able to complete the truncation.

Interpretation of a reading list

We wanted to know whether students were able to identify and locate items that are commonly prescribed on first year reading lists such as journal articles, books or chapters in a book. The questionnaire listed two citations, modified according to the citation style specified for each subject. For example, Question 10 on the questionnaire asked:

Is the following a book or a chapter in a book or a journal article?

White, A (2000) 'Eternal vigilance in Rear Window,' in Alfred Hitchcock's Rear Window. John Belton (ed.), Cambridge: Cambridge University Press, pp158-177.

From a multiple choice list, sixty-three per cent of commencing students were able to identify this citation as a chapter in a book. Similarly, seventy-five per cent were able to correctly select the journal article option. This is consistent with results from the 2002 study that found that seventy-three per cent of students were able to identify a journal article at the start of their university careers. (Salisbury & Ellis 2002, p213)

Students were then asked how they would search for these items in the library catalogue and the results for this task were much less satisfactory. For each item, students were asked the following questions:

If you searched the library catalogue for this item would you perform an author or a title search?

What words would you type in the catalogue search box?

Only twenty-two per cent of the total group would ever have located the journal article using the library catalogue, while thirty-six per cent of the total group would have found the chapter in the book. This corresponds exactly with the 2002 group where only twenty-two per cent were able to demonstrate that they would be able to find a journal using the library catalogue. (Salisbury & Ellis 2002, p213)

Previous library skills training at school did not significantly affect this result. For example, twenty-one per cent of the non-graduates who received library skills training at school were able to locate the journal article using the library catalogue. However, eighteen per cent of the group who reported no training at all were still able to complete the task.

Educational background was an advantage since sixty-three per cent of the graduates would have located the book chapter using the catalogue and forty-four per cent would have located the journal article. However, forty-four per cent is still not a very high level of competence in such a basic research skill.

Discussion

Previous library skills training

University teaching librarians need to be aware that a significant proportion of students may have received no library skills training at all before they commence at university. Even if they have received some library education at school, our findings suggest that this experience is of limited value in equipping students with the searching skills required by first year humanities students. At the very least, school library skills training does not appear to be transferred readily to the university environment. Students with secondary school training did not perform significantly better in any of the tasks than the students who had received no training at all.[7]

This should not be interpreted as a criticism of school experience since the expectations of university research are quite different. For example the secondary curriculum does not generally require independent research using journals. It would be pointless to focus upon skills that are not relevant to the needs of students at that stage of their education. The scale of the libraries is probably also significant since students are probably familiar with the physical location of relevant resources in the school library. In this smaller environment, information-seeking skills may not have the same imperative. In focus groups conducted at the University of Ballarat, Hartmann reported that students did not find their school library experience was helpful at university because of the overwhelming size of the university library (Hartmann 2001).

When students commence university studies we need to assume that they have not had skills training that equips them for the research tasks that they will be required to perform in the course of an Arts degree.

Information preferences

Information preferences were clearly linked to educational background and experience. Both graduates and non-graduates preferred searching the internet above other methods of searching for information. However experience acquired in the course of a university degree doubtless influenced the graduate group who set greater value upon other sources of academic information such as the library catalogue and journal databases. Even so, graduates are still clearly more comfortable using internet search engines.

As far as non-graduate commencing students are concerned, it is tempting to infer a simple relationship between their prior experience and information preferences. In this scenario, we could suggest that students who are new to university studies may not appreciate the value attached to journal articles in academic research therefore leading them to underestimate this source of information. The team conducting focus groups at the University of Ballarat reported that 'the heavy emphasis placed on the use of journal articles came as a surprise to them, as did the means of accessing them...' Conversely these groups also reported that they did expect to use the internet at University (Hartmann 2001, p115; D'Angelo 2001, p286).

However commencing students in this study do not appear to be so naïve about information sources. These students judged information from the library catalogue and journal databases as more reliable than information from the internet, thus indicating that they were both aware of these sources and had some appreciation of their academic value. The determined preference for the internet shows that students are actively exercising informed choice to use the internet instead of more reliable sources. While we might be disturbed at such undue emphasis on internet resources, student suspicion of the reliability of information from the internet is promising, suggesting that they are aware of the need for careful evaluation.

Library research skills

Commencing student preference for the internet over more reliable sources of information may be partially explained by familiarity with the tool. Such familiarity makes the internet a quick and easy choice, especially when library skills are inadequate. Put simply, many participants in this study do not have the basic skills that would enable them to use other search tools effectively. Even if they tried to search catalogues and databases, the chances are that the searches would be unproductive. Most library catalogues and journal databases require some structured searching ability in order to retrieve useful search results. These students do not appear to have requisite skills in these areas. Even if they were handed a reading list by a lecturer, they do not appear to have the knowledge of how to de-code the citations and find the items using the catalogue. Without these basic skills, it is no wonder that they prefer the internet.

By contrast, search engines such as Google do not require structured searches. An internet search usually returns so many hits that it is not necessary to think about keywords, synonyms, and constructing search statements. And, there is little doubt that Google would be a much more familiar and everyday route to information for most students.

In their reflective journals, several students recognised this deficiency in their search techniques and acknowledged the benefit of the library skills workshop in redressing the gaps in their knowledge. For example, one student commented 'I learnt a lot of little things about search requests on search engines that I didn't know before ... and they will certainly come in handy in the future!' Another indicated that the library session was 'most useful as leant new (and far more productive!) ways to search the "net"'. Another described the workshop as 'really helpful' and 'definitely a learning curve!' This was new learning for the students and with only a single exception, they did perceive the activities as worthwhile.

Several students also reflected that this learning could be readily transferred to new learning situations, understanding the relevance of library research skills for other subjects. This is a good example of how these students were able to build upon their new learning. For example, one student commented: 'Today's tutorial was a lesson in research methods for history (and other) essays.' Another described the session as 'Very useful for developing research skills for more than just medieval studies.' Another stated the session was 'not only helpful to history, but to all my other subjects as we were taught how to search various databases and how to access online resources.' And again, 'This will make researching for history as well as my other subjects, much more efficient.' These comments indicate that students were able to put the sessions in context and transfer information between subjects.

The reflective journals also indicate that at least some students made the connection between library research skills and more sophisticated learning tasks. They were aware that these were enabling skills giving them access to a range of interpretations and theories. For example, one commented on the importance of the 'large variety of resources' while another stated:

I chose the King Arthur topic and was amazed at how much information there was as to whether he existed or not. There were so many theories and papers on areas in England that could be related back to the Arthurian legend.

And another commented:

Similarly, how to search for journal articles related to the essay question I choose will also broaden the range of references that I will be able to use to complete my final essay.

Implications for teaching practice

'What people construct from a learning encounter depends upon their motives and intentions, on what they know already and on how they use their prior knowledge' (Biggs 1999, p13).

The perverse preference for the internet at the expense of reliability indicates that students in this study may be consciously choosing to use their knowledge of other academic sources in a limited way. The task for academic and information staff is to devise programs for first year students that will broaden student attitudes towards alternative sources of academic information as well as equip them with appropriate retrieval skills. This deals with motivation as well as skills.

Humanities students are commonly expected to complete research tasks that are based on critical reading of a broad range of materials. Depending on the assignment, this is likely to include scholarly and professional writing as well as perhaps examples from popular culture. Teaching librarians need to emphasise the big picture, the intellectual issues at stake, especially the connection between critical reading of a broad range of material and producing thoughtful and scholarly essays.

Even though students in this study exhibit a potentially unhealthy reliance on the internet, which we may consider narrow and limiting, it is still possible to acknowledge and carefully build upon this prior experience. Biggs (1999) describes teachers who tell students to forget what they already know as 'arrogantly anti-intellectual'. Instead he argues that teachers must build on the known, not reject it. He exhorts good teachers to 'choose familiar examples first, get students to build upon their own experience, draw and explain parallels while teaching, use cross-references.' (p73-4). He points out that such strategies help students to draw connections between old and new learning.

This appears to be good advice for teaching librarians since the reflective journal entries indicate that many students in this study have made connections between old and new learning in terms of their prior internet searching experience. This is most evident in their use of language for many students used the language of the web to describe their new learning. For example, one student described the session as learning a 'lot of little things about search requests on search engines'. Another student stated 'In today's tutorial we learnt how to use the internet'; another reflected that the class learned 'new ways to search the "net"'. Another student described journal indexes as 'browsers on the internet'.

This language was not generally used by the librarian during the session and it is a clear pointer to the way in which the students structured this new learning into their existing knowledge base. Teaching librarians would probably find it useful to start with internet searching examples and consciously build upon this experience, adapting the language of the web to assist students to connect old learning with new. This would produce a more student-centred program because it reflects how the students appear to learn.

In conclusion, this study indicates that students commencing an Arts degree at the University of Melbourne do not have the library skills required for basic university research. There is no doubt that the absence of these skills would adversely affect the quality of essays and probably contribute to over-reliance on the internet at the expense of other sources of information. There is a need for library skills programs that equip students with appropriate information seeking skills for university research.

However these skills programs should be presented as part of a coherent approach to the need for information within a discipline where students are encouraged to broaden their attitudes to other sources of information. The reflective journal entries confirm that the students appreciated the library skills workshops and that they were able to discern the value of these skills as part of the research process. The entries indicate that many students readily transferred this learning to other contexts and that they understood the ways in which these skills could add to their research tasks.

If we are to assist students to build upon their prior experience and existing knowledge base, then this study has a clear message. Students do have prior information-seeking preferences and experience and their clear preference rests with internet searching. The preference is so strong that it overrides other sources of academic information that they know are more reliable. If we are to assist students to broaden their attitudes towards other sources of academic information as well as equip them with the necessary searching skills, then we need to acknowledge their experience with the internet. We need to further investigate the nature of this experience and then frame programs that build upon this experience as the initial reference point for information literacy programs at university level.

Endnotes

1. This term was coined by Judith Peacock (2001).
   
   
2. A URL was given for further information but we have been unable to connect to this page. See Dunn (2002, p35) Note 14.
   
   
3. Another study by Seamans was also based on extensive qualitative data, probing the information seeking habits of nine freshmen entering university in Virginia. However both Seamans' and Hartmann's studies were based on small samples from which generalisation is difficult although they do yield useful insights into student information seeking habits and attitudes to information.
   
   
4. The authors acknowledge and thank staff and students enrolled in the following subjects in Semester one, 2003 who participated in this study - Medieval World A; Screen Writing History: History on Film; Australian Colonial: Claiming a Land; Environmental Change; Contemporary Culture and Media; Introduction to Language.
   
   
5. The Victorian Certificate of Education (VCE) is obtained following successful completion of secondary school in Victoria, Australia.
   
   
6. The total group is made up of all respondents including those who did not indicate their educational background. Respondents who did not indicate their educational background are not included in Figures 2 and 3.
   
   
7. This was also found by Geffert and Christensen (1998).

References

Biggs, John B. (1999). Teaching for quality learning at university: what the student does. Society for Research into Higher Education: Open University Press, Buckingham.

Carter, Elizabeth (2002). 'Doing the best you can with what you have: lessons learned from outcomes assessment' Journal of Academic Librarianship vol 28 nº 1-2, p36-41.

Churkovich, Marion and Christine Oughtred (2002). 'Can an online tutorial pass the test for library instruction? An evaluation and comparison of library skills instruction methods for first year students at Deakin University.' Australian Academic and Research Libraries vol 33 nº 1, p25-39.

D'Angelo, Barbara (2001). 'Integrating and assessing information literacy competencies in a gateway course', Reference Services Review vol 29 nº 4, p282-293.

Dunn, Kathleen (2002). 'Assessing information literacy skills in the California State University: a progress report' Journal of Academic Librarianship vol 28 nº 1, p26-35.

Geffert, Bryn and Beth Christensen (1998). 'Things they carry. Attitudes toward, opinions about and knowledge of libraries and research among incoming college students' Reference and User Services Quarterly vol 37 nº 3, p279-285.

Hartmann, Elizabeth (2001). 'Understandings of information literacy: the perceptions of first year undergraduate students at the University of Ballarat' Australian Academic and Research Libraries vol 32 nº 2, p110-122.

Kunkel, Lilith, Susan Weaver and Kim Cook (1996). 'What do they know?: an assessment of undergraduate library skills' Journal of Academic Librarianship vol 22 nº 6, p430-434.

Lupton, Mandy (2002) 'The getting of wisdom: reflections of a teaching librarian' Australian Academic and Research Libraries vol 33 nº 2, p75-86.

Peacock, Judith (2001). 'Teaching skills for teaching librarians: Postcards from the edge of the educational paradigm' Australian Academic and Research Libraries vol 32 nº 1, p26-42.

Prosser, Michael and Keith Trigwell (1999). Understanding learning and teaching: the experience in higher education. Society for Research into Higher Education & Open University Press, Buckingham [England]; Philadelphia, PA.

Ramsden, Paul (2003). Learning to teach in higher education. Routledge, London.

Salisbury, Fiona and Jenny Ellis (2003). 'Online and face-to-face: evaluating methods for teaching information literacy skills to undergraduate arts students' Library Review vol 52 nº 5, p209-217.

Seamans, Nancy (2002). 'Student perceptions of information literacy: Insights for librarians' Reference Services Review vol 30 nº 2, p112-123.

Biographical information

Jenny Ellis is the senior team member, Arts Learning Resources Team in the Teaching Learning and Research Department at the University of Melbourne. The Arts Learning Resources team is responsible for providing information literacy programs to the Faculty of Arts. She is currently on secondment to the Arts Faculty. Phone 03 8344 5243, e-mail: j.ellis@unimelb.edu.au.

Fiona Salisbury is the director of the Academic Centre at St Mary's College and Newman College at the University of Melbourne. Until recently she was team leader, Arts Learning Resources Team in the Teaching Learning and Research Department at the University of Melbourne. Phone 03 9342 1614, e-mail: fiona.salisbury@academiccentre.stmarys.newman.unimelb.edu.au