Knowings in practice: some insights into evidence-based practice implemented by students

Understanding the evidence of practice and creating new knowledge

The assertion that research in library and information studies (LIS) should be related to information work, either shedding new light on aspects of information provision and/or use, or being used to help solve problems of information handling and use, is commonly found in texts and research-oriented publications. Yet, the field of practice has grown to be vast, encompassing the traditional cultural organisations of libraries, museums and art galleries, but also spreading out to include business, government and non-government organisations. Further, in business organisations, information work may not be carried out only in departments concerned with information or records management. Similarly, the field of knowledge has grown in theoretical sophistication and in the breadth of the application of this theoretical knowledge. These changes in the field of practice and in the knowledge-base of information and library studies call into question the relationship between the knowledge base and the practices of the field.

Research-based papers have become more prevalent in the literature of the field. Cronin (2012) acknowledges the importance of research to the field of practice and to the field of knowledge, but at the same time notes that the field of practice is less focussed than before, with many information professionals employed in jobs which do not have information or knowledge in the title. In this context, he is critical of a drift he has identified in research in several aspects of the field, but particularly in user behaviour research; his concern is that many of the studies being published are irrelevant to the practices of information provision, and often of interest only to other researchers.

As the field has grown in theoretical sophistication, there has been an increasing emphasis on the use of evidence in solving problems of professional practice. The phrase evidence-based practice has become increasingly common in the literature, although there has been no agreed definition for evidence-based practice in librarianship (Booth, 2003). Drawing parallels with the practices of medicine and nursing, there has been some emphasis on the role of systematic reviews on key aspects of research into information practice (Eldredge, 2000; Haddow and Klobas, 2004). However, a number of studies of librarians and other information practitioners (see Howlett and Howard, 2015 and Koufogiannakis, 2013 for examples) have shown that the commonly held view of evidence as used in problem solving is very much broader than the systematic review with its origins in the consolidation of scientific knowledge. Whereas Partridge, Edwards and Thorpe (2010) go for a very broad definition: 'an array of approaches used for decision-making' (2010, p. 294), Koufogiannakis (2013) and Howlett and Howard (2015) found that the descriptions of what might be considered evidence could be separated into three categories which Koufogiannakis (2013) labelled local evidence, professional knowledge and research. Local evidence could come from sources such as user feedback, usage data and environmental factors such as the preferences of a senior manager. Professional knowledge included the individual's professional knowledge and experience, knowledge from other professionals in the network, and professional updates in magazines and websites. Research was likely to come from the professional literature, but could also be found in consultancy reports and reports of government projects.

According to Johannsen (2012), reviews of research could be separated into two categories, those concerned with the transfer of ideas and those concerned with interventions in the practice. This may seem to suggest the traditional divide between research as the creation of new knowledge, and problem-solving as the application of what is known to new problems, and it may well be that the reviews are aimed at different audiences. Yet, both approaches focus on the development of new understandings, whether for academics or for practitioners.

The development of knowledge and understandings in areas of professional practice has been an area of concern since the introduction of education for professions, such as teaching, social work, accounting and librarianship among others, into universities. Freidson (1986, pp. 226-227) provided a useful insight into the types of knowledge that exist in an area of professional practice. He identified three types. The first was formal knowledge, the knowledge found in textbooks, the research literature of the field and the literature of professional practice, that is, the knowledge codified and taught in formal programs of professional education. This knowledge is taken by managers and administrators in work-based settings and simplified to be re-presented within an organisation as the one best way. This is an organisational view of the knowledge of professional practice. According to Freidson, individual practitioners take from formal knowledg and from the one best way and apply these knowledges inconsistently and idiosyncratically to the problems of everyday professional practice.

Writing at a similar time, Donald Schön (1983) also took a socially constructed view of the epistemology of professional knowledge, seeing that knowledge as embedded in what individuals do in their professional practice. When practitioners are unsure about some aspect of their everyday practice, they reflect on the problem they face and on what they already know and come to some conclusion on what to do in the context. In that way, which Schön refers to as reflection-in-action, they create a new interpretation of professional knowledge. After a complex process in professional practice has been completed, practitioners will, consciously or sub-consciously, think about what they have achieved, what surprised or puzzled them and what they might have learned as a consequence and do differently in the future. This is known as reflection-on-action.

These two approaches, which were introduced to students in workshops and through readings, exemplify two ways in which new knowledge is created, one based in practice, with a focus on the problem and the workplace, and the other in reflection, with a focus on individuals and their understanding of how the solution to a particular problem extends or changes what is known. In the context of this study, the two approaches gave complementary insights into the processes used to solve the information handling problem of their client organisation and into their perceptions of the ways that these practices created new knowledges for the organisation and for themselves.

The term knowings is used in this study rather than a phrase such as applied knowledge for several reasons. Firstly, in the singular, it suggests active engagement with different knowledge creation processes; and secondly, in the plural, it indicates that knowledge creation and understanding by one individual in a group can lead to knowledge creation and understanding by others, in this context, by the student, for and by the organisation and by the academic staff.

The study and its background

The programmes in information studies and information management offered by the University of Technology Sydney have always placed emphasis on the building of a knowledge base for professional practice (e.g., Yerbury and Kirk, 1990). This has involved giving students an opportunity to bring together their formal learning and their embryonic professional judgement as they develop a solution to an information handling problem. As the field has grown in theoretical sophistication, it follows that students are presented with the formal knowledge of the field at a conceptual level of generalisation, which they are then expected to apply in professional practice.

This study explores how students demonstrate their knowledge of the field through an examination of the capstone subject in the Bachelor of Communication (Information and Media) offered through this university. This requires students to provide a solution, based on the literature, to an information handling problem in a real organisation. The subject assumes, in line with the University's expectations, that the students are using research-based knowledge to solve the problem, and in the process, they are knowingly creating new knowledge both for themselves and for the organisation. The part of the subject concerned with reflection and evaluation uses Schön's notion of reflection-in-action and reflection-on-action to help students come to terms with this notion of new knowledge.

This study takes the problem-solving reports of one cohort of students (thirty-eight students) and analyses them to track the use of theoretical understandings and conceptual models. Through this approach, it seeks to answer questions about how these students have integrated ideas from contemporary literature into their work. These problem-solving reports were required to have four components: a description of the organisation and its context; a statement of the problem; a user analysis drawing from the literature and possibly on other sources; a justification of the solution to the problem, using the literature. There was no set length for the report and no requirements for the number of references included, nor for the field from which they might come.

Approval for the study was granted by the University's Human Research Ethics Committee (ETH16-0725). In the last week of their final semester in 2016, students were told about the research study. Consent forms were distributed and collected by class members while the researcher was out of the room, and handed over in a large envelope at the end of the class. This form sought permission to include the project report in the analysis of the use of research-based knowledge in professional problem solving and explained that neither the identity of the student nor the organisation was relevant to the study. One student's work was covered by a confidentiality agreement that precluded any form of disclosure of the work done. All other students agreed to participate.

The information-handling problems

The information-handling problems the students worked with came from one of three sources: they were identified by the student, they were sourced by the subject coordinator from professional and community organisations or they were posted by the ShopFront, the part of the university that creates a link between the university and the community. Of the projects completed, students were responsible for identifying twenty-two projects, the academic staff sourced fourteen and two students selected a project from the Shopfront's list. The student-initiated projects reflect the experiences and interests of students in this cohort, with the majority of them working in retail or other small business or having the links that made a project in business possible. Only one student initiating her own project has continuing employment in a library.

Within class discussions, students worked together in peer support groups of five or six to identify the problems in information-handling terms and these were recorded in their final report. Although the origins and contexts for the problems were diverse, there were similarities in the problems themselves, indicated in Table 2 below. The analysis of the type of problem expressed in information handling terms gives an interesting insight into the opportunities for information management in workplaces where the primary focus is on commerce, for example, or for consulting to small business.

In this study, businesses employing only a small number of staff were sometimes owned or managed by people with little experience or understanding in establishing information-based activities; in this context, when a new approach was required, such as developing a social media strategy, no one in the organisation had the knowledge to complete the task effectively (lack of knowledge in client n=8). In the past, this lack of expertise has been a feature of volunteer-based organisations (Denison and Johanson, 2007), and although a lack of expertise hindered volunteer-based community groups, the students’ reports suggest that the organisations were able to articulate the problem. While some organisations had no systems in place to manage a particular kind of information or record of transaction (lack of organisation n=8), other organisations had multiple systems for the same content with no reliable way of managing the transaction (lack of standardisation n=8). Several of the problems were in retail, where a business may have had several branches or outlets, each of which had its own way of managing some part of the sales transaction, from categorisation of products to inter-branch communication. Across all major types of organisation, in businesses, libraries and community organisations, the problem was a perceived lack of awareness of services and products by the target audience (n=6 ). Problems with access (n=4) arose from paper-based systems which were only accessible on-site.

Although the problems seemed to fit neatly into six categories (see Table 2), solutions reflected the features of the context, the expectations of the client and to some extent, the imagination of the student and were not tied to the problem as shown in Table 3.

The largest category of solutions is develop technical solution. What brings these together is the identification of an established standard or practice and its implementation or the identification of a process to link diverse sources of data. In some instances, the solution was the first step for an organisation in having a formalised information system, as in the case of a galleries, libraries, archives, and museums (GLAM) institution which wishes to start digitising its collections or a foundation which houses sensitive paper-based information important for policy development which could only be used in the office. The reports on options (n=5) do map to the problem type lack of knowledge in client and mostly (4 out of 6), the marketing plans (including social media marketing) map to the problem type lack of awareness of target audience. The problem of lack of standardisation was approached in several ways, with analysis carried out by two students working in projects in the retail sector suggesting that a training package was appropriate whereas others working on similar problems in the same sector put the emphasis on developing and documenting standardised processes to be implemented from above. The solutions involving mobile data involved e-commerce, a shared online diary and access to hard copy headquarters documents from the field.

Knowings in practice

The ways that these students used what they might know to develop and implement solutions to the information-handling problems of their organisation can be identified from three aspects of the reports they wrote. Firstly, the literature that they drew on for the compulsory user analysis and for the discussion of the information design approaches they used in their solution and its documentation, secondly the kind of evidence they used to support the solution they developed, and finally the approach to implementation and the basis of the argument they used to justify the solution. These represent three different kinds of knowing.

Using the literature

Use of literature in the analysis of the user group or the target audience for the solution is stated as a requirement for the satisfactory completion of the report. In previous subjects, students have been introduced to user analysis specifically in two contexts, one being user-centred web design and the other being a conceptual overview of information cultures, where a range of concepts including information behaviour, information need and information use were covered. In the thirty-eight reports, four distinct approaches were evident: firstly, information needs and/or behaviour (n=16 ); secondly, personas (n=15 ) and scenarios (n=2); thirdly, stakeholder analysis (n=4) and target audience analysis (n=3) and finally, a focus specific to the context, such as stroke victims described in rehabilitation literature from health sciences, and their characteristics (n=5). A small number of students (n=7) used multiple approaches, most presenting user behaviour and then developing scenarios. There was little commonality in the literature referred to. The item most commonly referred to was an e-reading found in the university library about creating quality personas posted by a lecturer from the field of design. This item was used by all five students in one of the peer support groups.

The use of the literature of information design was not a requirement for the report; students had to demonstrate an understanding of the principles in presenting their solution and its documentation to their client. Thus, it was not surprising that fourteen of the students made no reference to the use of literature in their report. What was surprising was that eight of the students did refer to the same source: Pettersson’s 2013 e-book entitled Information Design. These students were in three separate peer support groups. As might be expected from thoughtful students, of those who did give references to the literature they used, ten had used material from specialist areas, including visualisation, the design of online documentation, development of learning materials and so on.

There are three noteworthy aspects in the students’ use of literature. Almost all of the references the students listed in their bibliographies, which included material to support or substantiate their solution, were electronic sources: e-readings from subject reading lists, digitised journal articles, e-books, blogs, and Websites. Secondly, there was nothing that could be considered a common source, used by the majority of students, except possibly the book by Pettersson. Thirdly, most of the material they referred to was not from the core of information studies. Some of the students expressed frustration at not finding useful material in the databases they commonly used and took some time to recognise the value of a broadly-based source such as Scholar Google in getting them started on unfamiliar topics such as the management of a health services facility, or the information behaviour of volunteers. Others were more pragmatic, seeking advice from their client or others in the industry they were working in on what sources were considered good.

Using evidence

An analysis of the narratives describing and justifying the chosen solution using the typology of Howlett and Howard (2015) revealed the types of evidence students had used. These are set out in Table 4.

Although some students used more than one source of evidence, almost every student (35 out of 38) had used what Howlett and Howard (2015) refer to as professional knowledge. That is to say, professional updates in magazines, websites and blogs as well as descriptions of professional practice, including instructions and checklists and statements of standards, found in professional journals. Blogs were a popular source of know-how, including the comments fields. Local evidence was used by almost half the students (17 out of 38); this included characteristics of the user group, established processes, policy constraints and the culture of the organisation. Research was used by only a small number of students (n=6), coming from the scholarly literature as well as from professional and government reports. While three students used only this type of evidence, the other three used research in conjunction with local evidence and/or professional knowledge. One third of the students brought together local evidence and professional knowledge. What was surprising was that some students reported interesting and useful information about the organisation and its problem but then did not use what they knew as evidence to support a particular course of action.

Justifying the solution

A final type of knowing used in the analysis was in the approach to the justification of the solution (see Table 5). Overwhelmingly, and not surprisingly, students adopted an approach which is acknowledged either directly or indirectly as an approach which carries authority. It would be possible to say that the majority of the students (31out of 38) set out a justification which said this is the best solution to the problem. However, this statement obscures a clear distinction within the justifications made by these students. This idea of the best solution was premised on one of two approaches: the first (used by 16 students) was that their solution was based on a notion of best practice and the second (used by 15 students) was that their solution was based on professional norms. An argument based on professional norms was usually expressed in terms of how an information professional would approach the problem, whereas an argument based on best practice was likely to focus on the problem and its solution.

Only five students presented a justification that was based on the context of the organisation; in four of these cases, the problem and its context was assumed to be unique and, in the fifth, it was the adaptation of the principles of information provision to the particular setting that was seen to be unique. Two students who worked with the integration of content in different technologies justified their solution with reference to the technologies rather than the users or the content they worked with. The table below gives examples of the approaches to justification.

Creating new knowledges

The reports of the students reflecting on their performance provided little evidence that they believed they were creating new knowledge for themselves or for the organisation they were working with, in spite of the requirement for providing updating documentation or training materials for the organisation and the topics to be covered in the evaluative report. The majority of the reports are procedural descriptions of the identification of the solution and its implementation (‘Once this was all marked into my calendar…’; ‘After the meeting, I went straight to …’), most often from a project management perspective (‘less than stellar time management’). A few students did note aspects they had learned from: ‘This was a big lesson for me’; ‘I had to learn how to become a teacher to my client’; ‘I was constantly asking questions’. From this, it seems that many students did not consider they were creating new knowledge for themselves or for others, although they were often aware of the challenges of putting learning into practice and moving from being a student to being an information professional: ‘I … experienced feelings of doubt of my own abilities as an information professional’; ‘as a professional I have a long way to go’; ‘I began to feel confident in my skills as in this moment I could identify the principles of information seeking and information management that I had learned throughout my degree’.

Exceptions to this were two of the problems with training packages as a solution, where lack of knowledge among staff was acknowledged as a major reason for lack of standardisation in practices. ‘The [training package] is now being used by … management for current and new staff. … Following the implementation of the [training package], the end of day reports have shown far less discrepancies.’ Since the solutions were complete and the projects were finalised, four clients have specifically commented on the new knowledge brought in to their organisation: ‘The concept of a ‘controlled vocabulary’ was entirely new to me … and we will certainly be taking on the idea as we have seen it to be imperative to the future success of the business’; ‘[The] clear and comprehensive set of guidelines for assessing the condition of the films … is particularly useful as many of our volunteers have no experience in assessing films …’; ‘What a ripper! Can't wait to operationalise this!’; ‘[The student’s] plan is way more insightful and helpful [than previous advice] … especially in using SEO [search engine optimisation] and SEM [search engine marketing]’.

New knowledge for educators

The findings in this study may run counter to the expectations of educators and experienced members of the profession. The use of the literature may present the first surprise. While the emphasis on the use of electronic sources is probably no surprise, the fact that there was no source or approach that was used by most of the students based on their previous studies was unexpected, as subject reading lists do highlight some readings as being more important than others. This was an unexpected outcome and further investigation would be needed to explain this finding that students do not recognise core knowledge.

The use of evidence, with its strong emphasis on the use of professional knowledge and relatively weaker emphasis on evidence derived from the local context, can perhaps be explained by the lack of experience that this cohort of students has in working in an information management environment and can be further supported by the low level tasks that most of them undertake in their current casual jobs. In other words, mostly they have not been exposed to strategic thinking and their clients (mostly), as mentioned above did not take a strategic approach to the problem, seeing it mostly as an obstacle to the smooth running of the organisation.

The approach to justification evident in the analysis of the students’ reports is worthy of comment. Students appear to use an organisational approach, approximating Freidson’s (1986) idea of organisational knowledge, encapsulated in the one best way. The students based their decision on the experience of others or on the application of international standards or on what one might see are marketing claims about the way that anyone confronting this kind of problem might be expected to act. However, what Freidson had in mind was the one best way for an organisation, taking into account the particularities of the context of the organisation. Only the five students using local factors in the context to justify their approach took an approach that Freidson would recognise as his category of the one best way. The students focussing on a best practice approach, as noted earlier, were able to make arguments based on the one best way to solve this problem. That raises a question about whether it would be possible to see that, instead of taking an organisational approach, they were using an approach developed from the formal literature of the profession. Again, it seems that this is not the case either. Although they took guidance from documented sources, it would be difficult to consider these documents as the formal literature of the profession. They are written by active practitioners around the world in blogs and newsletters, not by tutors and lecturers and are mostly not taught in university programs. However, they are ubiquitous in searches done in the Internet, perhaps creating a kind of filter bubble (Pariser, 2011), where a search will produce a list of references which has been tailored to match the question, the search habits and interests of the individual. However, perhaps it could be argued that the students whose justification was based on professional norms were using an approach developed from the formal literature of the profession. For some of the students (twelve out of thirty-eight), this argument holds; five of the six students who worked on a problem in the context of a library and one of those in a museum context used the formal literature of the profession. The other six used the literature of the profession to present an argument that experienced professionals in information management would make decisions in a particular way and this is some of the literature that shows that to be the case.

Freidson’s notion that individual practitioners apply what they have learned and interpret the organisation’s policies and guidelines inconsistently and idiosyncratically has been borne out in this study. Students have placed their own individual stamp on the solutions they have developed. Although there are clusters of similarity among the problems, they have pursued quite different solutions and used different literature to support similar arguments.

Educators may also be concerned to see so little emphasis in the students’ reports on Schön’s concepts of reflection and the creation of new knowledge. A capstone project is generally seen as a way to consolidate knowledge gained during a degree program in particular showing some development of the knowledge base of the individual. It is possible that the structuring of the semester around the stages of project management shifted the focus towards the completion of project deliverables and away from the intangibles of changes in the knowledge of the student or the organisation. Taken together with the implicit messages from the majority of clients that the solution was about removing an obstacle to the smooth running of the organisation, this may have shifted the focus away from the intellectual processes involved in solving a problem of professional practice.

The implications of this study raise something of a dilemma for those engaged in the education of future information professionals. There is no clear single interpretation of the findings. On the one hand, there is evidence of transfer of skills into areas not commonly associated with information management and of creativity and imagination in developing and implementing solutions to information-handling problems. The students found their information-handling problems in many settings, mostly outside of the context of libraries, records management, and knowledge management and so on, and they were clearly able to recognise and label these problems of information handling, in terms that would be recognisable to those working in libraries or information management centres in organisations. The solutions were imaginative, based on solutions they found mostly through electronic searches on the Internet.

On the other hand, the formal knowledge of the profession, as defined by Freidson, is not being used systematically. Students themselves (mostly) do not recognise that they are creating new knowledge for themselves and for the organisations whose problems they solved. The organisations, similarly, have not seen the student’s engagement with them as being one of bringing new knowledge into the organisation, but rather one of removing an obstacle. The consequence is that the new knowledge created in these diverse sectors does not flow back into the knowledge-base of the field. If it is recorded at all, internally it becomes part of the operations of the organisation integrating more or less smoothly into existing practices, and externally it is found in a Facebook post or a blog, presented as an individual’s achievement, rather than as the implementation of principles of an established field of professional practice.

Conclusion

The experience of students in this cohort matches Cronin’s (2012) assertion that the field of practice is less focussed than it once was; not only do the clients of these students rarely have information or knowledge in their job titles, but they barely recognise the specialist knowledge and skills of the student as coming from information studies. It is also clear that students are tending to find studies in the field irrelevant in solving the particular problems of practice they are confronted by. However, it is difficult to see how these interventions into a now diffuse field of practice can be recorded in a literature which follows the norms of Western scholarship and which may not be sympathetic to descriptions of the adaptations of core principles of information management to small-scale problems in such diverse aspects of business and community work. Putting that issue to one side, it is equally difficult to consider who should be seen as having the responsibility for documenting these interventions, recording the new knowledge created in the organisation and the impact these solutions might have on a business or on a community-based organisation and its contribution to community well-being. Further discussion of these key issues in the literature could be useful in identifying perceptions in practice and in scholarship on questions of responsibility for leading the debate and also of the broader implications for education for information management in the context of a field of practice lacking focus.

Acknowledgements

Without the engagement of the students in this cohort, the study could not have been undertaken; I acknowledge their support. I acknowledge the input from the anonymous reviewers which has helped to clarify the argument of this paper.

About the author

Hilary Yerbury is Adjunct Professor in Information Studies at the University of Technology Sydney. Her background in European social and political cultures, information management, anthropology and development studies have given her a broad-based approach to the use of information in everyday decision-making and in social change. She has extensive experience in working with young people on development issues. She can be contacted at Hilary.Yerbury@uts.edu.au

Appendices

Briefing notes provided to the students and workshopped in class and in individual consultation sessions

A variety of products may be produced for the client, depending on the nature of the problem and your proposed solution. Drafts of all work produced for the client (products/reports) should be submitted to your supervisor prior to submission to your client. In addition to the project materials, the project report contains two items: the report which builds on the proposal and the documentation which will enable the client to update or continue the solution. The report MUST contain an analysis of the user group/audience for the solution you are proposing. The principles of information design MUST be used in the development of the updating documentation and should be apparent in all of the material you submit.

As well as the project report, you must submit an evaluative report which requires you to reflect on your learning in the project and on its contribution to you as an information professional. In this report, you will identify, develop and apply criteria to assess the process you used in your project and reflect on your own performance as a professional; a project manager and an independent or lifelong learner, using the literature and examples of professional practice as points for comparison. You need to be able to show the literature and other sources of expertise you have used to help you carry out the project and its documentation and how you have been able to develop your professional skills through this project.