D-Lib Magazine
May/June 2008

Volume 14 Number 5/6

ISSN 1082-9873

Considering the User Perspective

Research into Usage and Communication of Digital Information

Kellie Snow¹, Bart Ballaux³, Birte Christensen-Dalsgaard², Hans Hofman³, Jens Hofman Hansen²
Perla Innocenti¹, Michael Poltorak Nielsen², Seamus Ross¹, Jørn Thøgersen²

1 Humanities Advanced Technology and Information Institute (HATII), University of Glasgow, UK
2 Statsbiblioteket, Aarhus, Denmark
3 Nationaal Archief, The Hague, The Netherlands

Abstract

In this article we present the methodology and initial results from qualitative research into the usage and communication of digital information. It considers the motivation for the research and the methodologies adopted, including Contextual Design and Cultural Probes. The article describes the preliminary studies conducted to test the approach, highlighting the strengths and limitations of the techniques applied. Finally, it outlines proposals for refinement in subsequent iterations and the future research activities planned. The research is carried out as part of the Planets (Preservation and Long-term Access through NETworked Services) project.

1 Introduction and Background

As the digital evolution becomes infused into everyday life, the ways in which society communicates and uses information are changing. New processes are emerging that were inconceivable in a solely analogue world. National libraries and archives, as the custodians of a society's information, have the responsibility to safeguard these records and to provide sustained access to digital cultural and scientific knowledge. If these organisations are to fulfil these responsibilities, as a community of practitioners we must understand the nature of new communication and usage processes, both to ensure the appraisal process captures the right material and to guarantee that the new kinds of emerging working procedures are supported by the institutions.

The primary aim of user and usage studies is to improve the quality of service. Knowledge about the wants, needs and activities of customers can be employed to change, adapt or update services, or to prioritize management and implementation decisions. A recent trend of user-centred research in archives and libraries has been to focus on two general issues, namely typologies of users, or user groups, and search strategies and techniques, essentially user behaviour (Sundqvist 2007). Introduction of new technologies has accelerated the process of including user studies as one of the basic elements in an overall program of customer service. The sophistication and richness of our understanding of the issues surrounding usage of digital objects and consequent implications for digital preservation requires more research.

Libraries and archives have traditionally been the guardians of analogue communication channels, whether these are scholarly publications housed in a library, or records preserved by an archive. The usual communication channel for scholarly research results, the journal article, has remained a relatively unchanged and consistent form, even with the arrival of digital e-journal versions (Lynch 2007). The digital evolution has since enabled the evolution of whole new processes and techniques for communicating and disseminating results. The emergence of e-government has created new communication channels affecting the business processes between government departments and between them and citizens. Scholarly communication itself is undergoing a transformation, with 'the nature of engagement with and use of scientific literature becoming more complex and diverse, and taking on novel dimensions' (Lynch 2007). Both the social activities of scholarly communication and the unit of information itself have been altered, with data sets, simulations and social networking venues all becoming accepted ways to share and disseminate research (Van de Sompel and Lagoze 2007). These recent transformations have affected the objects libraries and archives have a responsibility for preserving, and in turn affect even what we perceive to be a 'digital object'. Further investigations of these issues are therefore required.

2 Research Focus

Here we report on initial investigations of digital information usage and communication channels aimed at identifying how these factors affect the roles and services libraries, archives and data centres (might) offer. Our research is an aspect of Planets (http://www.planets-project.eu), a digital preservation research and development project co-funded under the Sixth Framework by the European Union. The seventeen institutions engaged in research on the Planets project intend to deliver a sustainable framework to enable long-term preservation of digital content, which will increase Europe's ability to ensure long-term access to its cultural and scientific heritage. Planets will do this through the development of planning, characterisation, and preservation action tools provided through a distributed service network. As part of the project, the usage model and user field studies workpackages are conducting the following research in order to develop an understanding of usage and communication. They are investigating the characteristics of digital object usage and communication in academia and e-government, which will help to influence the development of preservation planning and action components and tools. The research will deliver a deep understanding and appreciation of digital object creation and use as well as how these processes may impact on digital preservation, providing a fundamental level of knowledge in an area which has not been previously utilised. The first studies, which are covered in this article, commenced in spring 2007. The final results of the overall research are scheduled to be published in the summer of 2009.

The research has two main focuses. The first is concentrating on the usage of digital objects in libraries, archives and data centres, investigating the issues that are important to users when working with this information. Data centres are included as a third area to enable a more complete understanding of user requirements for preservation planning. The original and raw data that these centres hold has often been captured at a particular point in history and cannot be reproduced at a later date, effectively making it part of a society's cultural or scientific heritage in much the same way as materials held by libraries and archives. The results of the work will be represented as a conceptual usage model that will provide an apparatus to support the development of the Planets preservation planning tool. The second main research focus is investigating the communication channels being used in scholarly research and e-government, looking at the new processes emerging and how they are shaping what is seen as a 'preservable object'. The work shares a common methodology, which incorporates a number of existing techniques to provide a process tailored to the needs of the project.

3 Methodology

An underlying challenge for the research is to develop a robust and scientifically grounded methodology that provides rich and detailed data on the working habits of users interacting with digital material, and on the communication among researchers to understand the development and exchange of ideas. At the same time, the work has to allow for the eventual development of more specific outputs such as the usage model. A number of research methods, described below, have been considered and combined for the purpose of this project, in order to provide the most effective data collection and analysis techniques.

3.1 Overall Approach: Qualitative Research

Before work on any research project can begin, it is essential to decide on an overall methodological approach to the research. The overriding emphasis of this project is on capturing people's daily work activities and understanding their actions, and for this reason qualitative research is favoured. Qualitative research concentrates on understanding everyday activities, with a focus on behavioural and organisational contexts. It studies people's activities in detail, avoiding preconceived ideas and set questions (traditionally favoured in quantitative research) that may take focus away from the real issues (Esterberg 2002). This approach suits the aims of the research, which wishes to gather information on the ways people use digital objects and their behaviour and activities within a particular context. It aims to identify themes of usage and priorities rather than test pre-conceived ideas.

The small number of studies required for qualitative research has been subject to scepticism, and the relevance and validity of qualitative research is often questioned. It is suggested that the limited number of case studies or field studies is too specific and context-bound for the step to generalisation or application to other settings to be made (Esterberg 2002). It can be argued, however, that user studies, due to their focus on both behaviour and context, 'need both quantitative and qualitative approaches to be combined to produce both the holistic view and the robust data needed to triangulate and thereby validate data collected' (Banwell and Coulson 2004). In order to address this issue, offering a validation of the qualitative results through more quantitative methods, such as questionnaires, will allow the research to combine the advantages of qualitative and quantitative research.

3.2 Methodological Framework: Contextual Design

Although the qualitative approach provides a sound basis for research, the technique can in certain contexts benefit from a more specific form of implementation. Based on the eventual design and modelling requirements for some of the project's results, the Contextual Design process offers a useful solution. Contextual Design is a team-based user-centred design process developed directly from an understanding of how the user works (Beyer and Holtzblatt 1998 and Holtzblatt et al. 2005). It implies that any technological system embodies a way of working, and its functions and structure shape particular strategies, language, and workflow for its users. Its approach helps researchers to address what work is going on in a given environment, the cultural influences shaping that environment, the structure of the work practice, and the types of artefacts used for accomplishing the work.

Contextual Design uses a variety of methods depending on the information needed, but prefers a combination of background interviews and in situ observation. Other techniques such as focus groups can also be used. The method follows a six step process, beginning with the collection of data on customer work practices through interviews and observation (Contextual Inquiry), followed by modelling of the workflow (Work Modelling) and the analysis of results using affinity diagramming (Consolidation). The system is then redesigned (Work Redesign and User Environment Design) and tested with users before implementation (Test with Users). Thus, Contextual Design handles a qualitative research approach with the intention of building a software-based product.

Contextual Design has primarily been used for the design of computer information systems, including hardware and software (Curtis et al. 1999). However, it has also been partly adapted as a usability evaluation method (McDonald et al. 2006), and has been used more recently in the design of digital library systems (Notess 2005). To understand the characteristics of usage and communication that are useful for establishing preservation plans and developing models, the first three Contextual Design steps can be effectively adopted.

3.3 User Selection and Data Collection: Criteria and Cultural Probes

Even though Contextual Design provides a clear process to follow, there are certain steps that provide challenges for this specific project, or require further clarification in order for the research to progress effectively. Clear criteria for selecting users are needed to ensure rigorous procedures and a suitable variety of candidates. The research will be dealing with experts in a variety of subject fields, some areas of which the researchers will have limited knowledge.

A logical step is to enlist the help of experts, or 'key agents', who can provide more details on the discipline and help to identify where interesting participants may be found. To make the qualitative method successful, it is also important to adequately define and delineate its context, including its user focus. In recent user studies, researchers have become more aware of this and have come up with user segmentation, based on use of materials and characteristics of users. Sexton and others (Sexton 2004) have proposed a matrix of archival users that is formed on the basis of three main criteria, namely motivation for usage, research interest, and degree of familiarity with information sources. The research can therefore use these criteria as a basis for offering a structured approach to user selection.

The data collection techniques of qualitative research and Contextual Design pose a logistical problem for the research. The time and resources available will not cover prolonged and lengthy field studies. The 'data probe' concept is a data collection technique that can be used effectively whilst still allowing for the supplementation of observation where necessary. The idea of a data probe is based on the 'Cultural Probe', a term originally coined by Bill Gaver (Gaver et al. 1999). A Cultural Probe in its original sense was devised as a means to get inspirational responses from different communities during a design process. It can consist of various materials such as a camera, postcards with statements, blank photo albums, blank diaries etc., and is completed by the user/community about which one would like to learn more. Through use of the probe, the user generates feedback and inspiration that can be used in a design process. Gaver's original probe concept was meant solely as a source of inspiration, and the output of the probe was not structured, summarised or analysed (Gaver et al. 2004).

Since the introduction of Cultural Probes it has become a widespread practice to adopt a cultural or data probe approach to gain knowledge about various communities and use situations (Graham et al. 2005; Crabtree et al. 2004; Hutchinson et al. 2003 and Akselbo et al. 2006). The approach offers a way to collect data about one or more aspects of a subject's everyday life without direct participation from the investigator. In contrast to an interview where the subject tells the investigator directly about their work, the data probe allows more detailed capture of everyday activities that a participant may omit when recalling their workday. Additionally it encourages users to reflect on their own work through interaction with the probe and its different elements. This approach is therefore ideal in a situation where a significant amount of detail is required, but available resources restrict the length of onsite field studies.

4 Combined Research Process

Based on the methodologies outlined, the following research process has been adopted for the duration of the studies. The technique combines the Contextual Inquiry and Consolidation steps of Contextual Design with the Cultural Probe in order to gather a wealth of information on a user's work habits and communication methods. The various steps of the process are as follows:

Identification of participant (user selection) – the assistance of key agents is enlisted through informal interviews with collection maintainers or experts in relevant subject domains, to gain knowledge of the types of research and potential participants. Users are then approached or selected based on an established matrix of criteria. This ensures the most effective selection as well as diversity across the studies.

Semi-structured initiation and post-probe interviews – these are conducted before the probe is deployed and after it has been collected. The interviews are semi-structured in order to allow interesting points to emerge unique to the interviewee, whilst still ensuring key issues are addressed. The pre-probe interview provides an initial outline of the user and the user's context. The post-probe interview serves as a means for further contextualising the probe data by allowing the exploration of probe materials.

Completion of data probe – the data probe is used to gain information on an individual's daily work habits without the need for direct observation. It consists of the following basic elements:

• diary or logbook to record activities, either in analogue or digital form
• visual recordings of activities such as screen shots, saved files or photographs
• questions or statements issued to the user during the probe period to encourage further thoughts on issues key to the research

The specific probe elements will be refined over the course of the research.

Observation – whilst the probe is in use, short studies of interesting activities are undertaken. These activities are identified during the initiation interview and through continued communication between the participant and researcher.

Affinity diagram analysis – once the probe, observation and interview results have been gathered, the data is analysed using the affinity diagramming technique. This allows researchers to compare user statements and identify common themes in usage and communication across the target cohort.

Validation of findings through workshops, questionnaires and focus groups – the results of the analyses are defined further through a variety of methods in order to verify and validate the preliminary qualitative results. The results will ultimately lead to a conceptual usage model, defined communication channels and revised 'preservable object' definitions. The process is summarised in Figure 1.

Figure 1: Research process for Planets usage model and user field studies workpackages

The methodology for the research therefore combines several techniques that have been successfully used in past experimental studies. Through its unique combination of well established research methods used in a new setting, it is envisaged the work will provide new insights that both answer theoretical and fundamental usage issues in the digital environment, and allow for practical generalisations that can be applied to the project's tools.

5 First Studies

In order to test the agreed methodology and to gain some initial insight into users' thoughts on using digital objects, a pilot iteration of probe studies has been carried out. The three areas of libraries, archives and data centres were studied, with one user engaged from each environment. It was agreed that though some standardisation in the approach would be necessary, each study should be customised according to the specific environment to further test the most effective data collection techniques.

5.1 Approach

The basic approach consisted of locating relevant collections and conducting preliminary discussions with staff from the collection to gain an impression of the various types of users. Based on this information a user group was chosen, and volunteers were approached through a number of methods including contacts provided by collection staff, and advertisements in reading rooms. Suitable individual users were identified, and pre-probe interviews conducted. The chosen collections and users are outlined in Table 1.

The data probe was then refined and deployed. Initial discussions within the workpackage identified key areas essential in the probe design, but flexibility in the customisation and completion of the probe was favoured. A basic model was devised, consisting of the following elements:

• Activity map/logbook to record daily activities
• Statements to which the user should provide responses
• Request to provide daily screenshots of desktop
• Request to save or record any interesting documents and thoughts
• Guide on how to complete the probe

Each partner then tailored the probe components, questions and layout in order to fit the needs of their own specific users. The probe was completed over a period of two to four weeks. Table 1 shows the variations and customisation of the probe by each partner. Once the probe results were collected and viewed, a post-probe interview was carried out, based on the data provided. The information from the probe and interviews was then combined for the analysis phase.

Table 1: Probe elements and variations in first Planets usage study

5.2 Analysis

After completion of the probe period, researchers transformed all the information provided in the diary, screenshots, statements and interviews into short sentences that condensed the information into useable units for further analysis. By doing so, all partners were able to present information in a comparable format, which enabled the affinity diagramming analysis. During a two-day meeting researchers discussed their data, identified possible clusters, and began to plot these themes by posting the condensed statements on a wall. The activity proceeded until all statements were placed on the wall in a random number of groups. Participants then began to allocate titles to the different clusters using coloured post-it notes. During this stage some statements were moved to different groups, or entirely new groups were created. The diagramming was equally not restricted to textual material. Participants brought along the screenshots provided by users and other visual materials, which were also assembled on the wall. The process of analysis led to a final affinity diagram with a variety of statements and materials grouped into themes.

Figure 2: Images of affinity analysis process

5.3 Findings

The affinity analysis identified a number of themes that are significant to users when working with digital information. These themes will act as guidelines for analysis in next iterations where they will be augmented and refined. They can be grouped into the following areas:

• Authenticity
• Provenance
• Access and usability
• Search and selection
• Granularity of data

More significantly, the study also tested the suitability of the methodology used and highlighted areas for improvement. The exercise clearly demonstrated that the general research process was effective, with the qualitative research, Contextual Design and Cultural Probe techniques offering effective and compatible methodologies for gathering and analysing the data required. Qualitative methodologies applied in this first iteration proved to be useful for gaining information on how users of libraries, archives and data centres work with digital objects and an understanding of their priorities. The grounded theoretical framework of Contextual Design validated the approach further, and the aspects of Contextual Design which were applied correctly in the first iteration were effective. The Contextual Inquiry step allowed for the collection of data through a mixture of interviews and 'observation' (via the probe), providing a good balance between information reported in an interview setting, and the more detailed data recorded on a daily basis. The Consolidation step was also useful as a way of collaboratively analysing the data and identifying common themes in a relatively short amount of time.

The semi-structured interview technique was highly successful, as it allowed interviewers to explore individual topics raised by the user whilst still addressing the general themes. The data probe demonstrated its ability to collect detailed and rich information, with the various aspects of the probe all contributing to the gathering of a variety of data. The decision to customise the probes based on individual requirements was an effective way to test the methodology, with strengths and weaknesses identified that can be effectively addressed in future probe designs.

Conversely, the studies also highlighted some areas that require refinement. Although the Contextual Design process was used as a starting point for the research process, the steps adopted were not always fully utilised, or should have been followed more rigorously in order to take maximum advantage of the technique. In particular the Consolidation step concluded at the initial affinity diagramming stage, and it should have continued further to fully test the methodology adopted. A clear matrix of users was not consistently followed for participant selection, and this will be required in future iterations to ensure appropriate types of candidates are chosen. Finally, the decision to test various approaches and customise the probes was carried out at the expense of uniformity across the studies, leading to some challenges in integration and analysis of results. This outcome was somewhat envisaged due to the exploratory nature of the iteration, but customisation should be applied with caution in any subsequent studies.

6 Conclusions and Next Steps

This article has shown the methodological decisions made on research into usage and communication for the Planets project, and provided details of how the chosen approaches have been tested and analysed. The initial studies demonstrated the validity of the approach for gathering data on user work habits, but also a need for further refinement of the techniques to guarantee effective collection and analysis in the remaining research activities. Based on the outcomes of the pilot study, a number of revisions to the methodology will take place to ensure subsequent probes provide data collection techniques that are principally uniform across the different studies/fields. This will allow consistent data gathering and the ability to integrate results effectively. New enhanced probe iterations will include the following refinements:

• Participants will be chosen through a more structured process, with a clear user matrix and communication with experts or 'key agents' to ensure initial identification of user groups specific to the focus area domain.
• Semi-structured interviews will be standardised according to general discussion areas in order to allow for comparable results.
• Probes will consist of a semi-structured diary template, available in both digital and analogue formats, along with statements, screenshots, interesting files and audio recordings.
• In addition to the data probe, short field studies (half to one day) will be carried out to observe any interesting events and activities that occur during the probe period.
• Analysis of data will be more comprehensive according to the Contextual Design steps, and will include interaction with external experts to validate and expand results.
• The number of users and length of probe completion periods will be standardised within each iteration.

Subsequent iterations of the research will build upon the initial findings and conclusions through investigation of a wider variety of users. The qualitative research will then be validated as previously outlined through the use of questionnaires, and eventually through workshops and focus groups where findings will be discussed with a representative sample of users or researchers. The next set of probe studies will concentrate on the communication processes used in e-government and scholarly communication within the science and arts communities. They will integrate the previous findings and revisions, and ensure a robust methodology for collection and analysis of data.

Bibliography

Akselbo, J.L., Arnfred, L., Barfort, S., Bay, G., Christensen, T.B., Hansen, J.H., Jensen, H.T., Markussen, G.B., Morthorst, A.M. and Nielsen, M.P., 2006. The hybrid library: from the users' perspective. Available from: <http://www.statsbiblioteket.dk/om-statsbiblioteket/brugbarhed/feltstudier/feltstudier>. [Accessed 18 March 2008].

Anderson, I., 2004. Are you being served? Historians and the search for primary sources. Archivaria, 58, 81-129.

Banwell, L. and Coulson, G., 2004. Users and user study methodology: the JUBILEE project. Information Research [online], 9 (2). Available from: <http://informationr.net/ir/9-2/paper167.html>. [Accessed 18 March 2008].

Beyer, H. and Holtzblatt, K., 1998. Contextual design. Defining customer-centred systems. San Francisco: Morgan Kaufmann.

Carr, R., 2006. What users want: an academic 'hybrid' library perspective. Ariadne [online], 46. Available from: http://www.ariadne.ac.uk/issue46/carr/ [Accessed 18 March 2008].

Conway, P., 1986. Facts and frameworks: an approach to studying the users of archives. American Archivist, 49, 393-407.

Crabtree, A., Hemmings, T., Rodden, T., Cheverst, K., Clarke, K., Dewsbury, G., Hughes, J., Rouncefield, M., 2003. Designing with care: adapting cultural probes to inform design in sensitive settings. In: Proceedings of the 2004 Australasian Conference on Computer-Human Interaction (OZCHI2004). Brisbane: Ergonomics Society of Australia, 4-13.

Curtis, P., Heiserman, T., Jobusch, D., Notess, M., and Webb, J., 1999. Customer-focused design data in a large, multi-site organization. In: Proceedings of the SIGCHIConference on Human Factors in Computing Systems: the CHI is the limit (Pittsburgh, Pennsylvania, United States, May 15 - 20, 1999). CHI '99. New York: ACM Press, 608-615.

Duff, W., 2002. Understanding the information-seeking behaviour of archival researchers in a digital age: paths, processes and preferences. In: Proceedings of the DLM-Forum 2002. @ccess and preservation of electronic information: best practices and solutions. Barcelona, 6-8 May 2002. Luxembourg: Office for Official Publications of the European Communities, 331-339.

Esterberg, K.G., 2002. Qualitative methods in social research. Boston: McGraw Hill.

Gaver, B., Dunne, T. and Pacenti, E., 1999. Cultural probes. Interactions, 6 (1), 21-29.

Gaver, W., Boucher, A., Pennington, S. and Walker, B., 2004. Cultural probes and the value of uncertainty. Interactions [online], 11 (5) 53-56. Available from: <http://www.equator.ac.uk/index.php?module=uploads&func=download&fileId=209>. [Accessed 18 March 2008].

Graham, C., Cheverst, K. and Rouncefield, M., 2005. Whose probe is it anyway? Workshop on appropriate methods for design in complex and sensitive settings at OzCHI 2005. Available from: <http://www.caside.lancs.ac.uk/publications/whoseprobe.pdf> [Accessed 18 March 2008].

Harley, D., 2007. Why study users? An environmental scan of use and users of digital resources in humanities and social sciences undergraduate education. First Monday, 12 (1). Available from: <http://firstmonday.org/issues/issue12_1/harley/index.html>. [Accessed 18 March 2008].

Harris, C., 2005. Archives users in the digital era: a review of current research trends. Dalhousie Journal of Information & Management [online], 1 (1). Available from: <http://djim.management.dal.ca/issues/issue1_1/harris/index.htm>. [Accessed 18 March 2008].

Holtzblatt, K., Wendell, J.B., and Wood, S., 2005. Rapid Contextual Design: a how-to guide to key techniques for user-centred design. San Francisco: Morgan-Kaufmann.

Hutchinson, H., MacKay, W., Westerlund, B., Bederson, B.B., Druin. A., Plaisant, C., Beaudouin-Lafon, M., Conversy, S., Evans, H., Hansen, H., Roussel, N. and Eiderbäck, B., 2003. Technology probes: inspiring design for and with families. In: Proceedings of the 2003 CHI Conference on Human Factors in Computing Systems (CHI'03). Place of publication, ACM Press, 17-24.

Lynch, C., 2007. The shape of the scientific article in the developing cyberinfrastructure. CTWatch Quarterly [online], 3 (3). Available from: <http://www.ctwatch.org/quarterly/articles/2007/08/the-shape-of-the-scientific-
article-in-the-developing-cyberinfrastructure/>. [Accessed 18 March 2008].

McDonald, S., Monahan, K. and Cockton, G., 2006. Modified contextual design as a field evaluation method. In: Mørch, A., Morgan, K., Bratteteig, T. Ghosh, G. and Svanaes, D., eds. Proceedings of the 4th Nordic Conference on Human-Computer interaction: Changing Roles (Oslo, Norway, October 14 - 18, 2006). NordiCHI '06, 189. New York: ACM Press, 437-440.

Nardi, B. A. and O'Day, V. L., 1998. Information ecologies: using technology with heart. Cambridge, MA: MIT Press.

Neuman, W.L., 1991. Social research methods: qualitative and quantitative approaches. Boston: Allyn and Bacon.

Notess, M., 2005. Using Contextual Design for digital library field studies. Presented at: ACM/IEEE-CS Joint Conference on Digital Libraries workshop. "Studying digital library users in the wild: theories, methods, and analytical approaches", Denver, CO, June 10-11, 2005. Available from: Studying digital library users in the wild, JDCL 2005 workshop report studying digital library users in the wild, D-Lib Magazine, July/August 2005, 11 (7/8), <http://www.dlib.org/dlib/july05/khoo/07_notess.pdf>. [Accessed 18 March 2008].

Preece, J., Rogers, Y. and Sharp, H., 2002. Interaction Design. Beyond human-computer interaction. New York: John Wiley.

PSQG (PUBLIC SERVICES QUALITY GROUP) OF THE NATIONAL COUNCIL ON ARCHIVES, 2006. Survey of visitors to UK archives 2006: national report. Available from: <http://www.ncaonline.org.uk/materials/psqg_national_report_2006.pdf> [Accessed 18 March 2008].

Seadle, M., 2000. Project ethnography: an anthropological approach to assessing digital library services. Library Trends, 49 (2), 370-385. Available from: <http://findarticles.com/p/articles/mi_m1387/is_2_49/ai_72274401/pg_1> [Accessed 18 March 2008].

Sexton, A., Turner, Yeo, G. and Hockey, S., 2004. Understanding users: a prerequisite for developing new technologies. Journal of the Society of Archivists, 25 (1), 33-49.

Siatri, R., 1999. The evolution of user studies. Libri, 49, p.132-41. Available from: <http://www.librijournal.org/pdf/1999-3pp132-141.pdf>. [Accessed 18 March 2008].

Sundqvist, A., (2007). The use of records: a literature review. Archives and Social Studies: A Journal of Interdisciplinary Research, 1 (1), 623-653. Available from: http://socialstudies.cartagena.es/images/PDF/no1/sundqvist_use.pdf [Accessed 18 March 2008].

Van de Sompel, H., Payette, S., Erickson, J., Lagoze, C., 2004. Rethinking scholarly communication: building the system that scholars deserve. D-Lib Magazine [online], 10 (9). Available from: <doi:10.1045/september2004-vandesompel>. [Accessed 18 March 2008].

Van de Sompel, H., Lagoze, C., (2007). Interoperability for the discovery, use and re-use of units of scholarly communication. CTWatch Quarterly [online], 3 (3). Available from: <http://www.ctwatch.org/quarterly/articles/2007/08/interoperability-for-the-discovery
-use-and-re-use-of-units-of-scholarly-communication/7/>. [Accessed 18 March 2008].

Veen, J., 2002. Stalk your user. Dr. Dobb's Journal [online]. Available from: <http://www.ddj.com/dept/architect/184413411>. [Accessed 18 March 2008].

Weinberg, J.B. and Stephen, M.L., 2002. Participatory design in a human-computer interaction course: teaching ethnography methods to computer scientists. In: Proceedings of the 33rd SIGCSE Technical Symposium on Computer Science Education (Cincinnati, Kentucky, February 27 - March 03, 2002). SIGCSE '02. New York: ACM Press, 237-241.

Wildemuth, B.M., 2003. Why conduct user studies? The role of empirical evidence in improving the practice of librarianship. Keynote address at "Fine-Tuning Information Strategies", INFORUM 2003: 9th Conference on Professional Information Resources, Prague, May 27-29, 2003. Available <http://sils.unc.edu/~wildem/INFORUM-2003.Wildemuth.pdf>. [Accessed 18 March 2008].

Wilson, T.D., 2000. Recent trends in user studies: action research and qualitative methods. Information Research [online], 5 (3). Available from: <http://InformationR.net/ir/5-3/paper76.html>. [Accessed 18 March 2008].

Wixon, D. and Ramey, J., eds., 1996. Field methods for software and systems design. New York: John Wiley & Sons.

Top | Contents
Search | Author Index | Title Index | Back Issues
Previous Article | Next Article
Home | E-mail the Editor

D-Lib Magazine Access Terms and Conditions

doi:10.1045/may2008-ross