Key Concepts Blog: Data Collection Analysis Readings

Key Concepts Blog: Data Collection Analysis Readings 

Powell Ch. 9 

Statistical analysis, or “statistics” is concerned with the development and application methods and techniques for organizing and analyzing data (usually quantitative) so that the reliability of conclusions based on the data may be evaluated objectively in terms of probability. 

Rules and guidelines for the establishment of categories of data:

• The set of categories or values for any one variable should be derived from a single classificatory principle, which is determined by the research question of hypothesis being investigated.
• Each set of categories should be exhaustive.
• The categories within each set should be mutually exclusive.
• The development of categories should be based on a sound knowledge of the subject matter and an anticipation of likely responses. 

Wildemuth Ch.29-37 

29

Content analysis – the systematic, objective, quantitative analysis of message characteristics.

1. Choose sampling units (Ensure sample is unbiased)
2. Define recording units to be analyzed 
3. Develop a coding scheme

30

Qualitative analysis of content - A research method for the subjective interpretation of the content of the text data through the systematic classification process of coding and identifying themes or patterns.

Approaches to qualitative content analysis:

• Conventional qualitative content analysis
• Directed content analysis
• Summative content analysis

31

Discourse analysis, as a method, can be compared to a variety of other method, including content analysis. However, it assumes that the role of speech is to construct the speakers’ social world.

Conversation analysis – is generally limited to the analysis of conversations.

Hermeneutics – to identify the horizons that surround and define a particular phenomenon and to describe the interpretive community or communities for which it is meaningful.

32

Analytic induction – is a specific form of inductive reasoning used to analyze qualitative data.

Induction – the logical process of arriving at a general conclusion by examining a set of facts.

Deductive reasoning – the logical process of arriving at specific facts that must be true, given the broader tenets that are assumed to be true.

33

Descriptive statistics:

Measures of central tendency – focuses on how the values of a particular together.

Measures of dispersion – focuses on how the values of a particular variable spread apart.

34

Frequency distribution – a table used to analyze nominal data and organize cases of categories of variables.

Cross-tabulation – two-way table with the categories of one variable shown as rows and the categories of the second variable shown as columns. 

Chi-square statistic – measures the difference between what was observed and what would be expected in the general population.

35

Analyzing sequences of events

Questions about sequences of events:

• Questions about the patterns of the sequences 
• Questions about the independent variables that affect those patterns
• Questions about the dependent variables 

36

Correlation - a statistical analysis method that helps you examine the relationship between two variables. Specifically it is the proportion of the variability in one variable that is explained by the variability in the other variable.

Direction of the relationship is indicated by the sign of the correlation statistic.

37

Comparing Means: t Tests and Analysis of Variance

t Test – gives the probability that the difference between two large means can be attributed to chance.

ANOVA – extends the t test to situations where you have more than two groups to compare.

Key Concepts Blog: Data Collection Techniques Readings

Powell 5

Data collection techniques: Questionnaire, Interview and Observation.
Questionnaires and interviews are frequently used for obtaining information about a person’s perceptions, beliefs, attitudes and so on. With these techniques, heavy reliance is places on the respondents report; normally the investigator has not observed the events in question.

Observational methods are best suited for describing and understanding behavior as it occurs. They are less effective for gathering information about a person’s perceptions, beliefs, attitudes and so on.

Wildemuth (chpt 18-28)

18
Transaction Log analysis is a useful tool in understanding people’s information behaviors.
Strength – this method allows the researcher to capture and analyze large amounts of data and is a reasonably accurate representation of system users’ behaviors.
Weakness – it cannot tell us anything about users’ cognitive of affective responses during the system interaction.

19
Think-aloud Protocols are a research method used to understand the subjects cognitive processes based on their verbal report of their thoughts during experiments.
Strength – compared to other data collection methods, the process of collecting think–aloud protocols is simple.
Weakness – you will be able to collect valid data only if your study is designed well.

20
Direct observations – An observer present at the site being observed is still quite frequently used, but automatic methods for capturing some types of behaviors are also becoming more common.
Strength – observation methods can be used for a wide range of research questions.
Strength – As long as we are interested in people’s information behaviors, direct observation will be a useful research method.

21
Participant observation – directly observing information behaviors in the context in which they “naturally” occur.
1.    The researcher is a participant in the setting.
2.    Such participation leads to a better understanding of the people and social processes that occur within that setting.
3.    This understanding can lead to better theories about social processes in that setting and similar settings.

22
Research Diaries – Diary study methods comprise a diverse set of techniques and approaches that can be used for exploratory, in-depth, and theory-building research.
Strength – through these methods, researchers are able to gain entry to places they might not otherwise be able to go such as personal homes, the minds of individuals, and geographically dispersed locations.
Weakness – a significant amount of time and commitment can be required from participants and researchers.

23
Unstructured Interviews
Strength – most useful when you want to gain an in-depth understanding of a particular phenomenon within a particular cultural context.
Weakness – unstructured interviews are not useful when you already have a basic understanding of a phenomenon and you want to pursue particular aspects of it.
Also known as: information conversation interview, in-depth interview, nonstandardized interview, and ethnographic interview.

24
Semistructured Interviews:
•    Has predetermined questions, but the order can be modified based on the interviewer’s perception of what seems most appropriate.
•    Question wording can be changed and explanations given .
•    Particular questions which seem inappropriate with a particular interviewee can be omitted, or additional ones included.
Strength – semistructured interviews are one of the most useful data collection methods for studying a wide range of information behaviors.

25
Focus groups – a group of individuals selected and assembled by researchers to discuss and comment on, from personal experience, the topic that is the subject of the research.

Strength – focus groups can be a good choice for data collection if you believe that value will be obtained in the participants’ interactions with each other, rather than individuals responding to questions from the researcher.

26
Survey research
1.    Design the survey instrument carefully, including both the individual items and how they are organized into the survey as a whole.
2.    Carefully select a sample and use a variety of methods to ensure that your sample responds to the survey.
3.    Administer the survey in a form that is accessible to and comfortable for the sample members.

27
Measuring cognitive and affective variables
For research findings in ILS to form a strong foundation for improving practice, any measurements that are taken must be reliable and valid. This is particularly important when we want to measure the constructs that are not directly observable such as a library user’s satisfaction or a Web user’s cognitive style. The most efficient way to ensure our measures are reliable and valid is to use existing measures that have been implemented and evaluated in past studies.

28
Developing New Measures
When an appropriate instrument is not available when beginning a new study, it will be worth the time and effort required to develop a new instrument and to make it available to other researchers. While such research efforts are oriented toward methodical issues, rather than other types of research questions, they are of great importance to research and evaluation in ILS.

Article Review 2

Stephanie Glass
10/07/11
Article Review 2

Flavián, C., Guinalíu, M., Gurrea, R. (2006). The role played by perceived usability, satisfaction and consumer trust on website loyalty. Information & Management, 43 (8), pp 1-14.

Introduction
Researchers performed a study to determine the influence that perceived usability has on the user’s loyalty to websites that they visit. This study on website usability reflects my own interest in website design and usability.

Problem Statement
For this study there was a six part hypothesis based on the how perceived usability affected users. The positive influence of usability on trust, satisfaction and loyalty were assessed as part of a questionnaire.

Literature Review
The role played by perceived usability, satisfaction and consumer trust on website loyalty was influenced by prior studies. According to the conductors of the study, they were influenced by research on consumer’s perceptions of internet establishments. They were also affected by research on website attributes that influence the conduct of potential buyers. Based on these studies, Flavián, et al., believe that “perceived website usability” was an important part of a store’s image can influence shopping behavior (2006).

Method
The questionnaire, measuring variables using a 1-7 point Likert scale, was first pre-tested with a group of 30 users in order to ensure that the questions we understood correctly. The refined questionnaire was then made available to internet users. The valid data from 351 questionnaires was collected and analyzed. Users who completed the questionnaire were “very similar to the average Internet user”( Flavián et al., 2006).

Caveats:
There was one issue that was noted as a limitation for the study. The majority of the participants were Spanish speakers between the ages of 25 and 34, were highly educated, and had more than 5 years experience with computers. The conductors of the survey were uncertain of the reliability of results across nationalities. I would extend that statement to include representations of different generations (i.e. ages 18-65). I also feel that accurate usability tests should display results for all users, not just for those who are highly educated and experienced with the internet.

Key Concepts: Methods/Ethics/ Theory Readings

Powell 3 & 4

Case Study - a specific field or qualitative method and thus is an investigation of phenomena as they occur without any significant intervention of the investigators. It seems to be appropriate for investigating phenomena when :

1. a large variety of factors and relationships are included
2. no basic laws exist to determine which factors and relationships are important
3. when the factors and relationships can be directly observed

Bibliometrics - a special ype of documentary research or inquiry into the tools of library and information science.

Content Analysis - analysis of the manifest and latent content of a body of comminicated material (as a book or film) through a classification, tabulation, and evaluation of its key symbols and themes in order to ascertain its meaning and probable effect.

Survey - a group of research methods commonly used to determine the present status of a given phenomenon.

Types of survey studies:

• Exploratory Surveys
• literature surveys
• experience surveys
• analysis of "insight-stimulating" examples
• Analytical and Descriptive Surveys 
• Other Types of Surveys
• cross-sectional study
• trend study
• cohort study
• panel study
• approximation of a longitudinal study
• parallel samples study
• contextual study
• sociometric study
• critical incident study

Wildemuth 6

Middle-range theories - theories at a middle point on the continuum between special theories, which are applicable only to a very small range of phenomena, and general theories, which are highly abstract and applicable to a broad range of phenomena.

Information ground - an environment temporarily created by the behavior of people who have come together to perform a given task, but from which emerges a social atmosphere that fosters the spontaneous and serendipitous sharing of information.

Key Concepts: Literature Review Readings

The University of Toronto.  The Literature Review: A few tips on conducting it.
http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review

Literature Review
an account of what has been published on a topic by accredited scholars and researchers. Occasionally you will be asked to write one as a separate assignment (sometimes in the form of an annotated bibliography), but more often it is part of the introduction to an essay, research report, or thesis.

Annotated Bibliography
summarize each item briefly, while following through the themes and concepts and do some critical assessment of material. Use an overall introduction and conclusion to state the scope of your coverage and to formulate the question, problem, or concept your chosen material illuminates. Usually you will have the option of grouping items into sections and write a paragraph or so to introduce the focus of each section

A Literature Review lets you gain and demonstrate skills in:

• information seeking: the ability to scan the literature efficiently, using manual or computerized methods, to identify a set of useful articles and books
• critical appraisal: the ability to apply principles of analysis to identify unbiased and valid studies.

A Literature Review must:

• be organized around and related directly to the thesis or research question you are developing
• synthesize results into a summary of what is and is not known
• identify areas of controversy in the literature
• formulate questions that need further research

University of Santa Cruz University. Write a Literature review.
http://library.ucsc.edu/help/howto/write-a-literature-review

A literature review may constitute an essential chapter of a thesis or dissertation, or may be a self-contained review of writings on a subject. In either case, its purpose is to:
•    Place each work in the context of its contribution to the understanding of the subject under review
•    Describe the relationship of each work to the others under consideration
•    Identify new ways to interpret, and shed light on any gaps in, previous research
•    Resolve conflicts amongst seemingly contradictory previous studies
•    Identify areas of prior scholarship to prevent duplication of effort
•    Point the way forward for further research
•    Place one's original work (in the case of theses or dissertations) in the context of existing literature

Literature reviews should comprise the following elements:

• An overview of the subject, issue or theory under consideration, along with the objectives of the literature review
• Division of works under review into categories (e.g. those in support of a particular position, those against, and those offering alternative theses entirely)
• Explanation of how each work is similar to and how it varies from the others
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research

In assessing each piece, consideration should be given to: 

• Provenance—What are the author's credentials? Are the author's arguments supported by evidence (e.g. primary historical material, case studies, narratives, statistics, recent scientific findings)?
•  Objectivity—Is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point? 
• Persuasiveness—Which of the author's theses are most/least convincing?
• Value—Are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

Development of the Literature Review requires four stages:

1. Problem formulation: which topic or field is being examined and what are its component issues?
2. Literature search: finding materials relevant to the subject being explored
3. Data evaluation: determining which literature makes a significant contribution to the understanding of the topic
4. Analysis and interpretation: discussing the findings and conclusions of pertinent literature

Widener University. A beginning study - literature reviews.
http://muse.widener.edu/~aad0002/510review.htm


Typical ways of organizing a review of relevant literature

• Thematic organization - themes that emerge from literature are listed and information is included from previous studies that define, explore, illustrate and explain the themes and their importance to the research problem.
• Arguments in the field - major positions or stands on the issues surrounding the research problem are presented in a point/counterpoint fashion and the studies that illuminate these positions are introduced accordingly.
• Conceptual analysis - the concepts that define a research problem are organized in terms of part-whole relationships and studies that expand on the meaning of each concept are integrated in each section.
• Critical review of methodology - major flaws in the ways in which previous studies have investigated a research problem are explored and the studies that demonstrate these flaws are reviewed critically.

Avoiding common mistakes

• research topics that are dead end lines of investigation.
• research topics that are difficult to explore for political, economic, social reasons.
• resources on topics that have been
• Thoroughly researched and little that is new can be added.
• mistakes and mis-steps in the use of methods that have already proved to be flawed.

Key Concepts: Developing the Study

Wildemuth (2, 3, 5)

Research question development
1.    From one’s own experience or the direct observation of a situation or event in the field.
2.    From logical gaps that exist in our current state of knowledge of the field

Research question attributes
1.    Clear and easily understood
2.    Specific enough to suggest the data that need to be collected during the study
3.    Answerable with the data to be collected
4.    Interconnected with important concepts or phenomena
5.    Substantively relevant to the field

Evidence-based information practice – As information professionals make decisions about new services, systems, policies, etc., they should use the current literature base and conduct their own research to identify evidence that can inform those decisions.

Practice-based question - the researcher is a member of the organization that is being studied (e.g. a university study about student by another student).

Hypothesis – a conjectural statement of the relation between two or more variables.

Causation - The concept that one event predictably leads to another.

 
Williamson Ch. 3

Literature Review - Plays an important role in the generation of theory and in the formulation and refinement of research questions and hypotheses. It also assists in providing a background and context for a study.

Literature review writing steps:

• Categorize literature into subject/topic
• Introduction
• Significance and importance
• Contents of the review
• Body
• Organized under headings relating to research questions
•  Critically analyze relevant literature under those headings
•   Conclusion
•  Indicate gaps in literature
•  Other reasons why further research is needed
•  Highlight key concepts
•   Research questions the proposed research will investigate
•  Check that literature critical and evaluative

Patten (Topic14, 15, 16)

Strict replication - Mimicking the study in all important aspects to see if the same types of results will emerge.

Modified replication - A replication with some major modifications (i.e. examining a new population or using an imporved measurement technique)

Record - Contains all the information about a given article in electronic databases

Field - Exist within each record (e.g. title, abstract, descriptor)

Pyrczack (Appendix D)

Literature review evaluation questions:

• Does the researcher being by identifying a specific problem area?
• Are underlying theories adequately described?
• Has the researched cited sources for “factual” statements?
• Is the current research cited?
• Has the researcher distinguished between opinions and research findings?
• Has the researcher noted any gaps in literature?

Article Review #1

Stephanie Glass
9/20/11
Article Review #1

Bojko, A. (2006). Using eye tracking to compare web designs: A case study. Journal of Usability Studies, 1 (3), 112-120.

Introduction
The purpose of the study was to assess the validity of redesigned elements of a webpage using evaluation activities including eye tracking. The article reflects my research interests in Usability and Design.
 
Problem Statement
The focus of the study was to evaluate and compare a proposed redesign of the American Society of Clinical Oncology (ASCO) Web site against the original design. The specific goals were to determine if the redesigned home page would make it easier for users to locate the proper starting point for their tasks, and to locate potential areas for improvement within the proposed design. The report emphasizes the effectiveness of eye tracking methodology to collect data on attention deployment and target processing.
 
Literature Review
The eye tracking study expands upon past perception and visual search research. “However, it has not yet established itself strongly as a method to study the screen-level search behavior that is a frequent component of human-computer interaction” (Bojko, 2006).  Therefore, this study expands upon the use of the eye movement methodology when detailed evaluation of visual search is required. It also supports the use of this technique in future usability research within the Technology field.
 
Method
The study tested twelve participants who were current ASCO members and were, on average, monthly users of the current website. To compensate for the unfamiliarity of the proposed site, users were given the opportunity to familiarize themselves with the new design prior to the commencement of the study. Participants used a mouse to indicate responses to a block of 25 tasks on both the current and proposed web site design.  The tasks were the same for each site and were presented in a random order. The web pages, tasks and other stimuli were displayed on a 17” monitor with 1024 x 768 px screen resolution. A Tobii 1750 eye tracker was used to record eye movements.
 
Caveats
I feel as though an increased number and a wider variety of test subjects should have been used for this study. Though ages ranged from 30 – 55, it doesn’t seem likely that study conductors were able to get a realistic representation of current website users. Only having twelve participants seems inadequate.

 I felt researchers could not accurately compare the efficiency of each design because participants were familiar with the current version of the web site. I speculate that the results would naturally favor the current, more familiar, design. I feel that this test should have been performed by users unfamiliar with either design or the tasks should have been comprised of new, unfamiliar information and locations.