What is Web log analysis? Jim Jansen College of Information Sciences and Technology The Pennsylvania State University firstname.lastname@example.org Let’s make this a discussion!
Outline • Definition • Examples • Theory and Essential Construct • Data Collection • Method • Discussion
Web log analysis is part of the domain of … • ... Web analytics • The Web Analytics Association (WAA) defines Web analytics as the measurement, collection, analysis, and reporting of Internet data for the purposes of understanding and optimizing Web usage (http://www.webanalyticsassociation.org/) • Shares common theoretical and methodology characteristics with all forms of log analysis (e.g., Intranet logs, systems logs, OPAC logs, search logs, etc.)
W3C Extended Log Format W3C Extended Log Format -Variety of fields for examining visitors to Web sites. Other common format is NCSA Separate Log that is composed of three logs ( Common log – actions on the server, Referral log – where they came from, and Agent log – stuff about the client computer)
Variety of tools help make sense of this log data
Other Log Examples … Search Logs have some common fields, such as We can enrich the log time, queries, results, etc. with additional fields.
Theoretical Foundations • Part of the behaviorism paradigm • Behaviorism – an approach focused on the outward behavioral aspects of thought and emphases the observed behaviors • Behaviorism – Pavlov, Watson, & Skinner Ivan Petrovich Pavlov John B. Watson Burrhus Frederic Skinner
Behaviorism Characteristics • Inductive, data-driven and characterized by empirical observation of measurable behavior • Grounded on somebody doing something in a situation (all the environmental and situational features are embedded behaviors) • Critics of behaviorism as a psychological theory have issues with rejection of mental processes. I agree - people are more than “mediators between behavior and the environment” (Skinner, 1993, p 428)
What is a Behavior? … an observable activity of a person, animal, team, organization, or system. One can classify behaviors into three general categories. Behaviors are • something that one can detect and record • actions or specific goal-driven events with some purpose other than the specific action that is observable • reactive responses to environmental stimuli
What is a Behavior? • Behavior is the essential construct of the behaviorism and of log research • Logs record behaviors of users and systems (records behavior but can’t tell affective, cognitive, or situational aspects) • A behavior is the key variable (i.e., an entity representing a set of events where each event may have a different value)
Ethograms • a taxonomy or index of behavioral patterns • details the different forms of behavior that an user exhibits • categories of behavior are objective, discrete, not overlapping. This makes the definitions of each behavior (and category of behaviors) clear, detailed and distinguishable from each other
Example of an Ethogram Behavior Description View results Interaction in which the user viewed or scrolled one or more pages from the results listing. If a results page was present and the user did not scroll, we counted this as a View Results Page. With Scrolling User scrolled the results page. Without Scrolling User did not scroll the results page. but No Results in Window User was looking for results, but there were no results in the listing.
Selection Interaction in which the user makes a selection in the results listing. Click URL (in results listing) Interaction in which the user clicked on a URL of one of the results in the results page. Next in Set of Results List User moved to the Next results page. Previous in Set of Results List User moved to the Previous results page. GoTo in Set of Results List User selected a specific results page.
View document Interaction in which the user viewed or scrolled a particular document in the results listings. With Scrolling User scrolled the document. Without Scrolling User did not scroll the document. Behavior Description of the behavior
Execute What about Interact the data coll ion i e n whi ctio ch t n h me user ini etho tiate d? d an action in the interface. Execute Query Interaction in which the user entered, modified, or submitted a query without visibly incorporating assistance from the system. This category includes submitting the original query, which was always the first interaction with system. Find Feature in Document Interaction in which the user used the FIND feature of the browser. Create Favorites Folder Interaction in which the user created a folder to store relevant URLs.
Navigation Interaction in which the user activated a navigation button on the browser, such as Back or Home. Back User clicked the Back button. Home User clicked the Home button.
Browser Interaction in which the user opened, closed, or switched browsers. Open new browser User opened a new browser. Switch /Close browser window User switched between two open browsers or closed a browser window.
Relevance action Interaction such as print, save, bookmark, or copy. Bookmark User bookmarked a relevant document. Copy - Paste User copy-pasted all of, a portion of, or the URL to a relevant document. Print User printed a relevant document. Save User saved a relevant document.
View/Implement assistance Interaction in which the user viewed the assistance offered by the application. Implement Assistance Interaction in which the user entered, modified, or submitted a query, utilizing assistance offered by the application. Phrase User implemented the PHRASE assistance. Spelling User implemented the SPELLING assistance. Synonyms User implemented the SYNONYMS assistance. Previous Queries User implemented the PREVIOUS QUERIES assistance. Relevance Feedback User implemented the RELEVANCE FEEDBACK assistance. AND User implemented the AND assistance. OR User implemented the OR assistance.
Data Collection: Trace Data • can view the data collected in log files as trace data. • people conducting the activities of their daily lives many times create things, create marks, induce Wea r w o ear, or n a carpet reduce some existing material. • Within the confines of research, these things, marks, and wear become data • Classically, trace data are the physi Trash h cal eap remains of people’s interaction Computer storage media
Trace Data • In the past, trace data was often time consuming to gather and process, making such data costly. • logging software makes collecting trace data easy and cheap • Log data is control ed accretion data, where the researcher or some other entity alters the environment in order to create the accretion data • With the user of client apps (such as desktop search bars), the collection of data is nearly unlimited from a technology perspective What is cool about trace data for researchers?
Data Collection Log data has significant advantages as a data collection approach for the study and investigation of behaviors, including: • Scale: not a limiting factor as in lab user studies • Power: large sample size for inference testing; in fact, so large must account for the size effect • Scope: naturalistic; researchers can investigate range of interactions in a multi-variable context • Location: can collectin distributed environments • Duration: collect log data over an extended period
Methodological Foundations Use of logs to col ect trace data is an unobtrusive methods (a.k.a., non- reactive or low-constraint). Unobtrusive methods … Customer Behavior (video) • allows data col ection without directly interfering into the context and • does not require a direct response from participants Chemistry (surface marking)
Methodological Foundations Three justifications for unobtrusive methods: • Un E c x erta am in pl ty e: p etrinciple hnogr : res aphy ea st r uch di ers es ( in w therje er c e tted he into an r e es n e v air r onme cher “ nt bi rbe d come dogs” pa a rt st of th udy e p s artyis citem pant • Ob E s x erv am e plr e e: ffec no t: dif one fe s re ean r c c e h th es a f t is or pma or d n e i n t o a a l n ab activity or a person’s behaviors by being observed study of Web searching • Observer bias: observers overemphasize behavior the E y x ex am p plec e: t itso f wind hy a mnd edifail cal t o tr ino al t s ic e ar b e ehavi doubl or e th bli e n y d do not expect rather than single blind Trace data helps in overcoming the Uncertainty principle, Observer effect, and Observer bias in the data collection. Note for data collection but not data analysis
Methodological Foundations Inherent characteristics in the method of log data col ection; Web analytics has issues to address as a result: • Abstraction – how does one relate low-level data to higher-level concepts? • Selection – how does one separate the necessary from unnecessary data? • Reduction – how does one reduce the complexity and size of the data set? • Context – how does one interpret the significance of events? • Evolution – how can one col ect data without impacting application deployment or use?
Recap of Web Analytics Type of Data Trace Data Unobtrusive Collection Key Construct Behavior Query ser Theoretical R U es Behaviorism ponse Foundation Click puter
Research • Book: Jansen, B. J., Spink, A., and Taksa, I. (2009) Handbook of Research on Web Log Analysis, Hershey, PA: Idea Group Publishing – First chapter on theory of log analysis is free! • Lecture: Jansen, B. J. (Forthcoming) Understanding User – Web Interactions via Web Analytics. Morgan- Claypool Lecture Series. Gary. Marchionini (Ed). Morgan-Claypool: San Rafael, CA. – manuscript about Web Analytics, soup to nuts
Thank you! (open for questions and further discussion) Jim Jansen Col ege of Information Sciences and Technology The Pennsylvania State University j email@example.com