Sakai WG - Data Analysis
Easily accessible information about Sakai users, their attitudes, and their usage is critical to the work of Sakai-related staff including support staff, designers, researchers, developers, testers, and system administrators. Current capabilities to gather the required data and tools to present and interpret it are limited and vary across Sakai implementations. The Sakai Data Analysis group intends to fully describe the types of usage data needed by all stakeholders, compare current strategies for gathering and interpreting user and usage data, and specify new strategies for gathering and interpreting data. Development of these strategies may be undertaken as a Sakai project.
This Work Group has several different projects and tools under discussion and design. These include collaboratively building a question bank for user surveys, looking at use cases to help design usage analysis tools, and development news of statistics tools, as well as others. The site below is organized by headings and child pages.
Instructor / Student Survey Question Bank
In an effort to help institutions evaluate their Sakai instance, the community is building a shared question bank. While each institution may construct their surveys however they like, by using similar questions the community can more easily look across instances and compare users' attitudes. The question bank is being constructed on a child page. Click the link below to access the question bank.
strongLink to Question Bank
Focusing on the use cases allows us to see that while there are many stakeholders who need usage data, the data in many cases is the same. This site has been structured around the initial use cases shown below. My hope is that the use case focus will allow us to design/deliver meaningful and complete chunks of usage data as we progress. In addition, we can compare ways that we access particular types of data across institutions, helping us understand the complexities of global and local configurations.
Given the diversity of the Use Cases and Requirements a Sakai Warehouse seems to be a viable solution.
Sakai warehouse (definition):
- Provides an integrated and total view of learning/collaboration systems
Existing VLE/CLE don't provide integrated views while this is often required by some stakeholders like management
- Makes the systems current and historical information easily available for decision making
Availability of current and historical information is essential for decision-making
- Makes decision-support transactions possible without hindering operational systems
Usage of learning/collaboration enterprise systems generates a large volume of data that tend to be unmanageable and hinder operational systems.
- Presents a flexible and interactive source of strategic information
Strategic information should be presented in such a way that it can be understood by a stakeholder. Whether this is an instructor, or a board member
The Sakai Warehouse consists of three components: Data acquisition, Data Storage, and Information Delivery
Compared to existing VLE/CLE's data retrieval from sources and the delivery of information is split. This setup makes it possible to retrieve data without hindering operational systems and provides the information delivery a certain flexibility. Example of sources are the 'EVENT' table or the 'log' files
- Data acquisition
The 'Data acquisition'- component retrieves data from sources and pre-processes the data so that it can be stored
- Data Storage
Actual storage of pre-processed data that can be accessed by information delivery tools. Exmple of of storage are 'flattened' tables according to a STAR schema.
- Information delivery
This component retrieves data from the 'Storage' component. All kind of (different) tools can access the Data storage. A Sitestats tool for instructors, a PDF generated report for management, a Data Mining tool for researchers
Each use case below has a page to house discussion and examples. The image at right is a diagram linking the use cases, the actors and their responsibilities.
- Use Case 1 - Find out who did a particular action in a site
- Use Case 2 - Find out when a particular action in a site happened
- Use Case 3 - Find trends in usage over time
- Use Case 4 - Find out how often a particular widget is used
- Use Case 5 - Find out how often users visit each page
- Use Case 6 - Characterize use by course and department
- Use Case 7 - Characterize use by properties of objects in a tool or site
- Use Case 8 - Measure activity by user
Related Sakai 2.2 Requirements
- REQ-5: Site Usage statistics tool Community Ranking: 83 votes (28 critical, 31 essential, 22 desirable, 2 n/a)
- REQ-54: Log searching tool Community Ranking: 63 votes (18 critical, 12 essential, 32 desirable, 1 n/a)
- REQ-152: Server-wide usage statistics Community Ranking: 52 votes (15 critical, 21 essential, 15 desirable, 1 n/a)
- REQ-160: User activity information displayed with profile Community Ranking: 56 votes (8 critical, 11 essential, 29 desirable, 8 n/a)
- REQ-237: Documentation: Architecture document on the Sakai Event Mechanism Community Ranking: 40 votes (8 critical, 4 essential, 23 desirable, 5 n/a)
- REQ-330: Generate usage reports from within Sakai Community Ranking: 51 votes (14 critical, 18 essential, 17 desirable, 2 n/a)
- SAK-3794: SAKAI_SESSION table is unmanageable
Proposed High-Level Data Specifications
The following documents are proposals for how to access certain data, NOT YET designs for particular Sakai tools. Do these proposals cover the types of data in the above requirements?