Educational Analytics

We propose the concept of Contested Collective Intelligence (CCI) as a distinctive subset of the broader Collective Intelligence design space. CCI is relevant to the many organizational contexts in which it is important to work with contested knowledge, for instance, due to different intellectual traditions, competing organizational objectives, information overload or ambiguous environmental signals. The CCI challenge is to design sociotechnical infrastructures to augment such organizational capability. Since documents are often the starting points for contested discourse, and discourse markers provide a powerful cue to the presence of claims, contrasting ideas and argumentation, discourse and rhetoric provide an annotation focus in our approach to CCI. Research in sensemaking, computer-supported discourse and rhetorical text analysis motivate a conceptual framework for the combined human and machine annotation of texts with this specific focus. This conception is explored through two tools: a social-semantic web application for human annotation and knowledge mapping (Cohere), plus the discourse analysis component in a textual analysis software tool (Xerox Incremental Parser: XIP). As a step towards an integrated platform, we report a case study in which a document corpus underwent independent human and machine analysis, providing quantitative and qualitative insight into their respective contributions. A promising finding is that significant contributions were signalled by authors via explicit rhetorical moves, which both human analysts and XIP could readily identify. Since working with contested knowledge is at the heart of CCI, the evidence that automatic detection of contrasting ideas in texts is possible through rhetorical discourse analysis is progress towards the effective use of automatic discourse analysis in the CCI framework.

This paper proposes that Social Learning Analytics (SLA) can be usefully thought of as a subset of learning analytics approaches. SLA focuses on how learners build knowledge together in their cultural and social settings. In the context of online social learning, it takes into account both formal and informal educational environments, including networks and communities. The paper introduces the broad rationale for SLA by reviewing some of the key drivers that make social learning so important today. Five forms of SLA are identified, including those which are inherently social, and others which have social dimensions. The paper goes on to describe early work towards implementing these analytics on SocialLearn, an online learning space in use at the UK's Open University, and the challenges that this is raising. This work takes an iterative approach to analytics, encouraging learners to respond to and help to shape not only the analytics but also their associated recommendations.

Theoretical and empirical evidence in the learning sciences substantiates the view that deep engagement in learning is a function of a complex combination of learners' identities, dispositions, values, attitudes and skills. When these are fragile, learners struggle to achieve their potential in conventional assessments, and critically, are not prepared for the novelty and complexity of the challenges they will meet in the workplace, and the many other spheres of life which require personal qualities such as resilience, critical thinking and collaboration skills. To date, the learning analytics research and development communities have not addressed how these complex concepts can be modelled and analysed, and how more traditional social science data analysis can support and be enhanced by learning analytics. We report progress in the design and implementation of learning analytics based on a research validated multidimensional construct termed "learning power". We describe, for the first time, a learning analytics infrastructure for gathering data at scale, managing stakeholder permissions, the range of analytics that it supports from real time summaries to exploratory research, and a particular visual analytic which has been shown to have demonstrable impact on learners. We conclude by summarising the ongoing research and development programme and identifying the challenges of integrating traditional social science research, with learning analytics and modelling.

Satisfaction and its discontents 8 March 2012 The National Student Survey puts pressure on lecturers to provide 'enhanced' experiences. But, argues Frank Furedi, the results do not measure educational quality and the process infantilises students and corrodes academic integrity One of the striking features of a highly centralised system of higher education, such as that of the UK, is that the introduction of new targets and modifications to the quality assurance framework can have a dramatic impact in a very short space of time. When the National Student Survey was introduced in 2005, few colleagues imagined that, just several years down the road, finessing and managing its implementation would require the employment of an entirely new group of quality-assurance operatives. At the time, the NSS was seen by many as a relatively pointless public-relations exercise that would have only a minimal effect on academics' lives. It is unlikely that even its advocates would have expected the NSS to acquire a life of its own and become one of the most powerful influences on the form and nature of the work done in universities.

Wmatrix is a software tool for corpus analysis and comparison. It provides a web interface to the USAS and CLAWS corpus annotation tools, and standard corpus linguistic methodologies such as frequency lists and concordances. It also extends the keywords method to key grammatical categories and key semantic domains. Wmatrix allows the user to run these tools via a web browser such as Opera, Firefox or Internet Explorer, and so will run on any computer (Mac, Windows, Linux, Unix) with a web browser and a network connection. Wmatrix was initially developed by Paul Rayson in the REVERE project, extended and applied to corpus linguistics during PhD work and is still being updated regularly. Earlier versions were available for Unix via terminal-based command line access (tmatrix) and Unix via Xwindows (Xmatrix), but these only offer retrieval of text pre-annotated with USAS and CLAWS.

Data Visualizations - Organized by KATY BÖRNER

About Cohere The Web is about IDEAS+PEOPLE. Cohere is a visual tool to create, connect and share Ideas. Back them up with websites. Support or challenge them. Embed them to spread virally. Discover who - literally - connects with your thinking. Publish ideas and optionally add relevant websites Weave webs of meaningful connections between ideas: your own and the world's Discover new ideas and people We experience the information ocean as streams of media fragments, flowing past us in every modality. To make sense of these, learners, researchers and analysts must organise them into coherent patterns. Cohere is an idea management tool for you to annotate URLs with ideas, and weave meaningful connections between ideas for personal, team or social use. Key Features Annotate a URL with any number of Ideas, or vice-versa. Visualize your network as it grows Make connections between your Ideas, or Ideas that anyone else has made public or shared with you via a common Group Use Groups to organise your Ideas and Connections by project, and to manage access-rights Import your data as RSS feeds (eg. bookmarks or blog posts), to convert them to Ideas, ready for connecting Use the RESTful API services to query, edit and mashup data from other tools Learn More Subscribe to our Blog to track developments as they happen. Read this article to learn more about the design of Cohere to support dialogue and debate.

CORE IDEAS We decided on the name Learning Emergence because we are very much learning about emergence and complex systems phenomena ourselves, even as we develop our thinking on learning as an emergent, systemic phenomenon in different contexts. We must shift to a new paradigm for learning in schools, universities and the workplace which addresses the challenges of the 21st Century. Society needs learners who can cope with intellectual, ethical and emotional complexity of an unprecedented nature. Learning Emergence partners share an overarching focus on deep, systemic learning and leadership - the pro-active engagement of learners and leaders in their own authentic learning journey, in the context of relationship and community. We work at the intersection of (1) deep learning and sensemaking, (2) leadership, (3) complex systems, and (4) technology:

What is LOCO-Analyst? LOCO-Analyst is an educational tool aimed at providing teachers with feedback on the relevant aspects of the learning process taking place in a web-based learning environment, and thus helps them improve the content and the structure of their web-based courses. LOCO-Analyst aims at providing teachers with feedback regarding: *  all kinds of activities their students performed and/or took part in during the learning process, *  the usage and the comprehensibility of the learning content they had prepared and deployed in the LCMS, *  contextualized social interactions among students (i.e., social networking) in the virtual learning environment. This Web site provides some basic information about LOCO-Analyst, its functionalities and implementation. In addition, you can watch videos illustrating the tool's functionalities. You can also learn about the LOCO (Learning Object Context Ontologies) ontological framework that lies beneath the LOCO-Analyst tool and download the ontologies of this framework.

Features Gephi is a tool for people that have to explore and understand graphs. Like Photoshop but for data, the user interacts with the representation, manipulate the structures, shapes and colors to reveal hidden properties. The goal is to help data analysts to make hypothesis, intuitively discover patterns, isolate structure singularities or faults during data sourcing. It is a complementary tool to traditional statistics, as visual thinking with interactive interfaces is now recognized to facilitate reasoning. This is a software for Exploratory Data Analysis, a paradigm appeared in the Visual Analytics field of research.

Links to instruments from 2000 through 2011.

Australasian Survey of Student Engagement (AUSSE) Areas measured by the AUSSE The survey instruments used in the AUSSE collect information on around 100 specific learning activities and conditions along with information on individual demographics and educational contexts.The instruments contain items that map onto six student engagement scales: Academic Challenge - the extent to which expectations and assessments challenge students to learn; Active Learning - students' efforts to actively construct knowledge; Student and Staff Interactions - the level and nature of students' contact and interaction with teaching staff; Enriching Educational Experiences - students' participation in broadening educational activities; Supportive Learning Environment - students' feelings of support within the university community; and Work Integrated Learning - integration of employment-focused work experiences into study. The instruments also contain items that map onto seven outcome measures. Average overall grade is captured in a single item, and the other six are composite measures which reflect responses to several items: Higher-Order Thinking - participation in higher-order forms of thinking; General Learning Outcomes - development of general competencies; General Development Outcomes - development of general forms of individual and social development; Career Readiness - preparation for participation in the professional workforce; Average Overall Grade - average overall grade so far in course; Departure Intention - non-graduating students' intentions on not returning to study in the following year; and Overall Satisfaction - students' overall satisfaction with their educational experience.

National Survey of Student Engagement What is student engagement? Student engagement represents two critical features of collegiate quality. The first is the amount of time and effort students put into their studies and other educationally purposeful activities. The second is how the institution deploys its resources and organizes the curriculum and other learning opportunities to get students to participate in activities that decades of research studies show are linked to student learning. What does NSSE do? Through its student survey, The College Student Report, NSSE annually collects information at hundreds of four-year colleges and universities about student participation in programs and activities that institutions provide for their learning and personal development. The results provide an estimate of how undergraduates spend their time and what they gain from attending college. NSSE provides participating institutions a variety of reports that compare their students' responses with those of students at self-selected groups of comparison institutions. Comparisons are available for individual survey questions and the five NSSE Benchmarks of Effective Educational Practice. Each November, NSSE also publishes its Annual Results, which reports topical research and trends in student engagement results. NSSE researchers also present and publish research findings throughout the year.

Attempts to imagine the future of education often emphasize new technologies-ubiquitous computing devices, flexible classroom designs, and innovative visual displays. But the most dramatic factor shaping the future of higher education is something that we can't actually touch or see: big data and analytics. Basing decisions on data and evidence seems stunningly obvious, and indeed, research indicates that data-driven decision-making improves organizational output and productivity.1 For many leaders in higher education, however, experience and "gut instinct" have a stronger pull.

From the Semantic Web to social machines: A research challenge for AI on the World Wide Web Jim Hendler, Tim Berners-Lee Abstract The advent of social computing on the Web has led to a new generation of Web applications that are powerful and world-changing. However, we argue that we are just at the beginning of this age of "social machines" and that their continued evolution and growth requires the cooperation of Web and AI researchers. In this paper, we show how the growing Semantic Web provides necessary support for these technologies, outline the challenges we see in bringing the technology to the next level, and propose some starting places for the research.

Technical Report KMI-11-01 June 2011 Simon Buckingham Shum and Rebecca Ferguson Abstract: We propose that the design and implementation of effective Social Learning Analytics presents significant challenges and opportunities for both research and enterprise, in three important respects. The first is the challenge of implementing analytics that have pedagogical and ethical integrity, in a context where power and control over data is now of primary importance. The second challenge is that the educational landscape is extraordinarily turbulent at present, in no small part due to technological drivers. Online social learning is emerging as a significant phenomenon for a variety of reasons, which we review, in order to motivate the concept of social learning, and ways of conceiving social learning environments as distinct from other social platforms. This sets the context for the third challenge, namely, to understand different types of Social Learning Analytic, each of which has specific technical and pedagogical challenges. We propose an initial taxonomy of five types. We conclude by considering potential futures for Social Learning Analytics, if the drivers and trends reviewed continue, and the prospect of solutions to some of the concerns that institution-centric learning analytics may provoke. 

Learning networks, crowds and communities Full Text: PDF Author: Caroline Haythornthwaite University of British Columbia, Vancouver, BC Who we learn from, where and when is dramatically affected by the reach of the Internet. From learning for formal education to learning for pleasure, we look to the web early and often for our data and knowledge needs, but also for places and spaces where we can collaborate, contribute to, and create learning and knowledge communities. Based on the keynote presentation given at the first Learning Analytics and Knowledge Conference held in 2011 in Banff, Alberta, this paper explores a social network perspective on learning with reference to social network principles and studies by the author. The paper explores the ways a social network perspective can be used to examine learning, with attention to the structure and dynamics of online learning networks, and emerging configurations such as online crowds and communities.

Attention please!: learning analytics for visualization and recommendation Full Text: PDF Author: Erik Duval Katholieke Universiteit Leuven, Leuven, Belgium This paper will present the general goal of and inspiration for our work on learning analytics, that relies on attention metadata for visualization and recommendation. Through information visualization techniques, we can provide a dashboard for learners and teachers, so that they no longer need to "drive blind". Moreover, recommendation can help to deal with the "paradox of choice" and turn abundance from a problem into an asset for learning.

A unified framework for multi-level analysis of distributed learning Full Text: PDF Authors: Daniel Suthers University of Hawaii, Honolulu, HI Devan Rosen School of Communications, Ithaca College, Ithaca, NY Learning and knowledge creation is often distributed across multiple media and sites in networked environments. Traces of such activity may be fragmented across multiple logs and may not match analytic needs. As a result, the coherence of distributed interaction and emergent phenomena are analytically cloaked. Understanding distributed learning and knowledge creation requires multi-level analysis of the situated accomplishments of individuals and small groups and of how this local activity gives rise to larger phenomena in a network. We have developed an abstract transcript representation that provides a unified analytic artifact of distributed activity, and an analytic hierarchy that supports multiple levels of analysis. Log files are abstracted to directed graphs that record observed relationships (contingencies) between events, which may be interpreted as evidence of interaction and other influences between actors. Contingency graphs are further abstracted to two-mode directed graphs that record how associations between actors are mediated by digital artifacts and summarize sequential patterns of interaction. Transitive closure of these associograms creates sociograms, to which existing network analytic techniques may be applied, yielding aggregate results that can then be interpreted by reference to the other levels of analysis. We discuss how the analytic hierarchy bridges between levels of analysis and theory.