When are they learning? A #Moodle activity calendar heatmap

As a precursor to looking at withdrawal and drop-out points I wanted to visualise learner activity across an academic year. This could help determine the patterns of behaviour for different individuals. The idea being that you can quickly see at a glance which days each learner is using the system. Extending last week’s exploration with activity heatmaps I came across the lattice based time series calendar heatmap which provides a nice way of plotting this for exploration. It is quite a simple process that requires some date manipulation to create extra calendar classifications. Then I made a change to the facet to show each row as a different user rather than a year.

Calendar Heatmap

calendar_heatmap

In the calendar heatmap each row is a learner and the grid shows the spread of daily activity across each month in a familiar calendar format. The visualisation quickly reveals patterns such as activity tailing off in the final months for Student4, the extended Easter holiday during Apr for Student8, and the late starter or crammer that is Student10. A couple of students also broke the usual Christmas learning abstinence and logged in during the holidays. There are a few variants of this that are possible to achieve by playing with the facet or applying it to different summaries of the log data for example a facet on activity types within a course or activity participation for a single learner that I may explore in future.

Continue reading

Advertisements

Where is your learning activity? A #Moodle component heatmap.

Understanding which courses use which tools is a useful starting point for exploration and may be informative to staff development programs or used in conjunction with course observations. The Moodle Guide for Teachers, for example, could be used to help form an understanding of the tools in question. I’m interested in the exploration side, having started a new project in the last week with some former colleagues. We’re exploring what can be learned from learner data and so if I know where different types of activity are happening then I can drill-down into these areas.

I’m using an idea I picked up from Flowing Data to create a heat map of tool use in a Moodle LMS by category. The heat map visualisation site nicely with the existing tool guide so seems a good approach. The Moodle site has been recently upgraded and so the dataset has old style logs (mdl_log) and new style logs (mdl_logstore_standard_log) so the data extraction and wrangling has to account for both formats. Then it is a case of manipulating the data into the heat map format.

Heat Map

heatmap

I’ve focused on learner activity within each type of tool rather than the number of tools in a course. The intention is to show the distribution of learner activity. It shows clearly the dominance of resource and assessment type tools, as well as some pockets of communication and collaboration. In this instance the values are skewed by the large number of resource based activities and the dominance of a single department in terms of activity numbers, which can be seen in the bar chart below. However, the technique can be applied to comparing courses within a department or comparing users within a course, which may share more similar scales.

barplot

Continue reading

Can activity analytics support understanding engagement a measurable process? Inspiration from @birdahonk

I was pleased to find out my revised paper on this topic was accepted for publication in the September issue of the Journal of Applied Research in Higher Education. The basic premise of the paper is that engagement can be measured as a metric through the appropriation of ideas commonly used in social marketing metrics. For this post I’ll briefly discuss how I approached this by presenting engagement as a learning theory using the ideas of Freire and Vygotsky, as a process, and as a metric. I’ll also share my workshop slides from the conference if you want to try and create your own learner engagement profile. While I’ve started looking into different approaches, this post summarises some of the key principles developed throughout the paper that have guided my thinking of engagement.

UVUEngagement as a learning theory

The paper proposes a concept of engagement that draws on the work of Paolo Freire and Lev Vygotsky and the evolution of the learner voice. The first aspect of this is to re-position the learner as the subject within education and not the object of education, supplanting previous models which portray the learner as a passive recipient of pre-packaged knowledge. The second aspect is understanding the learner voice as a creative (Freire) and spontaneous (Vygotksy) expression within a socialised teaching-learning process that supports dialectical interactions between learner and teacher curiosity. This positions engagement as the process of recognising and respecting the learner’s world, which as Freire reveals is after all the ‘primary and inescapable face of the world itself’ in order to support the development of higher-order thinking skills. The repression of this voice is likely to result in patterns of inertia, non-engagement and alienation that are discussed widely in motivation and engagement literature. This triangulation between motivation and engagement remains a theme central to a range of learning analytics research. Correlation between learning and autonomy remains an interesting area of research.

Engagement as Process

For the paper I used Haven’s engagement process model and overlaid it with concepts from engagement literature reviews by Fredricks, Blumenfeld, & Paris (2004) and Trowler (2010)Haven posits that engagement is the new metric that supersedes previous linear metaphors encompassing the quantitative data of site visits, the qualitative data of surveys and performance, as well as the fuzzy data in between that represents social media. Haven and Vittel elaborate this into an expansive process that link four components of engagement: involvement, interaction, intimacy, and influence through the key stages of discovery, evaluation, use, and affinity (see below). To appropriate this into the educational literature research of Fredricks et al. one can explore examples of involvement and interaction as behavioural engagement, intimacy as emotional engagement, and influence as cognitive engagement. Furthermore when considering whether engagement is high or low in each component, Trowler’s categorisation of negative engagement, non-engagement, and positive engagement can be adopted. Engagement Process

Engagement as a metric

Learner Dashboard

The goal of positioning this as a metric was to create a learner engagement profile, similar to Haven’s engagement profile for marketing. I used Stevenson’s (2008) Pedagogical Model and Conole and Fill’s (2005) Task Type Taxonomy as ways of classifying log data and social network analysis to understand interactions between the different course actors. These were used to form dashboards such as the example above that could then be used to understand profiles, such as the one below (name fictionalised). One insight is that where simple raw VLE data might have suggested an engaged learner who is regularly online and features centrally in discussions, the engagement profile reveals the possibility of a learner who may lack academic support during their time online (evenings) and demonstrating a pattern of alienation based on an apparently strategic approach within an environment that is heavily structured through teacher-led inscription. Given the number of users who have not logged in or have yet to post to the discussions it might also seem sensible to target other learners for engagement interventions, however this would miss opportunities, revealed in the engagement profile, to provide useful support interventions targeting improved learner voice.

Engagement Profile

Continue reading