Where is your learning activity? A #Moodle component heatmap.

Understanding which courses use which tools is a useful starting point for exploration and may be informative to staff development programs or used in conjunction with course observations. The Moodle Guide for Teachers, for example, could be used to help form an understanding of the tools in question. I’m interested in the exploration side, having started a new project in the last week with some former colleagues. We’re exploring what can be learned from learner data and so if I know where different types of activity are happening then I can drill-down into these areas.

I’m using an idea I picked up from Flowing Data to create a heat map of tool use in a Moodle LMS by category. The heat map visualisation site nicely with the existing tool guide so seems a good approach. The Moodle site has been recently upgraded and so the dataset has old style logs (mdl_log) and new style logs (mdl_logstore_standard_log) so the data extraction and wrangling has to account for both formats. Then it is a case of manipulating the data into the heat map format.

Heat Map

heatmap

I’ve focused on learner activity within each type of tool rather than the number of tools in a course. The intention is to show the distribution of learner activity. It shows clearly the dominance of resource and assessment type tools, as well as some pockets of communication and collaboration. In this instance the values are skewed by the large number of resource based activities and the dominance of a single department in terms of activity numbers, which can be seen in the bar chart below. However, the technique can be applied to comparing courses within a department or comparing users within a course, which may share more similar scales.

barplot

Continue reading

Inside forum posts – politics, networks, sentiment and words! Inspired by @phillipdawson, @shaned07, and @indicoData #moodle #learninganalytics

Enhanced communication has long been championed as a benefit of online learning environments, and many educational technology strategies will include statements around increased communication and collaboration between peers. So in thinking towards an engagement metric for my current project and the need to get inside activities for my, in progress, PhD proposal exploring forum use is one of the more interesting analytics spaces within the LMS. I’ve used three techniques for my initial analysis: (1) a look at post and reply counts inspired by @phillipdawson and his work on the Moodle engagement block, (2) social network analysis inspired by a paper by @shaned07 on teacher support networks; and (3) sentiment and political view analysis provided by @indicoData as an introduction to text mining.

I’ll start with sharing the visualisations and where these might be useful and then finish with details of how I coded these.

Forum posts

Total weekly forum posts by student

Following Phillip Dawson’s work on the engagement block for Moodle, I decided to look into two posting patterns: (1) posts over time; and (2) average post word count. The over time analysis (above) compares the weekly posting pattern of each student in a group. For most students replies to peers and teachers are “in phase” suggesting that when they are active they discuss with the entire group and so learning design might focus on keeping them active. One can also notice that those who only reply to peers appear to have much lower overall post activity, which in the original engagement block would place them at-risk – learning design may consider teacher-led interventions to understand whether discussions with the teacher impact their overall activity. The average word count analysis (below) reinforces the latter case where those demonstrating that those who only reply to peers infrequently post shorter replies. Conversely those who post infrequent lengthy posts tend to target the teacher and do not follow up with many further replies discussion. There is some suggestion of an optimal word count around 75-125 for forum posts that might warrant further investigation.

Forum Posts

Social Network Analysis

Social Network Analysis

The network diagram (above) confirms what was emerging in the post analysis: that a smaller core of students (yellow circles) are responsible for a majority of the posts, and further reveals the absolute centrality of the teacher (blue circle) that highlight how important teacher-led interventions may be to this group. This is probably not surprising although the the teacher may use this to consider how they might respond more equally to the group – here the number of replies is represented by increasing thickness of the grey edges and they appear to favour conversations in the lower left of the network. A similar theme is explored by Shane Dawson (2010) in “‘Seeing’ the learning community”. One can understand this further by plotting eigenvalue centrality against betweenness centrality (below) where a student with high betweenness and low eigenvalue centrality may be an important gatekeeper to a central actor, while a student with low betweenness and high eigenvalue centrality may have unique access to central actors.

Centrality

Content Analysis

Sentiment analysis

Text analysis of forums provides a necessary complement to the above analysis, exploring the content within the context. I have used the Indico API to aid my learning of this part of the field rather than try to build this from scratch. The sentiment analysis API determines whether a piece of text was positive or negative in tone and rates this on a scale from 0 (negative) to 1 (positive). Plotting this over time (above) provides insights into how different topics might have been received with this group showing generally positive participation, although with two noticeable troughs that might be worth some further exploration. The political opinion API scores political leaning within a text on a scale of 0 (neutral) to 1 (strong). Plotting this for each user (below) shows that more politicised posts tend to be conservative (unsurprising) although there is a reasonable mix of views across the discussion. What might be interesting here is how different student respond to different points of view and whether a largely conservative discussion, for example, might discourage contribution from others. Plotting sentiment against libertarian leaning (below2) shows that participants are, at least, very positive when leaning towards libertarian ideology, though this is not the only source of positivity. Exploring text analysis is fascinating and if projects such as Cognitive Presence Coding and the Quantitative Discourse Analysis Package make this more accessible then there are some potentially powerful insights to be had here. I had also hoped to analyse the number of external links embedded in posts following a talk by Gardner Campbell I heard some years ago about making external connections of knowledge, however the dataset I had yielded zero links, which while informative to learning design is not well represented in a visual (code is included below).
Political leaning

Libertarian sentiment

Continue reading

Communication Plans and Agile

At a recent meeting I was prompted of the need to map the informal communications of everyday collaboration with the more formal expectations of project management – as the adage goes: ‘two monologues do not make a dialogue’. On the other hand two equally weighted discourses directed at the same referential object (or theme) must intersect and enter into a semantic bond (Bakhtin, 1984). While ideologically I might hope of encouraging innovative collaboration through renouncement of monologic habits and primitive definitiveness, in a practical sense I needed to integrate the formal plan with dynamic feature delivery. For me AGILE approaches better capture everyday collaboration while PRINCE2 better handles the formal project staging (taking the best bits of both).

The project, as most of mine do, involves the implementation of an integrated Moodle / Mahara platform enhanced with a range of customisations. So some features are delivered out of the box by the software and some require bespoke development.  Rather than sharing spreadsheets and (un)versioned documents, we have implemented Pivotal Tracker (@pivotallabs) which has proved effective in its simplicity. In order to transform a 20+ page document of requirements into a deliverable items requires a quick review and mapping of how we label and score stories. Using a macro to get the original document table into a workable spreadsheet, one can then apply a workflow mapping between the methodologies before importing into pivotal.

Workflow AGILE PRINCE2 
Icebox New ideas to be scoped for future iterations Requirements for later stages
Backlog Scheduled items for next iteration (outside current velocity) Requirements in the next work package
Current To be delivered in current iteration Requirements in the current work package
Done Features accepted as delivered Signed-off requirements

The other area I needed a mapping was between the notions of implementation (what the software does) and development (what we need to change). We also extended our point scale for development to map implementation items onto this (it remains to be seen if the value mapping is equivalent as velocity starts to be recorded).

Score Development Implementation
0 No action No action
1 Language string change Default feature / configuration
2 Minor interface change 3rd party plug-in
3 Exact requirements understood Module combinations / learning design
5 Good idea of requirements – refine through iteration Multiple options / possible training need
8 “Epic” – further investigation required Further investigation required

With a sensible labelling system to cross-map the system aspects (e.g. core, 3rd-party, development) to the requirements sections (e.g. content, assessment, communication) one can  filter on the key aspects in groups and check their status. The significance of tagging over categorisation for linked data approaches should not be underestimated here, as it allows the information to be presented within different hierarchies. A weekly review of the current and next iteration now simplifies the communication process.

I don’t claim to have done anything radical here, other than reinforce in my mind the importance of keeping the project focus on communication. Tools, methodologies and most importantly documents are only as good as the dialogues they mediate. While creative (or productive) ideas will originate in the informal everyday collaborations, if they cannot be scoped into the project then they may disappear – worse yet, this may then discourage new ideas and limit overall project innovation.

Once upon a time the most agile prince was a frog:

‘Its very funny to be a frog
You can dive into the water and cross the rivers
And the oceans
And you can jump all the time and everywhere’
M83, Raconte-Moi Histoire

I hope that the project can instil this light-hearted approach to its management.

Readings

Bakhtin, M. M. (1984). Problems of Dostoevsky’s Poetics. Minneapolis: University of Minnesota Press.