More course activity analysis tools

I’ve written before about the tools I’ve built to manage the workload and gather statistics on activity of students in my Data Structures and Algorithms course.

To recap, the tool is needed due to large number of students in the course (and others, going on simultaneously), while the teaching resources are quite petite, to put it mildly. Last year the total enrolment count was ~350, this Fall it is ~230. Repository count this year is (currently) ~200.

What kind of activity statistics? Like the total number of daily commits from the start of the course, done by students:

Bar chart of count of git commits per day in a programming course with ca. 200 students.

Someone made a git commit on the first day of the course, and the most active day so far was the ninth day with 35 commits in total.

Or the repositories having a specific count of commits currently:

Bar chart displaying the count of student git remote repositories having a specific number of commits so far. Zero commits in remote repository accessible to teachers does not indicate zero work. Student may have commits in their local repository or uncommitted work not yet in git.

The graph above shows that 133 repositories have zero commits at the time this analysis was done, 17 repositories have one commit, etc. The most active student has done 27 commits already.

This data graph shows what have the students been working on during the last seven days:

A chart displaying the most actively edited files based on commit counts and changed lines in all student repositories.

Which looks quite what is expected, since the first five Java files are supposed to be worked upon in the first three course programming tasks in the schedule this and previous course weeks.

These are all based on my Swift/SwiftUI app analysing the data from git and git logs, either stored in my app’s database or parsing from the output of git log command, as the analysis is executed and the graphs are generated.

But what about the activities related to the lectures? Assumption is, obviously, that the students first go through the lectures and lecture demonstrations and only then dwell into the associated programming tasks. An assumption that does not unfortunately always hold, but still…

We use Moodle as the learning environment. While Moodle has extensive and detailed tools to analyse and observe student activities, getting a quick overview from those tools can be a bit cumbersome.

For example, when you view the activity log of a link to a lecture video in Moodle, you can see who and when has accessed that link. But if the same student clicks the activity’s link several times, there are several rows in the activity log for the same student. Also, that log does not quickly tell me the total count of students who have accessed the lecture video at least once. I’d have to do the counting myself manually.

I’ve split the lecture videos to a manageable lengths (from the viewing perspective) which means there are 25 video links currently of the actual lecture content. In addition to that, there are some demonstration videos. Clicking through the Moodle analytics on all of these and counting by hand how many students have accessed the videos is therefore not something I’d do e.g. weekly. Too much manual, tedious and error prone work.

This is not a one time chore, but a thing to do many, many times during the course. Preferably weekly from September to December. Then the teachers would be on the map about the students advancing in the course, week by week.

Using the analytics data, we at least theoretically could be able to observe something relevant happening (or something not happening what was supposed to happen!) that could indicate problems somewhere in the course implementation. Maybe the data could also be used to improve the course for the next year, pinpointing places where the planned schedule and actual realities do not match and it is the course schedule that should be adjusted.

Fortunately, Moodle offers a download link for each activity log report. I can choose the format of the downloaded file, one of which is JSON. So, I implemented (no LLMs used, as usual!) a command line tool using Swift, which

  1. reads the downloaded Moodle JSON analytics files from a specified directory,
  2. reads the activity name from the JSON file contents,
  3. counts the unique student names in that file, and then
  4. updates a tab separated values (tsv) file where the statistics (no student data is stored) of each Moodle activity are stored, per analysis date, and
  5. the tsv file can be opened in a spreadsheet app and data can be visualised as a graph:
A bar chart displaying the Moodle activity access counts of course lecture videos, gathered from Moodle activity logs. The chart tells me that many students have not yet started to watch the lecture videos but are falling a bit behind of the schedule. Not yet alarmingly so, but let’s see how it goes. It is always possible that some watch videos together, only one student accessing the activity while many are actually watching. A very good thing in itself, but may influence the statistics.

The relevant part of the JSON file handling is:

func processJSONFile(_ file: String) throws -> (num: Int, act: String)? {
  let fileContents = try String(contentsOfFile: file, encoding: .utf8)
  if let jsonData = fileContents.data(using: .utf8) {
    var uniqueUserNames: Set<String> = []
    var target = "" // The Moodle activity name
			
    let jsonDecoder = JSONDecoder()
    let items = try jsonDecoder.decode([[MoodleStatisticsItem]].self, from: jsonData).flatMap( { $0 } )
			
    for item in items {
      // Ignore the listed teachers who clicked the activity link.
      if !ignoreNames.contains(item.kokoNimi) {
        uniqueUserNames.insert(item.kokoNimi)
      }
      if target.isEmpty {
          target = item.tapahtumanKonteksti.replacingOccurrences(of: "Verkko-osoite: ", with: "")
      }
    }
// ...

What is actually collected and later saved to the tsv file from this, is the uniqueUserNames.count, telling the number of individuals (excluding teachers) who have accessed the Moodle activity at the time the tool was executed, as well as the name of the Moodle activity from within the JSON file, in target.

Now, instead of spending tens of minutes or even hours, clicking and manually counting things from the Moodle activity logs, I’ll spend maybe 5 minutes max to download the JSON log files actually only like seconds, since I just changed the tool to handle just one JSON file containing all activities of the course, and one second to execute the tool and open the resulting file in the spreadsheet app. Huge time savings!

All this done, by the way, when I was (still am) sick from a flu and can’t go teach in the classroom. Implementation took less than a workday. No employer’s time wasted doing this, just my sick day, but saving employer paid work time later! [Do not work when sick. I did because I like programming and didn’t consider this work, actually, but an interesting problem to solve.]

When I used the tool first time, the resulting graph above shows the activity stats of course lecture videos for that date, Sept 15th. I plan to launch the tool once a week, so it is then possible to see the difference to last weeks’ viewing stats. The next time launched, a new entry is added to tsv file for that date stats.

Knowing the recommended course schedule, I can then use the tool to see how the students actually advance through the course lectures. This should also give more insight into the git based programming task analytics.

If, for example, the graph above shows that students are advancing into the Stack data structure lectures a week after the planned schedule, teachers could expect that the students are actively starting the related coding tasks that week or the next one. Then we could be better prepared to handle the inevitable issues and questions about stacks at that time. Or we could anticipate that when the deadline comes, a certain number of student are behind the schedule and may need additional support to actually pass the deadline without issues. Hopefully.

The only manual thing left to do is to actually download those 25 Moodle activity JSON log files. I bet I could somehow record a script that does that clicking and selecting type of downloads in the browser automatically too… 🤔

Course deadlines app

Courses have deadlines. Oftentimes it happens that the deadline comes and the student work is not ready. Even though the deadlines are clearly communicated, often in several places in course materials and website. There may be many reasons for missing a deadline, but whatever the reason, missing one usually leads in to problems in passing the course.

Missing a deadline also causes extra work. Questions are asked, replies sent, and if there is a really good or valid reason, there may be an exception to the rule and extension to the deadline is granted. But if this happens often, in a course of hundreds of students, all this is extra work burdening and stressing both the teachers and the students.

If everybody gets an extension to the deadline, time after time, the concept of deadline becomes irrelevant. If one student gets an extension to the deadline, all others with the same reasons should also get it, since students must be treated equally. Therefore, it would be really, really nice if everybody would acknowledge the deadlines and schedule their work so that no deadlines are missed. A useful habit or skill to have also later in working life…

To make sure a deadlines are not missed since “I forgot it” or “It came sooner than I expected”, I started to remind students of one course about the forthcoming deadlines on my weekly Monday lecture. Since apps are cool, and I like to tinker with things, I wrote an app for this. Obviously, I could have just shown the list of deadlines from the course materials, slideshow or website, but that is boring. Also the app is yet another example of how to implement things, a demo that I can use in various programming courses and a topic for a blog post 🤓

Presenting the Course Deadline Counter app:

Course deadlines app with a list of courses on the left. For the selected course, list of deadlines on the right. Deadlines have symbols and colors informing about the type and urgency of the deadline.
Course deadlines app with a list of courses on the left. For the selected course, list of deadlines on the right. For more screenshots, click the GitHub link in the text and view them from the readme documentation.

The app is localized to English and Finnish, above you can see the Finnish localization screenshot, in GitHub readme, you can see the English locale version screenshots.

Each deadline has graphics and formatting to convey something about the deadline:

  • A symbol (trying) to convey the type of the deadline, an exam (notepad) or a programming task (hammer).
  • Deadlines that are “strict” (dealbreakers; will lead to failing the course if missed, others may be more flexible) are shown with an orange warning sign.
  • Deadlines already passed are shown in gray, and
  • deadlines coming soon (are “hot”) are highlighted in red.

See the GitHub readme document for examples of these. The app is actually better than a static list of text in web or slideshow in that it will dynamically format the deadlines, as listed above and shown in the GitHub readme.

The deadlines are stored, for each course, in a separate JSON file in the Documents/CourseDeadlines directory on the user’s Mac:

func store() throws {
	let storagePath = URL.documentsDirectory.appending(component: Deadlines.appDocumentDirectory, directoryHint: .isDirectory)
	deadlines.sort()
	let fileManager = FileManager.default
	let encoder = JSONEncoder()
	encoder.outputFormatting = [.prettyPrinted, .sortedKeys]
	let data = try encoder.encode(self)
	let filePath = storagePath.appending(path: name + ".json").path(percentEncoded: false)
	print("Storing file \(filePath)")
	if !fileManager.createFile(atPath: filePath, contents: data) {
		throw DeadlineErrors.fileSaveError
	}
}

The design decision here was to make it easy to share the course deadlines — just send the JSON file to anyone with this app. Or another app anyone perhaps chooses to implement that is able to read the JSON file.

With the app, user can add end edit new courses, add and edit deadlines, and also delete deadlines and courses. Although the app was originally developed for a teacher (me), it could obviously be used also by students to manage the deadlines for the courses she takes.

I did present the deadlines with one course, weekly, using this app last Fall in the live lectures. I also sent a weekly newsletter in the course Moodle space and included a screenshot of the current deadline situation in the message.

I didn’t gather any data on the impact of the app. Did students actually miss the deadlines like in any other course? Cannot say. My main motivation was to make sure no one misses the deadlines because of not being aware of them. Hopefully the awareness of deadlines was improved, and there wasn’t (too much) annoyance in being repeatedly reminded of them, time and time again 😂

There are some Apple/Mac specific things in the JSON, an example of a course with four deadlines:

{
  "deadlines" : [
    {
      "becomesHotDaysBefore" : 7,
      "date" : 764027972.579756,
      "goal" : "Form pairs for course work",
      "isDealBreaker" : false,
      "symbol" : "person.2",
      "uuid" : "179C5A8D-2E3B-4000-A3F8-F821E98CDCAC"
    },
    {
      "becomesHotDaysBefore" : 7,
      "date" : 765464432.579756,
      "goal" : "GUI design finished",
      "isDealBreaker" : true,
      "symbol" : "document.on.clipboard",
      "uuid" : "E3A7AF6F-9322-4D6A-994C-346D5F1B6F43"
    },
    {
      "becomesHotDaysBefore" : 7,
      "date" : 767883632.579756,
      "goal" : "Learning tasks done",
      "isDealBreaker" : false,
      "symbol" : "checkmark.seal.fill",
      "uuid" : "DB7A7330-1CBD-4785-8FDD-AB5B5F35A596"
    },
    {
      "becomesHotDaysBefore" : 7,
      "date" : 768488432.579756,
      "goal" : "Implementation done",
      "isDealBreaker" : true,
      "symbol" : "hammer",
      "uuid" : "1AD3DBD3-4121-40EA-9C4D-368A10765999"
    }
  ],
  "name" : "Ohjelmointi 4",
  "startDate" : 763401692.579756,
  "uuid" : "33DCF108-1FAD-4DE7-9C6E-A9DC02073043"
}

The deadline date is not milliseconds from 1970 as integer (perhaps more common), but as double. The symbol for the deadline is a name for Apple SF Symbol. If implementing an app for other platforms to show the same JSON data, the developer needs to figure out a way to show something as the symbol (or ignore it). Maybe a custom drawn image or something.

I list some future improvement ideas in the GitHub readme, but don’t actually know if I am going to implement those. The app already fulfills the intended purpose, so anything else would be just something fun to implement, if I have the time or motivation to do them. One thing not listed in the readme is to provide the app also for iPhone/iPad, but since the original idea as a teacher tool used in lectures, did this only as a Mac app.

If you have any use for an app like that, or just want to use it for learning Swift/SwiftUI, the app is open sourced so just take it and use it.

Decoding and encoding class hierarchies in Swift

A while ago I was implementing (for fun) a chat client to work with a chat server used in teaching in a GUI programming course. The client is implemented with Swift and SwiftUI.

In the GUI programming course, students get the server and a sample console client, both implemented in Java. The goal for the students is to both design and implement a GUI chat application. Learning goals are related to GUI design, usability design and implementation of a GUI in a selected language and GUI framework. The server is accessed over TCP and data format is JSON.

One tricky thing was to figure out how the Swift Codable could handle a class structure with a base class Message, having different types of messages as subclasses. For example, ChatMessage (having the sender, message, timestamp, etc.), a ListChannelsMessage (containing the available channels on the server side to join), et cetera.

The Java console client class diagram, containing the same message classes as the Swift app.

The Swift Codable does not directly and simply bend to this kind of a class hierarchy. I searched the net and found a good basis here, but it didn’t fully show how to do both decoding and encoding with a class structure like I had.

So obviously, for fun, I banged my head against the wall until I got it working. I was supposed to write a blog post about that, but haven’t had the time.

But today someone in Mastodon asked the exact question; how to do something like this. A good motivation for me to actually do something about that! I didn’t want to publish the whole Swift Chat client app, since students are still able to start working on this project, and someone might want to select Swift/SwiftUI as the GUI for their app.

So instead, I extracted the code to show how to use Codable and JSONSerialization together to handle a class hierarchy like this to encode and decode objects to JSON and back. The code is now available as a simple example project in GitHub.

I shared the repository to the Mastodon discussion. What was exciting for me is that one of the core persons in developing the Swift language itself favourited my reply! ?

Nice idea for my git log visualiser

I could use this excellent idea in my Git log visualiser. Never thought I could do this! ?

Currently I am parsing plain text output from git log –numstat command, looking at certain order of lines and line beginnings to determine what is on the line.

Exporting from git log to JSON using jq like Simon does in his post would make parsing extremely simple using Swift Codable.

Posting this here to remind myself.

Sadness

Update March 26th: they seem to be up and running again!

I’ve been using weather.willab.fi weather service located here in Oulu, at the VTT research center, for years in many of my courses.

It has been a good demo for teaching many basic computing and networking topics, for example:

  • how to use curl to execute network requests and look at the different HTTP headers the server provides in the response;
  • how HTTP works on top of lower level protocols (TCP), using Wireshark for traffic analysis;
  • localization of simple Android and iOS apps;
  • what is JSON and XML and how they look like with simple weather data;
  • what is a server and what is a client on the Internet;
  • and maybe something else I do not remember now.

But anyways, the service has been down for some time, returning weather data from around December 2022. Now it is even worse; it just displays the nginx server welcome page, and HTTP 404 when you try to get the current weather contents using http://weather.willab.fi/weather.json.

Hopefully they will kick the service up and running soon! Otherwise I need to rewrite many exercises and demo apps to use some other simple service.

Merging two JSON files

Having two large JSON files with one common element, how can you merge them? Editing by hand not so nice if the files contain hundreds of elements.

For example, you have a large JSON file of locations:

{
  "name": "Oulu",
  "lat": 65.013784817,
  "lon": 25.47209907
},

And then you also have a large JSON file of location codes for the commune:

{
  "name": "Oulu",
  "code": "564"
},

And you would like to merge these into a one JSON file and parse the commune code, name, and location data at one go.

You can do this with jq using the following command:

./jq-osx-amd64 --slurpfile file2 codes.json.txt '
  INDEX($file2[]; .name) as $dict
  | $dict[.name] as $x
  | if $x then . + $x else empty end
'  coords.json.txt > out.json

And as a result, you get your JSON element merged in one file:

{
  "name": "Oulu",
  "lat": 65.013784817,
  "lon": 25.47209907,
  "code": "564"
},

Much faster than copying and pasting or implementing your own JSON merger.