When going through some course projects, there were some deliverables using a SQL database.
If you are just hacking something to quickly test or demonstrate a thing, you do not necessarily have to do things as they should be done. Course projects are usually not quick hacks, but should demonstrate the things you have been taught and things you have actually learned.
So, for course projects using SQL databases, this means that you should do things so that the database is not exposed to simple SQL injections. Or that you should actually encrypt confidential data, such as passwords saved in database tables.
Why I am being such an asshole, expecting these to be done properly in student course projects?
Because the way you implement things, demonstrates what you have learned. Demonstrating that in course projects is kind of expected.
The things you learn going through the study program courses, should be carried on from course to course, accumulating skills and knowledge.
Because the things you consistently design and implement, is embedded into your deeper knowledge and skills and is then easier to take into use when needed in real life settings.
Because when knowing at least the basics of these things, you perhaps do not expose real apps and systems to these vulnerabilities when you go to work in the industry. Something that happens way too often…
So, if you do use a SQL database in your course work, do the basic things the way they should be done. Use prepared queries instead of exposing the app to SQL injections. Encrypt the confidential data, at least the user passwords.
I made a demo app to show how this is done, and a (Finnish) YouTube video to demonstrate the things in action to anyone learning database programming basics.
Yes, I know this demo app is not perfect either. But the idea is to demonstrate (simply) how to consider these very basic, small things in database programming. Small things that still have a considerable impact on app security.
Next Fall is the fifth time when I teach the course Data Structures and Algorithms. I received the course on 2019 and after a year of familiarising with the course, started developing it.
First I switched from C to Java and implemented automated unit tests to verify the correctness of student implementations. Some unit tests also measured time performance with varying numbers of n. The measurements were then used to support the time complexity analysis of the implementations, in addition to analysing the pseudocode and implemented algorithms. Comparing the implemented fast algorithms and data structures to slower ones was also one of the things students did.
All this to support the students to learn to see the difference between algorithms and helping them to decide which algorithm or data structure to use in whatever situation. Many times a simple algorithm suffices and simple array and plain loops are enough. But sometimes, something more is needed.
In the last two years, one or two exercise tasks had a graphical user interface (GUI) instead of a console UI, most of the tasks have. The feedback from the students was that seeing the actual effect on using faster and better data structures and algorithms in a GUI was a very nice thing to have. It really gave them a tangible view on the benefits of using these algorithms (like partitioning a large array) and data structures (finding a path using Dijkstra’s algorithm in a maze game).
Based on this student feedback, I decided to implement a GUI to all exercise tasks for the Fall 2023 course. The experiment I want to make is to see if having a GUI helps in motivating the students to work on and learn from the course exercises, compared to the more common console UIs and seeing the results of unit tests passing (or not).
Yesterday the first version of the GUI was finished, including all the exercise tasks to teach concepts like:
sorting an array with simple (and slow, with large n) insertion sort algorithm,
sorting to both ascending and descending order using the Java Comparator interface,
searching using linear search algorithm,
after sorting, implementing faster search using binary search,
implementing a Stack to verify a JSON file has no issues with parentheses nor quotation characters,
implementing a Queue to implement a “call list” (either using an array or a linked list as the internal data structure of the Queue),
when dealing with larger data sets, using faster sorting algorithms like quick sort, heap sort or merge sort,
to make handling large data sets faster in adding data elements and searching for them, using a binary search tree as the data structure,
using a hash table to count the number of unique words used in a text file and list the top 100 words with counts (a classical problem in data structures and algorithms), and finally,
using a graph data structure and associated algorithms (like Dijkstra’s) to find a shortest path between two data items.
So in addition to the already existing unit tests and time efficiency measurements, to verify the correctness and time performance of their data structure and algorithm implementations, next Fall they will also see these working in a “real life” application, the TIRA Coders Phonebook app:
TIRA Coders Phonebook app screenshot
Each of the smaller windows and dialogs appear when the feature is used from the menu of the application.
On the lower right corner, there is a log window displaying time performance data when executing the different algorithms; filling the data structures, sorting the data, searching for data, etc.
Students are able to load phonebooks of different sizes, from 100 coders up to 1 000 000 coders, and see how their implementations of different data structures and algorithms cope with various numbers of n.
I still need to work on this app more, making sure that when I remove all the solutions for the data structures and algorithms I’ve implemented, the app is still functional enough to be a starting point for the students to start working on the course exercise tasks.
I also need to carefully point to the students the parts of the code they need to work upon, and parts of the code they do not need to care about, not to get confused by the things not relevant in learning the course topics.
It will be exiting to see, in a couple of months, how this works out and if having the GUI helps in learning or motivates students to work on the tasks.
What did I do (professionally) during the year 2022? A long post about all stuff done – and mostly in progress. And a prize at the end…
First, l’ll list my active GitHub repositories of 2022. Some of these were created earlier than 2022 but I did work on these during 2022. Then I will list all the major teaching and other work done as part of my post as lecturer at the university.
So to begin with, here are the major and/or new projects I either started or took major leaps forward in 2022:
The largest project for 2022 is GitLogVisualized app (bash shell scripts, git, JUnit tests, Swift, SwiftUI, macOS) . I started to develop it in November when I realised that I need a way to quickly get an impression how all those 275+ students are doing with their projects in the Data structures and algorithms course. This way we can focus on providing help to those students that have a risk of falling behind the schedule.
GitLogVisualized showing student projects (left column list) and the selected project timeline of commits and passing / failing tests. Student ids are redacted in this screenshot using the SwiftUI redacted feature.
Timed shell scripts are executed automatically every Monday night. Scripts do git pull from each student repository from GitLab, execute git log with certain parameters, then JUnit tests are executed, and all the data is saved in two log files per student repository. The app then loads these log files and shows the project state for each student repository.
Colour codes are used to quickly show if student hasn’t done commits in the last 7-14-19 days (blue, yellow, orange, red) so that we can immediately see who should be contacted to find out if we can do anything to help those left behind ahead in the course. Projects can be sorted ascending or descending order by student, number of git commits and days since last commit.
I am planning to release this app as open source when I get it cleaned up. And a demo video is also in my plans, as soon as I get to it.
Another major project is the TVT-Sanasto, a Java app (Java, Swing, JSON, SQLite database) students can use to learn the basic terms and definitions of different categories of computing and computer networks in Finnish/English. The app can also generate a graph (using GraphViz) showing how terms are related to each other.
TVT Sanasto Java app
Actually, the term categories can be anything, since the app downloads the terms from a server JSON file. So basically anyone can write simple terminologies with explanations in a JSON file, and that can be included in the terminology category index file. Currently these JSON files are hosted in GitLab. I am hoping that other teachers would contribute by writing additional term descriptions and perhaps even create new term category dictionaries for their own courses. See a YouTube video of the app.
I also have a Swift/SwiftUI version for the TVT Sanasto app that works in iOS, iPadOS and macOS (Swift, SwiftUI, SQLite database, localized for Finnish and English), but that is still a private beta. Hopefully I will finish this by next Fall when my courses begin. If I am still teaching them. Oh and one student actually implemented a web app using these JSON dictionaries. Nice that the app family expands this way and enables students use whichever tech they find suitable for their needs in learning the basic terminology of the field.
TVT Sanasto app running on an iPadOS emulator.
Guesswork demo app (Xcode, Swift, SwiftUI, iOS) I implemented for GUI design/programming course in Spring 2022. Idea here was to demonstrate how to consider different screen sizes, zoom levels and screen orientations (portrait/landscape) as part of designing accessibility features in a (mobile) GUI.
Guesswork demo app GUI
MiniGolf scorecard app (Xcode, Swift, iOS) – originally a demo about localisation and internationalisation for GUI design/programming course. I demonstrated how to consider accessibility and enable localisation for two languages and also take into account different calendars (Gregorian, Chinese, …), zoom levels, etc. in GUI design in SwiftUI/Xcode development environment. After the course, I decided to make this a side project but hey, we all know what sometimes happens to side projects…
MiniGolf side project, another one missing me while I am working for money at the day job.
QuestionGenerator (Swift), a command line tool to generate Moodle quizzes. This is for Devices and data networks course. The course is passed by an online Moodle exam, where I ask, among other things, conversions between different radices (numbering systems; binary, decimal, octal, hexadecimal) and simple calculations where values are from different numbering systems. This tool is really convenient in generating tens or even hundreds of random conversions and calculations for students to ponder. The tool exports them into an XML file Moodle can import as quiz questions. Saves time and work when teacher does not need to create these by hand and verify the correct result.
My Slippery Cities Apple Watch app – another side project, but this time already available in the App Store! – is going to get a new feature. I would have liked to release it during Fall 2022, but again too much day job work have pushed this forward, like many other side projects. The new feature is predicting slippery weather for pedestrians based on local weather conditions and forecast provided by Apple Weather service. I am beta testing it and hopefully releasing it this Winter.
Slippery Cities beta showing next five hours slippery warning data based on temperature and precipitation.
The last new app I started in July 2022 I actually never planned to initiate. It just happened. Minesweeper (Swift/SwiftUI, macOS) is another private side project that sporadically advances – or then not when I am too busy doing something I actually get paid for.
Minesweeper in action
The classic game, with three different mine field sizes and a top-10 list of results. Some nice SwiftUI animations and sounds (“composed” with GarageBand) when you step on a mine or happen to win.
Probably the reason I started this game project was to have a realistic real world example of using recursion for the students. When the user clicks on a tile, a recursive algorithm is used to open up all the tiles having no neighboring mines for the user. If you have played the game, you’ll know what I mean.
I am planning to move from SwiftUI animations to SpriteKit graphics. I’ve done some learning on the topic but haven’t yet actually started doing it. As a side project, this may happen later rather than sooner…. Unless I decide to continue with SwiftUI animations and then publish this one in the Mac App Store sooner.
OK, then next to the projects started earlier and/or having only minor updates during 2022:
Updated a Java console chat client (Java, JUnit, HTTP, JSON) used in the 2021 Programming 3 course (server side programming). Chat client was used as a test client so that students could test their servers. Client also has JUnit tests, sending requests to their servers as students test their HTTP server implementations.
Published a GUI chat client (Swift, SwiftUI, HTTP, JSON) for the same Programming 3 chat server. Wanted to try out implementing a GUI chat client using the (then) new Swift async / await features. I implemented this already in 2021 and wanted to release it, perhaps it will help students to try out and learn something else than the usual languages (Java…) used in GUI programming courses.
Warnings app (private; iOS/macOS with Swift, SwiftUI and Core Data with Apple Cloud support). Ongoing side project that fetches various warnings (weather, slippery conditions, air quality, etc) from several open data sources and displays alerts to user, warning about potentially harmful or dangerous environmental conditions. Haven’t touched this for a while due to being too busy with paid work. Surprise, surprise.
books-cpp demonstration about using C++ map data structures std::map and std::unordered_map in a single and multithreaded app. The demo relates to the Data structures and algorithms course. Learning goal here is to be aware of which container library data structure to select for performance, and that sometimes parallel processing may help but not always, at least significantly.
Books and Words (Swift; Xcode, iOS/macOS) – when I knew I would take over the Data structures and algorithms course a couple of years ago, I wanted to brush up my skills in the area and implemented this classic programming exercise. I bought the book Exercises in Programming Style by Cristina Videira and implemented some styles from there in Swift. This year I updated the project with new Swift releases, implemented enum style binary search tree data structure, took new performance measurements, etc.
Graphs is a demonstration app I have used in teaching graph data structures. I made just some tiny updates to it this fall.
Another demo for the Data structures and algorithms course, SortSpectacle (Swift, SwiftUI, iOS/macOS), now has a new sorting method, Block sort. It is a variant of merge sort.
Then to teaching work at the University. What I did in teaching at Spring/Summer 2022:
Taught exercises in Java programming basics in Programming 2 course (around 250 students) for Finnish and English students (separate groups).
Taught GUI design/programming (various programming languages) in Programming 4 course.
Participated in the national collaboration group of lecturers responsible for data structures and algorithms courses in various universities and tech institutes in Finland. We discuss the pedagogical, technical, etc. topics related to teaching, learning and organizing this kind of courses.
Interviewed prospective students over Zoom, applying to our MSc program on software engineering from various countries around the world.
Arranged summer courses for Devices and Data networks and Data Structures and algorithms. These were offered as independent study, without actual teaching.
Prepared improved materials for two of the courses I an responsible for; Devices and Data networks and Data Structures and algorithms. I made improvements to lectures and exercises based on last year feedback from students and colleagues.
And in Fall 2022:
Offered my course Devices and Data networks both in Finnish (252 students) and in English (23 students) separately with three other teachers assisting. Two of them were students I interviewed and selected as part time teaching assistants together with a really nice and professional colleague of mine. Very nice workmates, all of them!
Offered another of my courses, Data Structures and algorithms (313 students). As you can see from the projects above, quite a many of those are for this course.
Supported a colleague teaching the data structures course for international students. Didn’t participate in teaching the English group though.
Preparing (in December) to return – after a very long break – to participate in the study program BSc student projects as a supervisor. Projects launch in the beginning of January. Students plan and execute a software development project for local companies. Teachers supervise them and see that all goes OK, intervene if necessary. I was surprised to see they still use the project documentation templates I created a long time ago in 1998! They’ve been updated since then, obviously.
Last but not least – our study program student guild Blanko awarded me as the Distinguished teacher of the year! This was the second year in a row for me to get this honor! Really taken and humbled that the work I am doing is valued by the students. In addition to the nice award certificate I got a box of various delicacies like chocolate to enjoy. Thanks Blanko!
I am not giving any new year resolutions, but one goal I have is to finish all those side projects this year and put them to actual use somewhere, if anyone is interested or finds these projects of mine fun or useful. Until that happens, I should really, really not start any new side projects…. We’ll see how that goes.
Yesterday was the first day of teaching in this Fall semester. Times are a bit challenging; teaching resources for the courses I teach in have shrinked down to about 58-66% of last year’s teaching hours, depending on a course. Also the number of teachers for a course is down. While number of students is going up rather than down.
Face-to-face teaching is not possible with these resources and number of students (225-300+ in two of the courses ongoing currently). We’d be in the classroom every day from 8am to 8pm and still have overlapping teaching events. So online it is. Hello Zoom.
One feedback item from students last year was that the terminology is challenging in the Devices and data network course. Understandable: in five two hour lectures I should be able to give an overview of how computers work and the components they contain. The same for the networking, ten hours of lectures and they should get a grasp on how the Internet and packet networks do their thing. This means a constant shower of new terminology unfamiliar to 1st year students. Other courses have similar issues.
So I developed a dictionary app for students to use. Or actually two. One with Java for desktop computers:
Dictionary Java app for desktop computers
And another one using Swift and SwiftUI for macOS, iOS and iPadOS:
The iPhone version of the dictionary app.
Both apps get the dictionaries from the same source, a remote git repository with a JSON file, using HTTPS. Thus the dictionaries can be edited, appended to and updates are then downloaded to the users’ devices.
In the user device the terms are stored into a SQLite database, so they can be used offline. Users can search for the terms and follow the URLs to sources in the Internet to find out more.
The desktop versions also have a feature that generates a graph of the relationships between the terms by searching the terms from other terms’ description. For this, GraphViz must be installed on the device.
The Swift version has a Widget for all Apple platforms. The widget randomly shows a couple of terms a day from the dictionary. Here’s the iPhone widget (along with the GitHub app widget on top):
And here is the macOS Widget:
Clicking or tapping on the term on the widget opens the app and shows the description as well as English term for that specific ICT term.
The Java version is already public and available for installation. Swift version is still under development, I just need to decide how to distribute it to the various types of devices and prepare the App Store information with screenshots, descriptions and all. Then there is the inevitable App Store review waiting. Hopefully it’ll pass. This is yet another app for the category “downloads and renders JSON in a GUI”.
Hopefully the students find the apps useful for learning. Next I should focus on finishing the dictionary for the basic Internetworking part of my course and put also that available for the apps to download…
Well it was a fight but got the code coverage working with VS Code / Java and Coverage Gutters extension. Doing this for an exercise under preparation for the next Fall course on Data Structures and Algorithms. Probably this is just a demo or something students may take a look at, not a required task.
Code coverage visible, YEAH!!
What I have in the Maven pom.xml currently is
<argLine>-Xmx2048m</argLine>
within the properties struct, and in the build/plugins:
and the whole jacoco plugin structure is here, under the build/plugins structure (with some commented out sections):
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.8</version>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${project.build.directory}/jacoco.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco</outputDirectory>
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<!-- Ensures that the code coverage report for unit tests is created
after unit tests have been run. -->
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${sonar.jacoco.reportPath}</dataFile>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${jacoco.path}</outputDirectory>
</configuration>
</execution>
<execution>
<id>default-report</id>
<goals></goals>
<configuration>
<dataFile>${project.build.directory}/jacoco.xml</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco</outputDirectory>
</configuration>
</execution>
<execution>
<id>jacoco-check</id>
<goals>
<goal>check</goal>
</goals>
<configuration>
<rules>
<rule>
<element>PACKAGE</element>
<limits>
<limit>
<counter>LINE</counter>
<value>COVEREDRATIO</value>
<minimum>0.0</minimum>
</limit>
</limits>
</rule>
</rules>
</configuration>
</execution>
</executions>
</plugin>
and finally, you need to build from the command line with this Maven command:
mvn jacoco:prepare-agent package jacoco:report
So when viewing the code then, you can see the code coverage colors in the code editor sidebar on the left.
In the previous post I mocked the Java app that was hardcoded to use Windows C: disk as the default place to open files.
What then is the recommended way? One is to start looking from the user home directory:
JFileChooser fileChooser = new JFileChooser();
fileChooser.setCurrentDirectory(new File(System.getProperty("user.home")));
Or pick the documents directory. It is also a nice thing to save the selected directory and use that if the user would like to continue with the already selected directory next time.
What else is happening? It is now the last week of the course Data Structures and Algorithms I am responsible teacher. Students have been analyzing algorithm correctness and time complexity, implementing basic data structures, more advanced ones like hash tables and hash functions, binary search trees and binary search.
Lectures addressed graphs and graph algorithms too, but implementation of these was in an optional course project only, Mazes. When students finish that, they can play a game a bit like the PacMan.
They get to choose from two course projects: either that Mazes project or optionally implement the classic task: count the unique words from a book file, ignoring some words from another file. The largest test file is around 16 MB, having almost 100 000 unique words.
Processing the largest file, using naively two nested for loops takes on my Mac Mini M1 with 16MB of memory around two minutes to process. The fast versions (hash tables, binary search trees) take less than a second.
I have implemented several different solutions for comparison with three different programming languages, Java, Swift and C++. Each week in lectures I demonstrated some of these, and in the end we had this table for comparison (sorry, in Finnish only).
Hall of Fame of the different Books and Words implementations. Swift versions can be found from the link above to GitHub.
As you can see, the C++ with 8 threads was the fastest one. Next after C++ came a couple of Java versions. Swift implementations were not so fast as I expected. After some profiling, I suspect the reason is in the way Unicode chars are handled in Swift. All the book files are UTF-8 and students were expected to handle them correctly. I do not like that mostly in teaching programming languages the default is to stick with ascii and conveniently forget the existence of different languages and character sets.
Well, anyways, for some reason, processing these UTF-8 text files takes a lot of time with Swift. Maybe later I have time to find out if the issue is in my code and/or is there anything that can be done to speed things up.
Something very nice happened a week ago — the student guild of our study program, Blanko, awarded me this diploma for being a quality teacher. Apparently they had a vote and I somehow managed to come first this time. The diploma was accompanied by this Italian themed small gift box. A really nice surprise! I was so astonished to receive this, thank you so much if anyone of you is reading!
Nice award from the students for quality teaching.
The checkFileHash method will calculate a hash for the file and check if the correctHash matches with the calculated hash. If the hashes match, the file is identical to what was expected.
boolean checkFileHash(String fileName, String correctHash) {
...
File file = new File(fileName);
MessageDigest md = MessageDigest.getInstance("SHA");
FileInputStream fis = new FileInputStream(file);
byte[] dataBytes = new byte[1024];
int nread = 0;
while ((nread = fis.read(dataBytes)) != -1) {
md.update(dataBytes, 0, nread);
}
byte[] mdbytes = md.digest();
StringBuilder sb = new StringBuilder();
for (int i = 0; i < mdbytes.length; i++) {
sb.append(Integer.toString((mdbytes[i] & 0xff) + 0x100, 16).substring(1));
}
fis.close();
System.out.println("\nHash is " + sb.toString());
return sb.toString().equals(correctHash);
Well this does compile, run and produce a file hash, compare it against the correct one and returns the result.
If the correctHash was calculated earlier on a same OS, hashes will even match. But if the corrrectHash was calculated in a different OS with a different default character encoding, the hashes do not match.
You need to explicitly specify that UTF-8 encoding is used when reading the file — which also should be saved UTF-8 encoded. And when calculating the expected correctHash the UTF-8 encoding should also be used. And when the hash for the file is calculated when file is inspected in this method:
boolean checkFileHash(String fileName, String correctHash) {
...
// Use SHA for hashing, does not need to be secure.
MessageDigest md = MessageDigest.getInstance("SHA");
// Use buffered reader since we need to read explicitly using UTF-8.
BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream(fileName), "UTF-8"));
// Read the lines from the line to a String.
String line = null;
StringBuilder sb = new StringBuilder();
while ((line = br.readLine()) != null) {
sb.append(line);
}
br.close();
// Calculate the SHA digest (hash) from the string
byte [] mdbytes = md.digest(sb.toString().getBytes("UTF-8"));
// Convert the hash into a String.
StringBuilder hsb = new StringBuilder();
for (int i = 0; i < mdbytes.length; i++) {
hsb.append(Integer.toString((mdbytes[i] & 0xff) + 0x100, 16).substring(1));
}
String calculatedHash = hsb.toString();
System.out.println("\nHash is " + calculatedHash.toString());
// Check if the hash matches with the expected correct one.
boolean matches = calculatedHash.equals(correctHash);
if (!matches) {
System.out.println("Correct has is: " + correctHash);
}
return matches;
Using a platform independent programming language does not guarantee platform independent code.
Obviously UTF-8 specifically is not required, but using a same encoding everywhere.
Once I used (on a macOS) a Java app with file open dialog, that was hardcoded to show directory C:\ by default. Sooooo platform independent…
Just checked the first commit date of the completely renovated Data structures and algorithms course repository. It was around December 2020.
Since then I have designed and implemented about ten exercises with unit tests in Java. And two course projects with unit tests. Plus about ten additional demos with Swift. All with commentary and instructions.
In a week the new course begins. I’ve recorded about 20 hours of new lecture materials. Additional videos about graph programming with C++ will be included from last year demos.
One of the reasons for not posting too much lately ?
The video is about one of the course project implementing a graph data structure with various algorithms including Dijkstra’s shortest path finding algorithm. I thought applying this in a game context would perhaps motivate the students to dwell into the world of data structures and algorithms. We’ll see….
