Master’s coursework

A directory of my advanced coursework projects from my part-time Master of Computer Science degree at the University of Melbourne, including from my exchange semester to ETH Zürich.

Disclaimer: I do not grant permission to use this work. If you are a student, please do your own projects. You will learn more and develop your integrity.

Contents

Semester 1, 2019 (Part-time @ University of Melbourne):

• COMP90042 Web Search and Text Analysis
• COMP90045 Programming Language Implementation

Semester 2, 2019 (Part-time @ University of Melbourne):

• COMP90051 Statistical Machine Learning
• PHYC90045 Introduction to Quantum Computing

Semester 1, 2020 (Exchange @ ETH Zürich, interrupted but continued online 🦠):

• 227-0395-00L Neural Systems
• 252-0526-00L Statistical Learning Theory
• 252-1424-00L Models of Computation
• 363-0588-00L Complex Networks

Semester 2, 2020 (Part-time @ University of Melbourne, virtual campus 🦠):

• COMP90044 Research Methods

Selected projects:

• “Scalable massively multi-author profiling and attribution of micro-messages”, for COMP90051 Statistical Machine Learning, the University of Melbourne, 2019. [project details] [report] [code] [kaggle (3rd place)] [group work]

• “Expectile-based distributional reinforcement learning and dopamine-associated mental disorders”, for 227-0395-00L Neural Systems, ETH Zürich, 2020. [project details and code] [report] [group work]

• “The promise of distributional reinforcement learning”, for COMP90044 Research Methods, the University of Melbourne, 2020. [project details] [literature review]

• “Tailored expectile imputation algorithms for efficient expectile-based distributional reinforcement learning”, for COMP90044 Research Methods, the University of Melbourne, 2020. [project details] [research plan] [presentation]

COMP90042 Web Search and Text Analysis

Trevor Cohn’s subject on advanced topics in modern information retrieval and natural language processing. This subject was essentially my first formal exposure to machine learning. [Subject details]

Final grade: 98 (top)

The main assessment was a pair project to implement an automatic fact verification system based on the Wikipedia-derived ‘FEVER’ dataset.

My project partner and I were able to achieve early leading performance on the class leaderboard using a multi-stage evidence retrieval and classification system we dubbed SHERLOCK. (We were eventually left behind—it seems we should have just used BERT for sentence embeddings!)

Our work is summarised in our final report (the code is not publically available).

I worked closely with my project partner throughout the entire project. I was chiefly responsible for the first half of our multi-step pipeline.

COMP90045 Programming Language Implementation

Harald Søndergaards’s class on the principles of compiler design and implementation. [Subject details]

Final grade: 99 (top)

The main assessment was to implement, in teams, a compiler for a toy programming language called ‘Goat’. The compiler was implemented in Haskell. We used the Parsec monadic parser combinator library, and implemented a monadic code generation library too. We also explored lexer and parser generators for Haskell (alex and happy).

The code for our final compiler, along with project specifications, is available on GitHub.

We all worked closely together on the project, so it is difficult to identify any part which is chiefly my own contribution.

COMP90051 Statistical Machine Learning

Benjamin Rubinstein’s subject on advanced topics in statistics and machine learning. This subject pushed my statistics skills and broadened my perspective on machine learning techniques. [Subject details]

Final grade: 99 (equal top)

The major assessment was was an extreme authorship attribution group challenge—authorship attribution of micro textual data (Tweets) for over 10,000 users.

We placed third on the class leaderboard in the public and held-out evaluations. Our approach is summarised in our report and our code can be found on GitHub.

My team collaboratively implemented and evaluated our approach. I was chiefly responsible for the mathematical details and was able to contribute some algorithmic direction to help our programs scale (in hindsight, we might have eeked out even more performance by switching from pure python to numpy in key places). We also worked together closely on the report. After submission, I added the appendix on scalable implementation tactics.

PHYC90045 Introduction to Quantum Computing

A comprehensive introduction to quantum computing for non-physicists, from Lloyd Hollenberg and Charles Hill. We studied topics all the way from the basic one-qubit gates to the QFT, Shor’s algorithm, quantum annealing architectures, quantum communication, error-correcting surface codes, quantum computational complexity theory, and more. [Subject details]

Final grade: 99 (top)

In weekly exercises, we used the University of Melbourne’s Qauntum User Interface and IBM Q’s real quantum computers. (There was also a class on using a real DWave annealer, but I missed it!)

Assessment was through projects and an exam, but since these are likely to be re-used in future semesters, I won’t post my work.

227-0395-00L Neural Systems

Richard Hanhloser’s introduction to neuroscience for engineers. This subject pushed me far outside of my comfort zone, but rewarded me with a basic knowledge of the fundamentals of neuroscience. [Course details]

Final grade: 6.0 (6.0 scale), ECTS credits: 6.

The major assessment was an open-ended interdisciplinary team research project. Supervised by Richard Hahnloser, we were asked to “find some interesting parallel between machine learning and neuromodulation, and see if an insight from either of these fields can inform the other”.

We found DeepMind’s recent distributional hypothesis of dopaminergic signalling, and investigated implications of this hypothesis using simple models and simulations. The result was our final report.

A brief summary of the key points:

• DeepMind’s Nature paper claims that dopaminergic neurons encode distributional information in their activity, akin to signals in the nascent distributional reinforcement learning paradigm, and in particular that the mammalian reward system may implement expectile-based distributional RL.
• We fleshed this out into an architectural model of the reward system as an extension of the neural actor-critic architecture, and identified the expensive ‘imputation step’ of current expectile-based distributional RL algorithms as a biologically implausible bottleneck.
• We investigated more plausible imputation strategies, and we showed in proof-of-concept simulation experiments that the imputation process is vital to the consistency of learning, since plausible low-level defects we introduced in the model could manifest themselves at the level of behaviour.
• We speculated that if the distributional hypothesis is correct then similar defects could underlie reward-system-related mental disorders such as addiction and Parkinson’s disease. (It’s worth emphasising speculated, here).
• We found a number of other minor insights which we included in the appendix, including an apparently novel efficient algorithm for computing sample expectiles.

This was a group project. The whole group met regularly to discuss and eventually crystallise our hypotheses and model. I was partly responsible for understanding the technical RL side of the work, and I am indebted to my team members for helping me to understand the neuroscience side. I was also responsible for designing and running the majority of our simulation experiments. For the report, I was the main contributor to sections 1.1, 1.3, 2, 3, and the sole contributor to appendix A.

Some of the code we used in these experiments can be found on GitHub:

• an efficient algorithm for computing sample expectiles,
• tabular implementation of various distributional RL algorithms, and
• more working notebooks for exploring expectiles, distributional RL, the associated neural data, and our simulation experiments.

252-0526-00L Statistical Learning Theory

Joachim M. Buhmann’s advanced course on the physics of machine learning. This subject pushed my statistics skills to a new level, and deepened my theoretical understanding of machine learning. [Course details]

Final grade: 5.75 (6.0 scale), ETCS credits: 7.

The class involved fortnightly programming projects using Python’s scientific stack (numpy, scipy, etc.). I will not publish my work for these projects because, as I understand, the same projects are likely to be used in future semesters.

I have published some code I wrote will studying similar topics during the semester:

• Constant-shift embedding method for graph layout.
• Estimation of Gaussian mixture model parameters using EM algorithm.

252-1424-00L Models of Computation

Matthew Cook’s whistle-stop tour of many exciting and exotic models of computation (beyond the traditional FSAs, CFGs and TMs). The study of so many of models gave me a broader perspective on computation itself, which remains one of the most fascinating ideas I have ever studied. [Course details] [Course webpage]

Final grade: 6.0 (6.0 scale), ETCS credits: 6.

Assessment:

1. Challenging weekly puzzles exploring each new model of computation we studied. I was proud to complete every puzzle successfully. However, so as not to spoil these puzzles for future students I will not publish my solutions.

2. Final project to design and studying and then deliver a lecture about my own original model of computation.

   I am not certain how original my idea was, but I explored what I called a society of finite-state transducers—the result of composing the input/output streams of finite-state transducers in a (finite or infinite) network.

   I don’t have a recording of the presentation, which was partly at a virtual whiteboard.

363-0588-00L Complex Networks

Frank Schweitzer’s class on complex systems research using network models. [Course details]

Final grade: 6.0 (6.0 scale), ECTS credits: 4.

In weekly coding exercises, we implemented many ideas from complex networks such as random graph models and various measures of centrality. I will not to share my solutions since the same exercises are likely to be used in future semesters.

COMP90044 Research Methods

This subject introduced the fundamentals of research writing and conduct. We practiced research skills using a mock research topic—I chose distributional reinforcement learning. [Subject details]

Final grade: 94 (possibly top?)

Assessment:

1. A literature review on distributional reinforcement learning, written for a general CS audience. See: my literature review. In writing this review I challenged myself to distil the important ideas from this technical literature without using mathematical notation.

2. A research plan designed to investigate a research question identified during the review. See: my research plan. Note: I did not actually execute this research plan, indeed the plan is written for a well-resourced researcher.

3. A 5 minute talk describing my research plan. See: transcript and slide deck.