--- date: "Last updated: August 2026" title: News --- Welcome to my academic web log. Here I share news about my career and my research, and related topics. I occasionally link to longer [essays](far-be-it-from-me) reflecting upon research/personal projects. ## Coming soon In the works: - I am trying to write more. I have a lot to say, and more than ever it feels like I might miss my chance if I don't say it soon. Wish me luck! - I am studying a range of topics in learning theory. - My students and I are writing a paper on reward multiplicity as a unifying framework for understanding various issues in AI alignment, and principled ways to address these issues. Conferences: - **ILIAD 2026, Lighthaven.** I am excited to be joining for my first Iliad conference. - **NeurIPS 2026, Sydney.** Nah, yeah, bloody oath mate, d'ya reckon I'd be sprung missin' Chrissie down under? Strewth! You must 'av a few roos loose in the top paddock. ## 2026 August: - Website improvements: [RSS (Atom) feed](feed); copy KaTeX equations as LaTeX source; add `.md` to any URL for a plain-text version. Thanks for the help, Fable! July: - I passed my Transfer of Status milestone. According to the University of Oxford, I am no longer a Probationer Research Student (I am finally a Doctor of Philosophy student). - I wrote a [technical note](epiplexity) on the theoretical concept of epiplexity from the paper [Finzi et al. (2026)](https://arxiv.org/abs/2601.03220), "From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence." June: - I wrote a [technical note](variational-inference) on variational inference and the evidence lower bound, inverting the traditional presentation by talking about variational inference in its own terms rather than via the evidence lower bound. - I wrote [an essay](subscribing-to-arxiv) on why and how I try to read all of the titles posted to arXiv in CS/ML categories each day. - I attended the **CHAI workshop 2026** in Pacific Grove, California. It was my first visit to CHAI in person, since my 2021 internship happened during the pandemic. - I defended my thesis plan for my Transfer of Status milestone. May: - I published a workshop paper on how representations develop over long-run training time: ["Structure and scale in simplicial sequence modelling"](https://openreview.net/forum?id=aFlGLdTzuq). - Together with my students Afiq Abdillah Effiezal Aswadi, [Olly Britton](https://ollybritton.com/), and Ross Baker, I published a paper on memorisation versus generalisation under non-stationary training data: ["Temporal task diversity: Inductive biases under non-stationarity in synthetic sequence modelling"](https://arxiv.org/abs/2605.18281). - I attended the Technical AI Safety Conference ([TAIS 2026](https://tais2026.cc/)). It was rewarding to see my students presenting our research. It was fun to support the conference in my capacity as an OAISI member. Big thanks to the AI Safety events team at Noeon for bringing the conference to Oxford! April: - Soon I will have to complete *Transfer of Status,* which is my first major milestone of my doctoral studies (in which I transition from being a "probationary research student" to a proper DPhil candidate here in Oxford). - In preparation, I've reflected a lot on how I have spent the first third of my PhD and how I want to spend the rest. I am resolving to focus more in my selection of research topics, and redouble my efforts towards making my PhD an intellectually transformative experience, starting this summer! - I was also asked to write a literature review and a thesis plan. In my case, this turned into a short treatise on the state of the science of deep learning and a conceptual path forward unifying mechanistic interpretability, scaling laws, and computer science. I don't know if it's a realistic vision, but I plan to reflect more on this and write a publishable version soon. - Claude and I have been having some fun administering [MFR's tiny TPU cluster](https://github.com/matomatical/tpus) to support my experiments and those of my students. If you want to administer a TPU cluster for science of deep learning research, feel free to check out our open-source config. This cluster has pretty low utilisation, I'm happy to share access with people I trust who are in need of a little compute. *Update: Unfortunately, my TRC allocation was not renewed, so the cluster has been decommissioned.* - I helped developed a self-contained [invitation to singular learning theory](https://www.linkedin.com/in/kai-ogden-18b7a828a/) as part of [the Iliad intensive](https://www.lesswrong.com/posts/dWQnLi7AoKo3paBXF). Thanks to Iliad for making this great initiative happen and my coauthors [Zach Furman](https://zachfurman.com/) and [Kai Ogden](https://www.linkedin.com/in/kai-ogden-18b7a828a/) for your excellent contributions to making SLT more accessible for everyone. March: - I read *What is Life?* by Erwin Schrödinger and wrote [some reflections](what-is-life). See more readings on my [reading log](readings-2026). - I wrote [an essay about Kolmogorov complexity](complexity). - As part of my ongoing reflection on how to spend the rest of my PhD, I collected [a few nice quotes](quotes-on-what-to-work-on) on how to choose what to work on as a researcher. - I'm soft-releasing [matthewplotlib](https://matthewplotlib.far.in.net/), a Python plotting library that aspires to *not be painful.* - I finally tried agentic coding. Claude Opus 4.6 and I built [this website](https://all-you-need.far.in.net) together to track the "X is all you need" meme. - I read *Replacing Guilt* by Nate Soares. I took notes and wrote a [detailed summary](replacing-guilt), in addition to my regular reflection on this year's [reading log](readings-2026#replacing-guilt-by-nate-soares). - I dug up some notes from an old paper review, and posted a short [technical note](perceptrons-in-disguise) on how to view an attention head as a multi-layer perceptron for which the weights are calculated as linear transformations of the embeddings of the context. - People sometimes ask me why I love JAX so much. So, I [wrote it down!](jax) February: - As part of my ongoing studies of singular learning theory, I wrote a [technical note](tempered-posterior) on the derivation of the tempered posterior via the principle of maximum entropy. - With collaborators from Timaeus, I published a paper extending singular learning theory to the reinforcement learning context: ["Stagewise reinforcement learning and the geometry of the regret landscape"](https://arxiv.org/abs/2601.07524). - I read *Feline Philosophy* by John Gray and wrote some notes on my [reading log](readings-2026#feline-philosophy-by-john-gray). January: - I read *Deep Utopia* by Nick Bostrom and wrote some notes on my [reading log](readings-2026#deep-utopia-by-nick-bostrom). - I reflected on all of the books I read [between 2019 and 2025](readings-2019-to-2025), and started writing (in more detail) about the books I am reading [in 2026](readings-2026). - I published (most of) my reboot of [Hi, JAX! An introduction to vanilla JAX for deep learning research](https://github.com/matomatical/hijax). There are currently 11 lectures (approx. 13hrs) covering the fundamental concepts needed to train and analyse deep neural networks in JAX. I plan to record a couple more lectures after some upcoming deadlines pass---consider [subscribing](https://www.youtube.com/@MatthewFarrugiaRoberts) if you are interested! - I published an experiment in rationalist fiction, entitled ["One size fits all"](one-size-fits-all). If you give it a read, message me what you think the moral of the story is! ## 2025 December: - I wrote a story about the time I spent my Christmas/New Years break [freeing 118 GB of W&B experiment data from a broken binary format](free-wandb). - I recorded 13 hours of JAX tutorials for a reboot of "Hi, JAX!", coming soon! October: - I attended the UK AI Security Institute's Alignment conference in London. It was a great event and I made a lot of progress reflecting on research ideas, some of which I hope to write about soon. - I signed the [FLI Statement on Superintelligence](https://superintelligence-statement.org/) "We call for a prohibition on the development of superintelligence, not lifted before there is (1) broad scientific consensus that it will be done safely and controllably, and (2) strong public buy-in." - I joined the teaching team for Oxford's first course on [AI Safety and Alignment](https://robots.ox.ac.uk/~fazl/aisaa/). I wrote and delivered a workshop on [Specification gaming and goal misgeneralisation in grid worlds](https://colab.research.google.com/drive/1rAGN4Ngn2Rk1sJekfmnei1gTgwlEFqYM?usp=sharing). I had a great time! Thanks to the rest of the teaching team and all of the awesome students for your sustained engagement throughout the one-week intensive. September: - I spent the summer vacation studying statistical decision theory and singular learning theory. I made a lot of progress and had a lot of fun studying these topics. I hope this isn't the last time I get to enjoy learning like that, but it feels like it's getting harder and harder to justify these investments. August: - I wrote a short [technical note](blowing-up) on visualising the simplest non-trivial example of blowing up a point in a plane, a component of resolution of singularities with applications in singular learning theory. - I wrote a [technical note](schwartz-distributions) with an elementary introduction to Schwartz distributions, a theory of generalised functions with applications in singular learning theory. - [Like many others](https://www.asteriskmag.ai/p/reflections-on-asterisks-ai-fellows), I was not selected for the Asterisk AI blogging fellowship. I often apply for things and I am often rejected, but this one hurt more than most because I put a bit more of my soul into the application. July: - I wrote a [technical note](value-ambiguity) on a toy model of instrumental/intrinsic value ambiguity, which is one potential cause of goal misgeneralisation. - Years ago, I decided to smile when I lock eyes with people. I've never looked back. I wrote [a post](smile) reflecting on this. - Our paper ["Loss landscape degeneracy and stagewise development in transformers"](https://arxiv.org/abs/2402.02364) was accepted to **TMLR!** Joint work with collaborators from Timaeus and Monash University. June: - I posted a [technical note](turing-trees) on the idea of Turing trees, a graphical perspective on some concepts in the theory of computation. This is intended to be the first in a series of posts about modelling computation. - I dug up some notes from an old coursework project on distributional reinforcement learning and posted two brief notes about [why expectiles are cool](expectiles) and [how to compute them from a sample](computing-expectiles). I have more to say about expectiles but it will have to wait for another day. May: - In the first of a series(?) of posts on reflecting on my goals for grad school, I published an [essay](balanced-academic-orbit) on finding a sustainable balance in the pursuit of long-term research goals. - Our paper on mitigating goal misgeneralisation using minimax regret autocurricula was accepted to RLC 2025. [Preprint](https://arxiv.org/abs/2507.03068) on arXiv. Joint work with [Karim Abdel Sadek](https://karim-abdel.github.io/) and other collaborators from Krueger AI Safety Lab and Google DeepMind. March: - I was thinking about the attention mechanism from transformers, and published a [technical note](attention) that reviews various perspectives on the attention computation. - I joined the executive committee for the Oxford AI Safety Initiative, the AI safety student society in Oxford. February: - In the process of preparing a paper on mitigating goal misgeneralisation using minimax regret autocurricula, I published two short notes on the relationships between various [exact](minimax-three-ways) and [approximate](approximinimax-three-ways) formulations of a minimax training objective. - We reflected on, rewrote, and republished the preprint "The developmental landscape of in-context learning" (now titled ["Loss landscape degeneracy drives stagewise development in transformers"](https://arxiv.org/abs/2402.02364)) from last February. We realised that what makes this paper special is that it is the first to make an empirical case for a fundamental link between loss landscape degeneracy and development in modern deep learning. Joint work with collaborators from Melbourne and Timaeus. - We published a position paper on arXiv: ["You are what you eat: AI alignment requires understanding how data shapes structure and generalisation"](https://arxiv.org/abs/2502.05475). Joint work with collaborators from Melbourne, Timaeus, and beyond. January: - We published a preprint on arXiv: ["Dynamics of transient structure in in-context linear regression transformers"](https://arxiv.org/abs/2501.17745). Joint work with collaborators from Melbourne and Timaeus ## 2024 December: - A paper reporting some of the theoretical results from my Master's thesis was published at the [Machine Learning and Compression Workshop @ NeurIPS 2024](https://neuralcompression.github.io/workshop24). - After participating in the ICLR 2025 review process, I was curious about how much discussion actually happens between authors and reviewers. I learned to use [the OpenReview API](https://docs.openreview.net/reference/api-v2) and conducted [a 'volumetric' analysis](iclr2025-volume) of the review process. - Flexing the website's new support for KaTeX, I published [a short note](universal-icl) presenting a proof that transformers are universal in-context function approximators. The proof is adapted from [a paper](https://openreview.net/forum?id=YE6N8htoFQ) currently under public, double-blind review for ICLR 2025. - I added support for static KaTeX rendering to this website. Actually, this was not much more complicated than adding [pandoc-katex](https://github.com/xu-cheng/pandoc-katex) as a pandoc filter to my makefile. I'm proud that no JavaScript is involved. - I started collecting my writings on [a page called 'far be it from me'](far-be-it-from-me), which you could consider to be my blog. I'll continue to announce new posts here when they are first released, whereas the new page lists posts by topic. October: - I built a home for [todo club](todo-club), an intermittent casual online co-working group. - I officially joined Magdalen College and the University of Oxford! September: - I moved to Oxford to start a DPhil in the Department of Computer Science! July: - I wrote a [critique](zuckerberg) of Mark Zuckerberg's recent [letter](https://about.fb.com/news/2024/07/open-source-ai-is-the-path-forward/) about open source and the future of AI. - We published a paper at the [HiLD workshop](https://sites.google.com/view/hild2024) at ICML 2024, ["Loss landscape geometry reveals stagewise development of transformers"](https://openreview.net/forum?id=2JabyZjM5H). The paper received a **best papers of HiLD award!** Joint work with collaborators from Melbourne and Timaeus. - I ran [Hi, JAX!](hijax-2024), a free online introductory JAX course from July 11 to September 8. May: - I gave a [guest lecture](future-ethics-lecture) on ethics and the future of intelligence for the subject [COMP90087 The Ethics of Artificial Intelligence](https://handbook.unimelb.edu.au/2024/subjects/comp90087) at the University of Melbourne. A [recording](future-ethics-lecture) is available. February: - Together with collaborators from Melbourne and Timaeus I published a preprint on arXiv: ["The developmental landscape of in-context learning"](https://arxiv.org/abs/2402.02364). ## 2023 December: - I attended [NeurIPS 2023](https://neurips.cc/Conferences/2023) in New Orleans. ``{=html} November: - I attended the [2023 Developmental Interpretability Conference](https://www.lesswrong.com/posts/QpFiEbqMdhaLBPb7X/apply-for-the-2023-developmental-interpretability-conference) at Wytham Abbey, Oxford. October: - I led a [virtual workshop](https://far.in.net/tpu-go-brrr) on using TPU virtual machines to accelerate machine learning research, for people without experience using a VM. A [recording](https://www.youtube.com/watch?v=VgnVLBpqmUo) is available. September: - The first of two papers based on the results in my Master's thesis was accepted for poster presentation at NeurIPS 2023: ["Functional equivalence and path connectivity of reducible hyperbolic tangent networks"](https://arxiv.org/abs/2305.05089). - I moved to Cambridge, UK, to work for three months as an RA in [David Krueger's AI safety lab](https://www.davidscottkrueger.com/), working with [Usman Anwar](https://uzman-anwar.github.io/) on understanding goal misgeneralisation. August: - I got a grant to work on independent AI safety research for six weeks. I shared some of my progress in a standalone blog: [Seven Saturdays with Singular Learning Theory](seven-saturdays-with-slt). Though the grant has expired, work for this project is ongoing. July: - I was interviewed by Michaël Trazzi for The Inside View. \[[youtube](https://youtu.be/jYxON9MIkJM?t=530), \~8 minutes\] - I attended my first machine learning conference: ICML 2023, in Honolulu, Hawai'i. I presented the results of my 2021 CHAI internship, in reward learning theory, together with co-authors [Adam Gleave](https://www.gleave.me/) and [Joar Skalse](https://www.fhi.ox.ac.uk/team/joar-skalse/). - I finished my RA contract with Tim Miller at the University of Melbourne. - I participated in the Australian Government's [consultation on safe and responsible AI in Australia](https://consult.industry.gov.au/supporting-responsible-ai). I signed the open letter ["Australians for AI Safety"](https://www.australiansforaisafety.com.au/) drafted by the [Good Ancestors Policy team](https://www.goodancestors.org.au/policy) for submission to the consultation. I also wrote [my own open letter](https://far.in.net/responsibility-letter) for submission to the consultation. June: - I attended the inaugural [Singular Learning Theory & Alignment Summit](https://singularlearningtheory.com/2023), in Berkeley, California. I presented a talk on the results of my thesis: "Hidden unit acrobatics: An introduction to swaps, flips, and other symmetries of singular networks and their applications". \[[youtube](https://www.youtube.com/watch?v=BZaMNAhMgX0), 45 minutes\] \[[interactive notebook](https://hidden-unit-acrobatics.far.in.net/)\] - I published another preprint on arXiv, based on the second of the two major results in my Master's thesis: ["Computational complexity of detecting proximity to losslessly compressible neural network parameters"](https://arxiv.org/abs/2306.02834). May: - "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." I support the above [Statement on AI Risk](https://www.safe.ai/statement-on-ai-risk) from the Center for AI Safety. I attempted to sign the statement officially, but my signature is not listed, perhaps because my I was considered insufficiently notable as a research assistant. - I published a preprint on arXiv, based on one of the two major results in my Master's thesis: ["Functional equivalence and path connectivity of reducible hyperbolic tangent networks"](https://arxiv.org/abs/2305.05089). April: - The paper from my 2021 virtual CHAI internship was accepted at ICML 2023: ["Invariance in policy optimization and partial identifiability in reward learning"](https://arxiv.org/abs/2203.07475). March: - I was one of the initial signatories to the Future of Life Institute's open letter calling to [Pause Giant AI Experiments](https://futureoflife.org/open-letter/pause-giant-ai-experiments/). February: - I joined the teaching team for [COMP90087 The Ethics of Artificial Inteligence](https://handbook.unimelb.edu.au/2023/subjects/comp90087) for semester 1, 2023. January: - I started a 6-month contract as an RA in Tim Miller's explainable AI lab at the University of Melbourne. - I attended metauni's [Festival 2023](https://metauni.org/festival/). I gave a talk entitled "Considering a Career in AI Safety" \[[youtube](https://youtu.be/G5tyKpG20Wc)\]. ## 2022 December: - I finally graduated from my Master of Computer Science degree at the University of Melbourne. November: - I gave [several talks](mthesis#recorded-talks) on my Master's thesis results at the metauni [SLT seminar](https://metauni.org/slt/). October: - I submitted my [minor thesis](mthesis) on structural degeneracy in neural networks to the School of Computing and Information Systems at the University of Melbourne. July: - Our full paper ["Teaching simple constructive proofs with Haskell programs"](https://arxiv.org/abs/2208.04699v1) from TFPIE 2022 was accepted for publication in the official EPTCS proceedings. - Our paper ["Programming to learn: logic and computation from a programming perspective"](https://dl.acm.org/doi/10.1145/3502718.3524814) was presented at ITiCSE 2022 in Dublin, Ireland, by my co-author [Bryn Jeffries](https://scholar.google.com/citations?user=n_PgbpwAAAAJ). - I married Annie-Emma Jean Italiano. May: - I finally made a personal website. (I didn't want to let myself graduate from my *second* computer science degree without one.) The website is written in markdown, compiled to HTML with pandoc, styled with hand-made CSS, and hosted with GitHub pages. March: - I attended [TFPIE 2022](https://wiki.tfpie.science.ru.nl/TFPIE2022), a virtual conference hosted in Kraków, Poland. I presented our extended abstract "Teaching simple constructive proofs with Haskell programs" with my co-author [Harald Søndergaard](https://people.eng.unimelb.edu.au/harald/).