Welcome to my academic web log. Here I share news about my career and my research, and related topics. I occasionally link to longer essays reflecting upon research/personal projects.

§2025

October:

• I attended the UK AI Security Institute’s Alignment conference in London. It was a great event and I made a lot of progress reflecting on research ideas, some of which I hope to write about soon.

• I signed the FLI Statement on Superintelligence “We call for a prohibition on the development of superintelligence, not lifted before there is (1) broad scientific consensus that it will be done safely and controllably, and (2) strong public buy-in.”

• I joined the teaching team for Oxford’s first course on AI Safety and Alignment. I wrote and delivered a workshop on Specification gaming and goal misgeneralisation in grid worlds. I had a great time! Thanks to the rest of the teaching team and all of the awesome students for your sustained engagement throughout the one-week intensive.

September:

• I spent the summer vacation studying statistical decision theory and singular learning theory. I made a lot of progress and had a lot of fun studying these topics. I hope this isn’t the last time I get to enjoy learning like that, but it feels like it’s getting harder and harder to justify these investments.

August:

• Like many others, I was not selected for the Asterisk AI blogging fellowship. I often apply for things and I am often rejected, but this one hurt more than most because I put a bit more of my soul into the application.

• I wrote a technical note with an elementary introduction to Schwartz distributions, a theory of generalised functions with applications in singular learning theory.

• I wrote a short technical note on visualising the simplest non-trivial example of blowing up a point in a plane, a component of resolution of singularities with applications in singular learning theory.

July:

• I wrote a technical note on a toy model of instrumental/intrinsic value ambiguity, which is one potential cause of goal misgeneralisation.

• Years ago, I decided to smile when I lock eyes with people. I’ve never looked back. I wrote a post reflecting on this.

• Our paper “Loss landscape degeneracy and stagewise development in transformers” was accepted to TMLR! Joint work with collaborators from Timaeus and Monash University.

June:

• I posted a technical note on the idea of Turing trees, a graphical perspective on some concepts in the theory of computation. This is intended to be the first in a series of posts about modelling computation.

• I dug up some notes from an old coursework project on distributional reinforcement learning and posted two brief notes about why expectiles are cool and how to compute them from a sample. I have more to say about expectiles but it will have to wait for another day.

May:

• In the first of a series(?) of posts on reflecting on my goals for grad school, I published an essay on finding a sustainable balance in the pursuit of long-term research goals.

• Our paper on mitigating goal misgeneralisation using minimax regret autocurricula was accepted to RLC 2025. Preprint on arXiv. Joint work with Karim Abdel Sadek and other collaborators from Krueger AI Safety Lab and Google DeepMind.

March:

• I was thinking about the attention mechanism from transformers, and published a technical note that reviews various perspectives on the attention computation.

February:

• In the process of preparing a paper on mitigating goal misgeneralisation using minimax regret autocurricula, I published two short notes on the relationships between various exact and approximate formulations of a minimax training objective.

• We reflected on, rewrote, and republished the preprint “The developmental landscape of in-context learning” (now titled “Loss landscape degeneracy drives stagewise development in transformers”) from last February. We realised that what makes this paper special is that it is the first to make an empirical case for a fundamental link between loss landscape degeneracy and development in modern deep learning. Joint work with collaborators from Melbourne and Timaeus.

• We published a position paper on arXiv: “You are what you eat: AI alignment requires understanding how data shapes structure and generalisation”. Joint work with collaborators from Melbourne, Timaeus, and beyond.

January:

• We published a preprint on arXiv: “Dynamics of transient structure in in-context linear regression transformers”. Joint work with collaborators from Melbourne and Timaeus

§2024

December:

• A paper reporting some of the theoretical results from my Master’s thesis was published at the Machine Learning and Compression Workshop @ NeurIPS 2024.

• I started collecting my writings on a page called ‘far be it from me’, which you could consider to be my blog. I’ll continue to announce new posts here when they are first released, whereas the new page lists posts by topic.

• After participating in the ICLR 2025 review process, I was curious about how much discussion actually happens between authors and reviewers. I learned to use the OpenReview API and conducted a ‘volumetric’ analysis of the review process.

• Flexing the website’s new support for KaTeX, I published a short note presenting a proof that transformers are universal in-context function approximators. The proof is adapted from a paper currently under public, double-blind review for ICLR 2025.

• I added support for static KaTeX rendering to this website. Actually, this was not much more complicated than adding pandoc-katex as a pandoc filter to my makefile. I’m proud that no JavaScript is involved.

October:

• I built a home for todo club, an intermittent casual online co-working group.

• I officially joined Magdalen College and the University of Oxford!

September:

• I moved to Oxford to start a DPhil in the Department of Computer Science!

July:

• I wrote a critique of Mark Zuckerberg’s recent letter about open source and the future of AI.

• We published a paper at the HiLD workshop at ICML 2024, “Loss landscape geometry reveals stagewise development of transformers”. The paper received a best papers of HiLD award! Joint work with collaborators from Melbourne and Timaeus.

• I ran Hi, JAX!, a free online introductory JAX course from July 11 to September 8.

May:

• I gave a guest lecture on ethics and the future of intelligence for the subject COMP90087 The Ethics of Artificial Intelligence at the University of Melbourne. A recording is available.

February:

• Together with collaborators from Melbourne and Timaeus I published a preprint on arXiv: “The developmental landscape of in-context learning”.

§2023

December:

• I attended NeurIPS 2023 in New Orleans.

November:

• I attended the 2023 Developmental Interpretability Conference at Wytham Abbey, Oxford.

October:

• I led a virtual workshop on using TPU virtual machines to accelerate machine learning research, for people without experience using a VM. A recording is available.

September:

• The first of two papers based on the results in my Master’s thesis was accepted for poster presentation at NeurIPS 2023: “Functional equivalence and path connectivity of reducible hyperbolic tangent networks”.

• I moved to Cambridge, UK, to work for three months as an RA in David Krueger’s AI safety lab, working with Usman Anwar on understanding goal misgeneralisation.

August:

• I got a grant to work on independent AI safety research for six weeks. I shared some of my progress in a standalone blog: Seven Saturdays with Singular Learning Theory. Though the grant has expired, work for this project is ongoing.

July:

• I attended my first machine learning conference: ICML 2023, in Honolulu, Hawai’i.

  • I presented the results of my 2021 CHAI internship, in reward learning theory, together with co-authors Adam Gleave and Joar Skalse.
  • I was interviewed by Michaël Trazzi for The Inside View. [youtube, ~8 minutes]

• I finished my RA contract with Tim Miller at the University of Melbourne.

• I participated in the Australian Government’s consultation on safe and responsible AI in Australia.

  • I signed the open letter “Australians for AI Safety” drafted by the Good Ancestors Policy team for submission to the consultation.
  • I also wrote my own open letter for submission to the consultation.

June:

• I attended the inaugural Singular Learning Theory & Alignment Summit, in Berkeley, California.
  • I presented a talk on the results of my thesis: “Hidden unit acrobatics: An introduction to swaps, flips, and other symmetries of singular networks and their applications”. [youtube, 45 minutes] [interactive notebook]
• I published another preprint on arXiv, based on the second of the two major results in my Master’s thesis: “Computational complexity of detecting proximity to losslessly compressible neural network parameters”.

May:

• “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” I support the above Statement on AI Risk from the Center for AI Safety. I attempted to sign the statement officially, but my singature is not listed, perhaps because my I was considered insufficiently notable as a research assistant.

• I published a preprint on arXiv, based on one of the two major results in my Master’s thesis: “Functional equivalence and path connectivity of reducible hyperbolic tangent networks”.

April:

• The paper from my 2021 virtual CHAI internship was accepted at ICML 2023: “Invariance in policy optimization and partial identifiability in reward learning”.

March:

• I was one of the initial signatories to the Future of Life Institute’s open letter calling to Pause Giant AI Experiments.

February:

• I joined the teaching team for COMP90087 The Ethics of Artificial Inteligence for semester 1, 2023.

January:

• I started a 6-month contract as an RA in Tim Miller’s explainable AI lab at the University of Melbourne.

• I attended metauni’s Festival 2023.

  • I gave a talk entitled “Considering a Career in AI Safety”. [youtube]

§2022

December:

• I finally graduated from my Master of Computer Science degree at the University of Melbourne.

November:

• I gave several talks on my Master’s thesis results at the metauni SLT seminar.

October:

• I submitted my minor thesis on structural degeneracy in neural networks to the School of Computing and Information Systems at the University of Melbourne.

July:

• Our full paper “Teaching simple constructive proofs with Haskell programs” from TFPIE 2022 was accepted for publication in the official EPTCS proceedings.

• Our paper “Programming to learn: logic and computation from a programming perspective” was presented at ITiCSE 2022 in Dublin, Ireland, by my co-author Bryn Jeffries.

• I married Annie-Emma Jean Italiano.

May:

• I finally made a personal website. (I didn’t want to let myself graduate from my second computer science degree without one.) The website is written in markdown, compiled to HTML with pandoc, styled with hand-made CSS, and hosted with GitHub pages.

March:

• I attended TFPIE 2022, a virtual conference hosted in Kraków, Poland.
  • I presented our extended abstract “Teaching simple constructive proofs with Haskell programs” with my co-author Harald Søndergaard.