Writing by Matthew Farrugia-Roberts https://far.in.net/feed.xml 2026-07-26T00:00:00Z Matthew Farrugia-Roberts https://far.in.net/time-to-write 2026-07-26T00:00:00Z

I’m writing from the liminal space between London and San Francisco. Time works differently here (roughly speaking, three hours become ten). There’s also no internet, and that means no agents or students to prompt. Seems like the perfect opportunity to prime myself for my visit to Berkeley. Let’s see what I can make of it.

https://far.in.net/epiplexity 2026-07-08T00:00:00Z

Everyone seems to be talking about the paper by Finzi et al. (2026), “From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence.” I finally got around to studying it this week.

https://far.in.net/variational-inference 2026-06-30T00:00:00Z

In this short note, I recall variational inference and the evidence lower bound (ELBO). In contrast to the introductions I have seen, I start by describing variational inference as a loss minimisation problem in its own right without reference to the ELBO, and then derive the ELBO afterwards.

https://far.in.net/subscribing-to-arxiv 2026-06-29T00:00:00Z

How do you find out about new papers? Do you curate an academic Twitter feed? Do you set up alerts on Google Scholar for specific keywords? Do you keep an eye on the #papers channel in each of the Slacks/Discords you frequent? Do you ask an LLM for literature recommendations?

https://far.in.net/what-is-life 2026-03-20T00:00:00Z

This post spun out from my reading log.

https://far.in.net/complexity 2026-03-16T00:00:00Z

What is complexity? Etymologically, the prefix “com-” traces back to Latin, meaning “with, together,” and “plex” traces back to PIE, meaning “to plait,” as in, “to entwine.” Complex entities are those that are formed by entwining multiple elements together.

https://far.in.net/quotes-on-what-to-work-on 2026-03-16T00:00:00Z

A poem “The Church-Porch,” by George Herbert (1593–1633) (link).

https://far.in.net/replacing-guilt 2026-03-07T00:00:00Z

https://far.in.net/perceptrons-in-disguise 2026-03-05T00:00:00Z

Attention is a key element of the transformer architecture. It can be understood from several related perspectives: parallel soft search, reading from / writing to residual streams, loosely coupled QK/OV circuits, and message passing.

https://far.in.net/jax 2026-03-01T00:00:00Z

JAX is a Python library for writing and transforming array programs, such as arise in the course of deep learning research. The JAX API for writing array programs is similar to that of NumPy, but the JAX API for transforming array programs is utterly unique, effortlessly powerful, and delightfully elegant.

https://far.in.net/tempered-posterior 2026-02-27T00:00:00Z

Most of the results in singular learning theory are developed in the context of a generalised form of Bayesian inference, in which the posterior distribution has an additional (inverse) temperature parameter. In this note, I introduce this so-called “tempered posterior” and contrast it with the usual Bayesian posterior. I also discuss various motivations for departing from the Bayesian suggestion, including a derivation of the tempered posterior using the principle of maximum entropy.

https://far.in.net/readings-2019-to-2025 2026-02-01T00:00:00Z

Once upon a time, at an end-of-year teaching team dinner, I was talking with a friend of mine about New Year’s Resolutions. Every year, she said, she set herself the goal of reading 52 books the following year. What an amazing achievement! To read one book a week! I realised I wanted to spend substantially more time reading and, one day, achieve this myself.

https://far.in.net/one-size-fits-all 2026-01-02T00:00:00Z

Adrianne ran her fingers over the darts and let out a sigh of relief. The dress was a perfect fit! Part of her wasn’t surprised, of course: she had followed the pattern to the letter and measured every cut three times! But the rest of her wasn’t ready to believe it without proof.

https://far.in.net/free-wandb 2025-12-29T00:00:00Z

I spent the final weeks of 2024 migrating 5,203 deep learning experiment logs out of the Weights & Biases (W&B) cloud and the undocumented .wandb binary file format so that I could more flexibly and efficiently analyse the data in the course of writing a paper.

https://far.in.net/schwartz-distributions 2025-08-17T00:00:00Z

This is a technical note intended as an elementary introduction to Schwartz distributions and their calculus. Schwartz distributions are a concept from functional analysis used in singular learning theory to define the density of states, a key step in the bridge between algebraic geometry and Bayesian statistics.

https://far.in.net/blowing-up 2025-08-10T00:00:00Z

Blowing up is a concept from algebraic geometry. Blowing up turns a neighbourhood in a Euclidean space into a more complex manifold (called the ‘blow up’ of the original neighbourhood). The transformation preserves the Euclidean geometry in most parts of the neighbourhood, but expands a particular subspace to have extra dimensions. This can help to resolve singularities, with applications in singular learning theory.

https://far.in.net/value-ambiguity 2025-07-25T00:00:00Z

In this note, I introduce a toy environment that demonstrates instrumental/intrinsic value ambiguity. An agent repeatedly faces choices between two kinds of actions: intrinsically valuable actions (which are directly rewarded), and instrumentally valuable actions (which are not directly rewarded, but enable later intrinsically valuable actions). Crucially, I show how training in a subset of environment configurations creates a situation where the agent can’t reliably distinguish between these two kinds of value.

https://far.in.net/smile 2025-07-15T00:00:00Z

I’m not sure if it was just part of being an introvert or a product of a childhood spent playing Pokémon games, but for a large part of my early adulthood, I was slightly scared of locking eyes with strangers in public.

https://far.in.net/turing-trees 2025-06-15T00:00:00Z

I sometimes think about models of computation as an infinite network of infinite binary trees. I call the trees Turing trees and the network the Turing web. These objects offer an elegant graphical perspective on various concepts in the theory of computable functions. In this note, I explore this perspective. (I assume readers are already familiar with Turing machines.)

https://far.in.net/computing-expectiles 2025-06-11T00:00:00Z

Expectiles are a class of summary statistics generalising the well-known expected value. They have been relatively neglected since their introduction. Perhaps this is because the expectiles of a sample lack a well-known, simple and efficient calculation procedure?

https://far.in.net/expectiles 2025-06-11T00:00:00Z

Expectiles are a class of summary statistics generalising the well-known expected value [1, 2]. They have been relatively neglected since their introduction [3]. Perhaps this is because they lack a well-known, immediate interpretation like that of the expected value, or that of quantiles?

https://far.in.net/balanced-academic-orbit 2025-05-25T00:00:00Z

What is the difference between flight and orbit? Flight is a never-ending fight against gravity. It requires consistent effort to sustain. Given a finite fuel source, landing becomes inevitable. In contrast, orbit uses gravity to its advantage—falling only spurs you forward. Progress becomes essentially inevitable.

https://far.in.net/attention 2025-03-29T00:00:00Z

In this technical note, I explore attention, a key element of the transformer architecture driving recent progress in deep learning applications. I review four standard perspectives on how to understand the attention mechanism. The four perspectives are as follows.

https://far.in.net/approximinimax-three-ways 2025-02-25T00:00:00Z

This is a short technical note written in the process of working out some definitions for a paper about using an approximate minimax regret training objective to mitigate goal misgeneralisation in advanced deep reinforcement learning systems. I define three different approximate relaxations of the minimax objective, and show that the definitions are related under certain assumptions.

https://far.in.net/minimax-three-ways 2025-02-23T00:00:00Z

This is a short technical note written in the process of working out some definitions for a paper about using an approximate minimax regret training objective to mitigate goal misgeneralisation in advanced deep reinforcement learning systems. I define a minimax objective in three different ways, and show that the definitions are equivalent under certain assumptions.

https://far.in.net/iclr2025-volume 2024-12-08T00:00:00Z

I was a reviewer for ICLR 2025. I spent a lot of time writing my reviews and engaging with the review process. Much more than (most of) the reviews on my submissions. I was curious about the size of other reviewers’ reviews and also the amount of author–reviewer discussion that goes on during the review period.

https://far.in.net/universal-icl 2024-12-05T00:00:00Z

This is a short technical note adapted from a presentation I gave, apropos of nothing, at the metauni SLT seminar on Thursday, December 5th, 2024.

https://far.in.net/zuckerberg 2024-07-26T00:00:00Z

In internet tradition, I am putting aside my other duties and taking the day to write.

https://far.in.net/future-ethics-lecture 2024-05-23T00:00:00Z

COMP90087 The Ethics of Artificial Intelligence is a Master’s subject at the University of Melbourne designed as an introduction to moral philosophy and applied ethics for students with a technical background. It offers students who will go on to contribute to building AI systems of the future a framework for evaluating, critiquing, and designing AI technology in a socially responsible manner.

https://far.in.net/responsibility-letter 2023-07-28T00:00:00Z

An open letter to the Hon. Ed Husic, MP, Minister for Industry and Science. Submitted through the consultation on supporting responsible AI run by the Department of Industry, Science and Resources from June 2023 to August 2023.