This email is brought to you by the Baillie's Reading Room of the Mitchell Library (again), this time from the mezzanine which has a different carpet to the rest of the room! It's a good one and it's giving knowledge and learning.

What's up

V&A Digital Design Weekend was great. It was honestly a really positive experience from start to finish, huuuge shouts to the team who run the event for making the setup and pack down really enjoyable. I felt the work was received really well, and had so many good conversations over the weekend. Sometimes these things can feel vanishingly rare, having honest, meaningful and good faith chats about technology and data and AI, and it makes me really appreciate these spaces.

Autumn has hit and I'm wondering about hobbies. Yesterday I was riding home at the end of the day and came across a huge group of other cyclists congregated outside of Queens Park, I would guess it was for something like Critical Mass? I was struck by how class it would be to do something fun and wholesome like that but instead I kept cycling so I could keep writing this email, and I feel aware at the minute that I fill most of my free time with ambitions of extracurricular productivity while I'm also skirting on the edge of burnout. The gym is the closest thing I have to a hobby right now and even that is just a sort of goal-driven organised productivity measurement platform. I think it would be cool to do more things solely for the joy of doing them. Do you have hobbies? Please tell me how to make this part of my life.

If you're in London, I'll be at Peckham Digital again this month, as always give me a shout if you'd like to chat and if you're in London make sure you don't miss it. If you're in Glasgow, I'll be at the Lovelace-Hodgkin Symposium next week which I've not been to before but it sounds like it might be good!

Below is: a collection of links to fun things to read relating to what we talked about last time, a take on all the chat from tech CEOs that's been filling headlines and how that relates to what we've been talking about, and then some promises that future emails will be more entertaining.

Follow ups from last time

I have to admit I was concerned as I sent my last email that my take on how we talk about AI's power – that we're doing the technology overlords a favour when we make proclamations of doom and disaster sans nuance, bolstering the clout of their products – would be received as a bit controversial; that maybe it would be offensive to suggest this in the face of the scale of the harms we were talking about. So it's been encouraging to come across a lot of articles echoing the same ideas and adding some valuable thoughts:

• Eryk Salvaggio had some good things to say (as usual) in his piece Models Don't Go Rogue, the headline giving away the key point: the AI models do exactly what the AI models are set up to to do by the people who set up the AI models, and saying they "went rogue" every time they cause harm is just a convenient way of relinquishing responsibility.
• via Sara (thanks!) a good insta graphic , key quote:

AI companies are selling the tech and warning it may destroy the world. That contradiction is the business model! The more world-ending the technology sounds, the more indispensable and valuable the handful of companies that claim to control it become.

• From The Guardian: Why tech companies might be only too happy for us to believe AI will ‘kill us all’, nice quote:

“The framing is, ‘we’re inventing fire’… And that means governments have to treat them like magical gods and let them set the rules on their mysterious creation.”

• Henry Farrell writes quite a lot about how we easily misattribute agency and intelligence to machines that do things they're simply programmed to do, because the ways they do these things are more complex than we can follow. A recent piece, Machine god metaphors eat your brain, linked back to something he wrote in 2024 which summarised nicely:

LLMs, like markets, are not conscious actors themselves but complex outcomes of the interactions of myriad agents, filtered through an imperfect technology of representation.

• and finally! I was extremely excited to read Timnit Gebru and Emily M. Bender (if you know you know!) in MIT Technology Review with Don’t be fooled by this summer of AI hype summarising virtually everything I was trying to say last month with much more compelling and meaningful arguments. Nothing says it better than this:

Describing them as “superintelligence” or “rogue models” ascribes agency to products rather than to the companies building them. This framing markets these companies’ products as “superhuman” and, at the same time, helps the companies evade accountability for their actions.

In future, I am available if you would like to hire me to provide hot takes roughly one or two or maybe even twelve days before they hit MIT.

How do you feel about…

"market slowdown"

In the interests of thinking out loud and in earnest, I'd like to develop a bit on the working hypothesis I shared last month. Previously I'd suggested that the reason AI corporations wax lyrical about the extreme and dangerous power of their products is not so that they can nobly safeguard humanity, but rather so that they can increase their offering's value to their biggest customers such as governments and militaries. This is true, but I think we can still go deeper and detect something telling within the specificity of several big tech CEOs' press releases over the past few weeks, in which they have been calling directly for government intervention to enforce a slowdown of the market. Ostensibly this is because they want to curb the dangerous potential of AI, but in reality I suspect the motivation might be this: these companies really need to stop spending money, or they'll collapse, but at the same time they really can't stop spending money, or they'll collapse.

The market slowdown would theoretically be an opportunity for the big AI players to drastically reduce spend, recoup some investment, and bolster market confidence. OpenAI, for example, reported a loss of over 20 billion dollars last year. They spent $34bn against revenue of $13bn, and more than half of their expenditure ($19bn) was on research and development. This is obviously some extremely simplistic maths coming from someone who knows nothing about economics, but I do know that training models is very, very expensive and notionally I reckon if you'd just spent $19bn doing it then you'd be looking at your company's $20bn loss and sensing some money-saving potential.

OpenAI and their regulatory frenemies (Anthropic, Google, Meta, etc) can't voluntarily halt their aggressive R&D programmes without tanking their company's value in a cascade that would likely see the entire enterprise collapse almost immediately. Capitalism demands eternal growth and punishes stagnation, of course, but these companies are also built on blitzscaling: a last-man-standing game of expanding rapidly, outspending everyone and using unfathomable amounts of investment to run at outrageous levels of loss until all your rivals have collapsed and you can monopolistically mop up with as much predatory pricing as your heart desires in order to finally recoup the gigantic losses you accrued along the way. Each of the big AI players' entire value proposition is that they are at the bleeding edge of AI capability (or at least within 3-6 months of catching up to it); any deviation from this state undermines the raison d'etre of the venture and will likely see investors and consumers flee elsewhere, interrupting the cashflow that was keeping the company in the blitzscaling game. Once you're out, there's no coming back.

But! If an external power were to intervene and halt everyone's development, you could stop spending all that R&D cash without everyone thinking you've lost your clout: it's not that you couldn't afford to keep burning money, it's that you had to because of the government. This is all very theoretical though, and the tech CEOs' calls for a market slowdown are probably pretty futile: the US Government is wildly opposed to imposing a market slowdown on its AI corporations for the same reason the AI corporations won't impose it on themselves; they're in an arms race with the rest of the world and can't halt their aggressive economy without tanking their country's value in a cascade that would likely see the entire enterprise collapse almost immediately. The real-world outcome of the calls for a market slowdown has been Trump mediating a very soft "morally binding" and "self-policing" non-legally-binding agreement.

Let's stay in our hypothetical dreamland for a moment, however, and consider the potential outcomes of some real, meaningful legislation if it were to happen. Much as we all should be reticent about siding with the billionaire oligarchs here, AI legislation is creating all sorts of unlikely alliances and a market slowdown seems like a pretty good shout. AI is indeed powerful and dangerous (we're allowed to say this here because we have nuance), and it would in the short-term reduce some of the immediate potential threats of the technology. Since you and I are not idiots, we don't believe unregulated markets are good for anyone other than oligarchs.

I am concerned, though, that as powerless critics of the oligarchs we might get a little bit blinded by a thirst for something that looks like vengeance; excited by the schadenfreude of a billionaire's plans for market domination being legislatorially hindered in a world where this seems so rare. Regulation, however, isn't justice. Stating the obvious of course, but the aim of market regulation is to make regular the market, to encourage it to flow uninterrupted, and in this sense the market slowdown is really a kind of market sponsorship: a government acknowledging that it is protecting the industry and its key players and ratifying the state's support for the venture.

We can posit that the market slowdown is a means to protect consumers by preventing the industry from causing them harm, but this strikes me as a remarkably generous reading. Given the interests of the parties involved and the fiduciary benefits it offers them, it's smart to be sceptical; I reckon we should see the market slowdown not as a governmental slap on the corporate wrist but as a remarkably convenient support package which can handily be framed as altruism by nearly everyone involved.

So what!

We need deep scepticism by default when tech giants make big proclamations, regardless of whether they're announcing Eden or Armageddon. In particular we should know that if their interests are opposed to ours, then it's not contrarian to assume that the legislation they're advocating for is opposed to us. Vice versa, any legislation that will take power out of the hands of oligarchs and put it in the hands of individuals or the state is going to make the oligarchs sweat, and this is the litmus test. If they're not kicking and screaming, it's probably not good enough!

Next time: Jev, and long-promised content

I'm really keen to talk about something called Jev, especially because it relates so heavily to what we were talking about last time. I'll hold on to it for now (because the 2,000 words we've already hit is longer than any of these emails should be) but to summarise:

In my last email I talked about how back in 2024 I worked out a hack solution to get a small LLM to make decisions and perform actions similar to how an AI agent does. Coincidentally, a couple of weeks ago a company called TypeSafe announced a model called Jev which is designed to do exactly this sort of task. Jev spent about a week dominating the tech news headlines and sparking arguments on Hacker News. There's a lot of bullshit doing the rounds about what it is or isn't and what it can or can't do and how revolutionary any of it is. Obviously I want to talk about that! I also really want to talk about the significance of AI models which are designed to be translucent, buried within code, making decisions on/for/on behalf of users. Hold tight!!

I also promised long ago that I would send you a summary of some of the cool things I came across at xCoAx this year, and I'm still planning to share that. I also love doing very quick media reviews of things that I've watched and read and listened to, and haven't given you any of those in a while. Next time! For now I'll just say you should probably watch Silo, maybe read Ian Bogost, and definitely listen to new Durutti Column.