Running on Empty

This is a weird moment. I have a hunch that we’ll look back later and tell ourselves stories in which it seemed obvious where things were going. So I wanted to lay down a little time capsule and register just how confusing everything felt at the time. This is a personal view from the trenches. 1. If you’re a normal person, you may feel that AI has been good at math for a long time.1 But for “real” math, this is roughly what it felt like: When Feeling 2024 LLMs are toys Summer 2025 LLMs are interesting toys Early Autumn 2025 LLMs can do useful stuff sometimes Mid Autumn 2025 …a lot of stuff Late Autumn 2025 …a lot Early 2026 holy god Even having been astonished by AI progress in the past, it was still astonishing to see AI move in a matter of months from a mildly-useful toy to something that many people now consider almost impossible to do without. 2. Personally, I’d say that current AI makes me around 50% more productive at “doing math”. That’s despite the fact that AI can now often do in a few minutes things that would have taken me hours to do myself. The main reason is that my concept of “doing math” involves various steps, like coming up with ideas for theorems, trying to prove them, writing things down, reorganizing everything to make it more clear, and so on. AI is good at some parts of this, but still pretty bad at others, so the overall speedup isn’t as high as you might expect. (Even if planes could go 10× faster, your door-to-door travel time from your apartment in London to your friend’s place in Rome wouldn’t decrease that much, because you still have to get to the airport and wait to board.) 3. But then, I’m still (sort of; more on this later) trying to follow my old “doing math” workflow, just automating certain parts of it. That can’t be right, can it? If we found a way to make batteries at 1% of the current cost, we wouldn’t just say, “cool, batteries are cheap now”. We’d find all sorts of new ways to use batteries that didn’t make sense before, and change the ways we generate power to account for all these new batteries. Probably our whole energy system would end up transformed. Analogously, AI being good at proofs should lead to a very different equilibrium. But what is that?2 4. It seems healthy, at the moment, to maintain a fluid view of what “doing math” means. But at bare minimum it surely involves understanding the math that “I” am “doing”. So long as that’s part of the workflow, it would seem to impose a limit on how quickly “I” could “do” math, no matter how good AI is. One obvious conclusion would be that my tiny organic brain should be removed from the workflow entirely. Maybe! But it’s interesting to speculate about how much faster things could go while still keeping a human brain in the loop. I think the answer is: A lot. 5. Contrary to what you may have heard, humans are not obsolete for math. Not yet. The most obvious reason is that AI is still bad at writing. People often complain that AI writing overuses various patterns or lacks a unique voice or whatever. I don’t disagree, but I always felt like there was a deeper issue that I couldn’t quite put my finger on. With math, this becomes very clear: The problem is it doesn’t make any goddamned sense. Really, I think it’s deeper than being bad “at writing”. It seems like AI is bad at thinking in certain ways that manifest as bad writing. 6. When novices do math, they often create “spaghetti-code math”: Imagine a long document with lots of equations, but if you point at any one equation, it’s hard to say what assumptions underpin it, or how it fits into the bigger picture. Any random idea that was mentioned 17 pages ago might be used again at any time with no warning. With experience, people learn to break complex arguments down into modular pieces with a good “API”, in the sense that the assumptions and conclusions for each part are relatively simple. Somehow, AI has developed these amazing technical skills, while still being bad at creating abstractions So the arguments it gives are long and tangled and bewildering. This is what makes AI math seem so alien. 7. Now I know what you’re thinking. Maybe AI isn’t bad at creating abstractions. It’s just bad at creating abstractions that work for you, human with tiny organic brain. From a distance, that seems plausible. And it might be true to some degree. I have high-bandwidth access to a human brain. It’s not crazy to imagine that this gives me an advantage in predicting how a human brain would react to a given presentation of ideas. But still, I claim that current AI is objectively bad at creating abstractions, regardless of if the consumer of those abstractions is human or artificial. Here’s why: If I ask AI to prove something hard, it will produce a proof that starts simple and gradually descends into a rampage of complexity. If I tell it to break that proof down into self-contained pieces, it will produce lemmas that each take a full page just to state. If I ask, “Why? Why have you done this to me?”, the AI will say, “This is as simple as it gets. It’s not that bad. What’s your problem?” But if I argue with the AI for a while, I can start to understand the basic contours of the argument. And I’ll ask, “Why don’t you first just try to prove blah using only assumptions blahblah, without worrying about blahblahblah? Wouldn’t that be easier?” Often, this doesn’t work, either because my idea was bad, or because the AI isn’t smart enough to do it. (Which one? Who knows!) But after a few attempts, I can usually find something that that works, and the AI can use it to re-write the proof in a somewhat simpler way. If I repeat this many times, I can gradually chip away at the complexity. But maybe that’s still just making the proof easier for humans to understand? No! For one, this usually makes the proof shorter, and sometimes dramatically shorter. More importantly, during this process, someone (maybe me, maybe the AI) will often realize that there’s a different proof strategy that gives a stronger result. Instead of proving x < 9.7, now we can prove x < 7.1. Thus, artisanal human abstractions are “objectively” useful. 8. Some people might claim that it’s already possible to automate some of what I’m doing with some multi-agent Rube Goldberg thing. I’m not sure, because my experience has been that investing lots of time into clever AGENTS.MD or custom skills or harnesses isn’t a great use of time. That stuff is somewhat helpful, but things are changing so fast that the knowledge has a short half-life. Also, the majority of information about these tricks seems to come from people who are excited about the technology itself. On the other hand, the automator told me to try typing, “ask independent agents for an adversarial review”, and I’ve found those seven words to be quite helpful. I wish I could find a source with boring best practices for people who are happy to stay away from the bleeding edge. 9. Another reason AI doesn’t improve my productivity more is that it is often the green light, the orgastic future that recedes before me. I’ll ask AI to do something, and it will almost seem to do it. So I’ll ask for changes, and it seems to help, but it’s still not quite right. Hours later, I’ve accomplished nothing and learned nothing. Eventually, I start over, either doing it myself or with AI on a very tight leash, and I’m done in 30 minutes. Each time this happens I feel foolish and tell myself not to do it again. But it’s very hard to judge in real-time. 10. Of course, I’m not claiming that AI is intrinsically bad at writing or thinking or abstraction-making. It’s gotten better at all of these and I expect it will continue to do so. All I’m saying is that as of Autumn 2026, humans still have a lot to contribute, and it’s not just better exposition. 11. Did AI actually get much better at math in Autumn 2025? Or perhaps, in the grand scheme of things, human math ability falls into a narrow range? 12. Some people are responding to the new reality by gleefully using AI to generate lots of semi-OK-ish papers and puking them up on arXiv. I understand the incentives that drive this. In some sense, it seems like the logical response to the new reality. (After inventing cars, we started taking a lot more low-value trips.) But I don’t think this is socially useful, and I tend to think everyone will soon agree that this is embarrassing. But… I’m not sure. Maybe it is socially useful? And I find myself feeling vaguely jealous of the people who are wired this way, who instead of feeling blinkered, see opportunity and plunder. 13. Want an optimistic pro-AI, pro-progress take? OK. If I take my old “doing math” workflow, AI makes it strictly better. A lot better. It’s like having a hyper-conscientious intern that is smarter than me in many ways, works at incredible speed, and knows approximately everything. Yes, AI sucks at writing and making abstractions. But if I’m disciplined and try to mostly do the math “myself” while asking the AI for help, I can explore ideas much faster than before, make far fewer mistakes, and produce something just as clear as before, while developing just as much understanding. I can ask, “What happens if you do a Kofuni expansion?”, and get an answer instantly. I can ask for illustrations or interactive visualizations and see them instantly. Arguably, we’re incredibly lucky to be alive at this moment, when these new tools have opened up this new territory to explore. 14. And isn’t that what I wanted? Back in 2024, I bemoaned that what AI most seemed to struggle with was getting the facts straight: Perhaps the old science fiction robots were a projection of our fantasies. I want my style and voice to be precious and uncopyable. I want an AI assistant that will tirelessly hunt down facts while I provide creativity and panache and magic. But that’s not how it is. So we should remember: In a scenario where AI is useful but humans retain some comparative advantage, we might not like where that comparative advantage lies. At least with math, and at least for the moment, I get to be Kirk, while the AI is Spock. (Even with other types of writing, my impression is that since 2024, AI has improved more along Spock dimensions than along Kirk dimensions.) So that’s nice. 15. In the last few months, it frequently happens that a collaborator will use AI to generate something, and then send it to me and ask me to check if it’s useful. I can see why this seems reasonable. (Really, I do.) But when it happens a lot, one can’t help but wonder what the point of the human collaborator is. 16. On the other hand—with creative writing, my advice is not to use AI to write, at all. Ask questions if you want, but write every damned word yourself.3 With math, I don’t feel like such a hard line makes sense. It seems that creating something good still requires a lot of human involvement, to the point of agonizing over notation, repeatedly re-organizing things, and heavily editing individual sentences. Still, I find that if people make that effort, they can create something just as good as they could by hand, while still being perhaps 50% AI written.4 So I don’t mind if people send me “centaur math”, provided they take responsibility for the contents. The problem is that when working with AI, it takes constant paranoid vigilance to avoid letting things slip beyond your understanding. I often find myself wanting to awkwardly ask people, “OK, you obviously used AI to make this… which is OK… but how much of this do you vouch for? If we have a meeting to discuss it, will you crumble the moment I start asking questions?” 17. When I’m in abstraction-creation mode, I feel like I’m doing something intelligent. When yell at AI about bad notation and writing, I feel like I’m providing taste. But I often find myself giving feedback that seems incredibly basic. Things like: This is a mess, clean it up. Or: Do you need that assumption? Or: If you can’t solve this problem, why don’t you create a simpler version and try to solve it first? On the one hand, I guess a lot of what good human managers do amounts to giving very basic feedback. On the other hand, it’s hard to imagine that I have much durable advantage in doing this kind of thing. So I wonder how much durable advantage I have in abstraction-making or taste, too. 18. A common reaction to AI getting better at proofs is that humans need to focus more on exposition. This has an obvious Bourdieu-esque criticism: Human painters used to strongly value photo-realism, but after cameras were invented they convinced everyone that, oops, photo-realism isn’t important, what matters is expression, which coincidentally cameras can do. Isn’t a shift towards exposition in math just a way for humans to move the goalposts and justify their own continued relevance? The only credible way I see to avoid that criticism seems to be to bite the bullet and say that, in fact, math should have already have focused more on exposition, even before AI. Personally, I strongly believe that this is true. 19. I think the biggest reason I’m not more productive is: What’s the point? In the past, if you came up with an elegant proof, you’d tell yourself, “I made a thing! Other people will read and use my thing! I am advancing human progress.” You can still tell yourself that now. But you have to wonder, will anyone actually read your proof? Or will they just ask another AI to explain it to them? Will your proof help their AI explain it better? For how long? If you just waited a year, would some other AI produce that same proof independently? Alternatively, if you’re a normal person, you might think that LLMs still struggle with basic arithmetic? ↩ AI progress is so fast, compared to society’s ability to adjust, that we may never actually see what “the current equilibrium” looks like. ↩ This essay, if you’re wondering, was created by writing every damned word myself and not asking AI anything. ↩ You could have a whole debate about what that precentage means. It’s not the number of words written by AI that really matters, in my opinion, so much as who is responsible for the overall structure of the ideas. ↩

I always liked stories about eccentric royals doing mad things. Queen Christina of Sweden reportedly had a miniature cannon made for firing tiny cannonballs at fleas in her bedroom. Louis XIV made a proto-reality-show of his daily routines, allowing an audience of courtiers the honor of watching him dress, bathe, and use the toilet. Peter the Great famously returned from […]

The (Re)insurance AI Landscape

to become the version of yourself needed in the next chapter