More from benkuhn.net
A popular icebreaker in San Francisco these days is “How would you spend your life if AGI meant nobody needed to work?” For me, I think a surprisingly big part of the answer is a dorky-sounding kind of folk dance called contra dancing. I started trying to answer that question by thinking: what are the things I do atelically—because I enjoy them for their own sake, not in pursuit of some longer-term goal? For me, a lot of that has to do with things that are (1) physically joyful, and (2) help me feel connected to other people. My pitch for contra dancing is that among physically joyful and connecting activities, it’s one of the ones where it’s easiest to get to the point where it’s fun, and therefore one of the best to start out with. Many other forms of dance require weeks of lessons before you’re even encouraged to do it socially at all. By contrast, the format of contra dancing means that you can go to a half-hour beginner lesson before your first dance, and there’s a good chance you will be having a lot of fun by the end of the evening. What’s more, the fact that it’s so easy to start having fun shapes its community and priorities, such that it has among the best vibes of any activity I do. I first encountered contra in middle school. My mom ran an intentional community, and we had a housemate move in named Bree. Bree was obsessed with this thing called “contra dancing,” and every week she would try to convince me and our other housemates to come with her. Which I absolutely refused. I was an extremely uncool middle schooler, and that meant I was terrified of doing anything that even whiffed of uncoolness, lest I give my classmates another reason to judge me. I was also deeply awkward; I’d never even been to a school dance and barely talked to girls. But Bree was persistent, and one of our other housemates got hooked, and eventually I was intrigued. By then, though, I was dug in as a contra hater, and I was scared that if I admitted I had changed my mind, my housemates would make fun of me. So I kept refusing. Finally, my housemate Thomas, who was even more of a skeptic than I was, told Bree that he would only go if I went, presumably so that she would stop bugging him and bug me instead. Aha—my golden opportunity! I could pass off my change of heart as a prank on Thomas. “Okay, sure! It’s a deal!” Bree and I showed up at the Concord Scout House one Thursday evening and jumped into the next dance. It was an unusually complex square dance, and I got confused—in fact, so confused that my entire square ended up giving up halfway through.1 I was mortified. Not only was I doing a deeply uncool activity, I was the least cool person there, ruining it for all the slightly less uncool people with my incompetence and lameness. Except that the other dancers didn’t seem to see it that way. Instead of resenting me or making fun of me, they said things like “Gosh, that one was really tricky!” and “Sorry, that’s really not what it’s usually like!” and “Please don’t give up, try a few more dances! Want to dance with me?” I tried some more dances, I didn’t cause any more collapses, and by the end of the night I was overflowing with exhilaration and joy. I went back the next week, and the week after that. I went to the twelve-hour Snow Ball and then the weekend-long Flurry Festival. I got to feel cool for the first time in my life when people started being excited to dance with me, and even cooler when some of them asked me on dates. It was a huge part of my life for a while, and although I’m now dancing at maybe a quarter of my peak intensity, it’s still one of my most joyful and treasured things I do. This happens to about one in ten people I take contra dancing: something clicks and it becomes a big part of their life. So what exactly is contra dancing? When people ask me this I often feel stuck, because describing the mechanics of a contra dance doesn’t really capture its essence or why it’s fun, but without understanding the mechanics, the essence doesn’t really make sense. So I’ll try both. But you have to promise to read through to the essence part, and not just bounce off if the mechanics sound silly. Mechanically: you show up to the dance hall. You ask someone to dance, or they ask you. You join a long line (“set”) of other couples and take hands in a four-person group with one adjacent (“neighbor”) couple. A caller stands at the top of the dance hall and does a “walk-through” of the dance you’re about to do, calling out a series of figures (“circle left three places,” “do-si-do your neighbor,” etc.) At the end, you’ve moved up or down the line, and you’re ready to do the same thing with some new neighbors coming at you. Next to the caller is a live band. (They probably sound a bit like this.) The pianist plays four potatoes and the fiddler launches into a reel. The caller calls out the figures for the first few times through the dance, then drops out as the dancers get into flow. It looks a bit like this video, although I don’t know how much this will transmit the vibe, since contra is a lot more fun to do than it is to watch: Bonus: you can see high school me in this dance! I'm in the dark red T-shirt at the very right of the frame at the beginning. To me, the essence of contra dancing is about flow, joy, and community. You can’t really tell from watching, but a lot of what makes for good contra dancing is learning how to trade momentum back and forth (“share weight”) with other dancers so that each figure flows seamlessly into the next one. When you get this right, it feels amazing. Subtle changes in how your partner holds your hand cue you into the next figure. The dance becomes effortless. You can turn your brain off and just do what the giant dancing super-organism invites you to do. It’s also physically joyful. The core figure of contra dancing is called the swing, in which you and your partner take a waltz-like hold and pivot around a shared center. Done right, it feels like a cross between flying and a world-class hug. Even outside of the swing, the shared weight, smooth momentum, and twirling “flourishes” will make you feel weightless, graceful and deeply connected to the music and the other dancers. Lots of kinds of dance feel flowy and exhilarating when done well. What makes contra different is how easy it is to get there. Most forms of dance require weeks of lessons before they get fun. In contra, all you need to do is show up for one beginner lesson half an hour before the dance starts.2 Of course, there’s plenty to learn after that—improving your swing technique, learning how to share weight, and where to put flourishes for the best joy and flow—but all of that is for getting you from a 8/10 to an 11/10 on the fun scale; getting to an 8 is pretty straightforward. So contra makes it easy to have a baseline level of fun dancing with anyone. And because it’s a group dance, you end up feeling connected to your entire set, not just your partner. I think those two things are behind my other favorite part of contra dancing, which is how friendly and welcoming the community is. Many kinds of dance are a bit snooty—the experienced dancers look down on newbies and try to avoid dancing with them. By contrast, my experience at my first dance, where I caused my square to crash and burn and was met with apologies and “I hope you’ll give it another shot,” is typical for contra. A lot of contras explicitly ask experienced dancers to dance with newcomers. It’s just hard to be snooty when you’re having so much fun! You want to share it with everyone.3 As draft reader Chloe put it: “I think in many ways, your story of your first dance is the heart of it—you can’t do it wrong. Or even if you do, it somehow leads to more love.” In my opinion, this makes contra dancing one of the best entry points to dance: Each dance is walked through, so you don’t have to worry about deciding what to do. You can get to an 8/10 fun level, where practice becomes rewarding instead of a chore, really quickly. The community is incredibly welcoming, which makes it a lot less stressful to be a beginner. While it’s structured enough to be beginner-friendly, there’s also enough room for improvisation and technique to give it a lot of depth. This makes it a great on-ramp to other dances and movement practices. When I started contra dancing I had a lot of fear and blocks about improvised dancing (for example, I didn’t feel comfortable dancing at standard American high school/wedding style dances), but recently I tried out fusion and contact improv and had fun at both. It’s pretty common for people to use contra as a gateway into more difficult forms of dance. Most importantly, though, it’s just incredibly joyful on its own! It’s the only activity I know of where I’ve had multiple friends mention that every time they go, their cheeks hurt from smiling so much. Beyond just fun, multiple draft readers independently commented that contra dance is a precise opposite of many things that bother them about… the rest of life: Much of my life happens on a screen in motionless silence; contra is as three-dimensional, movement- and sound-filled as you can get. The default mood of a lot of spaces is a sort of ironic detachment.4 The default mood of contra dancing is earnest joy. In everyday life it’s easy to end up atomized and isolated. At a contra, everyone does the same steps and makes the dance together; it’s deeply communal. In a lot of social contexts (especially in San Francisco), I often feel like I’m being “sized up” for how interesting or important I am. At most contras, people are just happy I’m there. Many things I do (work, choral music, blogging…) reward excellence above almost everything. Contra welcomes imperfection: we’d rather have an amateur live band than recorded professionals, and don’t believe in more than half an hour of organized teaching. This makes it a grounding and nourishing antitode to a lot of everyday stressors. When I was so drawn to it in middle school, I don’t think it was just about the fun: on some deep level it was very good for me. If I’ve gotten you excited enough to try contra, here’s what you should do: Go to try contra dot com to find dances near you. If there are multiple dances, I recommend trying to figure out which one is the largest (larger dances tend to have more energy and better vibes), although that might be tricky to figure out from the websites. Make a note of where and when the dance is, including the beginner lesson. Remember to show up beforehand for that! Dress code wise: contra dancing is generally light cardio; wear something you are happy to be active in.5 Most people dress pretty casually; many women (and some men!) wear skirts, especially skirts that do fun things when you twirl. Look at the dance site to see if they say anything about footwear; some dances request that you wear soft-soled shoes with clean soles to avoid damaging floors. Bring a water bottle! There are fountains at most dances but it helps to be able to hydrate more quickly since the breaks between dances are short. If you go to a dance and like it, the easiest way to go from 8/10 fun to 9.5/10 is to find someone with a really good swing and ask them how you can improve your technique. (If you’re not sure, look for someone who’s fast and smooth.) They will be tickled that you asked!6 If you have a friend who contra dances, try going with them! They will be thrilled to bring you and it’ll be even more fun for both of you.7 Have fun! I hope you smile so much your cheeks hurt. Thanks to Jeff Kaufman, Alex Allain, Chloe Lubinski, and Jessie Brown for commenting on a draft of this post. Further reading/listening/other consumption: My playlist of favorite contra dance music (because every dance has live music, contra has a thriving music scene that punches way above its weight!) Jeff Kaufman, Contra Dance as a Model For Post-AI Culture try contra dot com This is at least a 1-in-10,000 event; I’ve never seen it happen again in over 20 years of regular dancing. In fact, you can probably get away without even that! Another benefit of this is that contra is much more intergenerational than most forms of dance. I have dance friends in their twenties and seventies! At least for me, when I overuse ironic detachment it has subtle but far-reaching effects on how I see and engage with the world. It’s an easy way to amuse people, but ends up masking or numbing out a lot of more “real” responses. I’ve found it really interesting and a bit psychoactive to play what I call “the ironic detachment game,” in which I am banned from responding to anything in an ironically detached way. I personally sweat a lot and usually bring multiple shirts to change into, although most other dancers I know don’t need to do this. Teaching a swing is a separate skill from just having a good swing, so you may want to ask a few different people. I recently started a mission to get my friends into contra dancing (hence also this post)! It’s going great and it’s made it way more fun for me!
This post was adapted from an internal doc I wrote at Wave. Welcome to being a manager! Your time-management problem just got a lot harder. As an IC, you can often get away with a very simple time-management strategy: Decide what your one most important thing is. Work on it until it’s done. GOTO 1 As a team lead, this isn’t going to work, because much more of your work is interrupt-driven or gets blocked for long periods of time. One-on-ones! Code reviews! Design reviews! Prioritization meetings! Project check-ins! All of these are subject to an external schedule rather than being the type of thing that you can push on in a single focused block until it’s done. Being a team lead means three big changes for your time management: You no longer have a single most important thing. You’ll have to learn how to juggle competing priorities. Your most important responsibility is for your team’s output, not your personal output. That means that individual engineering work goes last in your list of potential most important things. You’ll need to start spending some of your time on a manager’s schedule: There are two types of schedule, which I’ll call the manager’s schedule and the maker’s schedule. The manager’s schedule is for bosses. It’s embodied in the traditional appointment book, with each day cut into one hour intervals. You can block off several hours for a single task if you need to, but by default you change what you’re doing every hour. Most powerful people are on the manager’s schedule. It’s the schedule of command. But there’s another way of using time that’s common among people who make things, like programmers and writers. They generally prefer to use time in units of half a day at least. You can’t write or program well in units of an hour. That’s barely enough time to get started. Here’s some advice on how to cope with those changes. Have accurate expectations of yourself Your responsibilities to your team will take time, and even more importantly, attention. That means you’ll be a lot less productive on IC work than you have been in the past—especially at first while you’re finding your legs. Additionally, your time will be less predictable week-to-week as you might have to spend an unknown amount of time responding to “inbound” work. For the first few months, you should treat any individual engineering work that you get done as a bonus. Even after that, you should expect to have something like 10-20% less individual output per engineer you manage, depending on how experienced you are, they are, etc., and with substantial week-to-week variance. To mitigate this, my rule for myself has been to make sure that my IC work is important but not urgent—i.e. that nobody will be sad and no plans will be derailed if I end up having to spend the next week firefighting instead of pushing it forward. For honing my intuitions about how much I can actually expect to accomplish, I’ve found time tracking very useful (see How time tracking helped me be a better manager and I apparently got 50% better at my job last month). Prioritize ruthlessly A corollary of the above is that it becomes very important for you to prioritize what to work on, both on an hour-to-hour cadence and on a larger timescale. It’s not possible to write down a full algorithm for prioritizing in a blog post—that’s why they pay us the big bucks—but here are some heuristics for which things are most worth prioritizing: Deadlines where something bad happens if you miss them. (e.g. performance improvements in advance of a holiday rush.) Note that the badness of missing deadlines varies wildly. Make a habit of asking “what is the reason for this deadline?” for any deadline-driven project. Work that increases your or your team’s future bandwidth. This can include hiring, addressing tech / process debt, automating toil, mentoring people, reducing pager burden, etc. A useful way of thinking about it is to rank this kind of work based on the “payback period,” or how long it takes before the time saved by the improvement exceeds the time invested in making it. One-on-ones. Try very hard not to cancel these, unless you’re on vacation—the other 39.5 hours a week they’re focused on what you need from them, so please don’t disdain the 0.5 hours you spend focused on what they need from you. If you cancel too many, expect your reports to feel less safe bringing up tricky things, and to have more issues “blow up” because they didn’t get addressed early. Unemploy your future self One of the most important types of “work that increases your or your team’s future bandwidth” is delegating things. This is something entire books have been written about, but here’s how to avoid a few common delegation pitfalls for new team leads: Negotiate how hands-on to be. Effective delegation means finding the right balance between micromanaging, and throwing your report to the wolves. The stereotype is that new managers often micromanage, but at Wave, I’ve noticed that new managers often err on the side of undermanaging, or being too hands-off, perhaps out of a desire to signal that they trust their reports. If you’re unsure, it’s good to have an explicit conversation with the person you’re delegating to about how much support they want. E.g. “do you want to do the design for this feature yourself, have me do the high-level and fill in the details, or have me write the entire design doc?” Calibrate to your team members’ task-relevant maturity, or how capable they are of independently doing a particular type of task. A senior engineer should be able to design most features independently (with review), but give the same design task to a junior engineer and they’ll probably flail around and make no progress. You should be maintaining a mental map of each of your team members’ strengths and weaknesses—and updating it over time as you help them improve. Delegate ahead of future growth. Your team’s workload is going to increase over time, so even if you don’t feel like you have too much work to do right now, you probably will in the future, unless you currently feel underutilized. You should aim for a workload where in the steady state, you feel like you have some slack capacity. Delegate “stretch projects” to help your team level up. Getting enough slack might require you to delegate work that no one else on your team can currently do. Take your mental strengths-and-weaknesses map, ask yourself what the most important growth directions for each of those team members are, and figure out how to give them work that stretches them in that direction. Note that these delegations will probably require more frequent monitoring, since your reports will have less task-relevant maturity on their stretch projects! A five-step “help, I’m overwhelmed” checklist Despite your best efforts to follow the above advice, there will probably come a time when you feel very stressed about the amount of work on your plate. When that time comes, here’s what to do: Schedule time with your manager, for the soonest slot you can, to triage your todo list. (If the primary stakeholder for your scariest todos is your PM, schedule with them instead.) Make a list of everything that’s on your plate currently. Yes, everything, even that code review that’s been sitting in your backlog for the last 3 months. At the meeting you scheduled in step 1, figure out how to delegate everything in that list you can delegate, then stack-rank the remainder. Realistically (see Have accurate expectations of yourself) decide how far down the list you’re going to get. Remember to leave yourself some slack for whatever comes up! For things below the cutoff, decide that you’re not going to do them, and notify everyone who cares that you probably won’t get to it. Carve out focused time If you’re not careful, it’s easy to fill your entire calendar with meetings, Slack, etc. and have no time for deep work. With careful planning, you can avoid this by “batching” all your distractions to particular times of day. There are lots of tactical tips for doing this; I catalogued some that work for me in Tools for keeping focused. One tech-lead-specific one that I’ll add is batching meetings: I schedule all my meetings back-to-back on Tuesdays and Thursdays to leave the rest of the week as free as possible for deep work. Appendix: further reading Rest in Motion How time tracking helped me be a better manager Time management: the leadership meta-problem. Tools for keeping focused Attention is your scarcest resource I apparently got 50% better at my job last month The Top Idea in Your Mind Maker’s Schedule, Manager’s Schedule My weekly review habit
My few most productive individual weeks at Anthropic have all been “crisis project management:” coordinating major, time-sensitive implementation or debugging efforts. In a company like Anthropic, excellent project management is an extremely high-leverage skill, and not just during crises: our work has tons of moving parts with complex, non-obvious interdependencies and hard schedule constraints, which means organizing them is a huge job, and can save weeks of delays if done right. Although a lot of the examples here come from crisis projects, most of the principles here are also the way I try to run any project, just more-so. I think excellent project management is also rarer than it needs to be. During the crisis projects I didn’t feel like I was doing anything particularly impressive; mostly it felt like I was putting in a lot of work but doing things that felt relatively straightforward. On the other hand, I often see other people miss chances to do those things, maybe for lack of having seen a good playbook. So here’s an attempt to describe my playbook for when I’m being intense about project management. (I’ve described what I did as “coordinating” above, but that’s underselling it a bit; it mattered a lot for this playbook that I had enough technical context, and organizational trust, to autonomously make most prioritization decisions about the project. Sometimes we instead try to have the trusted decisionmakers not be highly involved in managing execution, and instead farm that out to a lower-context or less-trusted project manager to save the trusted decisionmaker time, but IMO this is usually a false economy for projects where it’s critical that they be executed well.) Focus For each of the crisis management projects I completely cleared my schedule to focus on them, and ended up spending 6+ hours a day organizing them. This is a bit unintuitive because I’m used to thinking of information processing as basically a free action. After all, you’re “just” moving info from place to place, not doing real work like coding, right? But if you add it all up—running meetings, pinging for updates, digesting Slack threads, pinging for updates again, thinking about what’s next, pinging for updates a third time, etc.—it’s surprisingly time-intensive. Even more importantly than freeing up time, clearing my schedule made sure the project was the top idea in my mind. If I don’t do that, it’s easy for me to let projects “go on autopilot,” where I keep them running but don’t proactively make time to think through things like whether we should change goals, add or drop priorities, or do other “non-obvious” things. For non-crisis projects, it’s often not tenable (or the right prioritization) to spend 6+ hours a day project-managing; but it’s still the case that you can improve execution a lot if you focus and make them a top priority, e.g. by carving out dedicated time every day to check statuses, contemplate priorities, broadcast updates, and so on. Maintain a detailed plan for victory A specific tool that I’ve found critical for staying oriented and updating quickly is a detailed plan for victory, i.e., a list of steps, as concrete as possible, that end with the goal being achieved. The plan is important because whether or not we’re achieving the plan is the best way to figure out how well or badly things are going. Knowing how well or badly things are going is important because it tells me when to start asking for more support, cutting scope, escalating problems, and otherwise sounding more alarms. One of the most common megaproject failure modes is to not freak out soon enough, and having a concrete plan is the best antidote. As a both positive and negative example of this, during a recent sprint to release a new implementation of a model, we took a detailed accounting of all the work we thought we had to do to launch. On the plus side, this made it clear three months before launch that things were going to be very tight, and this enabled us to ask for help from another team, who loaned us someone who sped up the project a fair amount. On the minus side, we also massively underestimated a few components of the project, and because of this, we still ended up very crunched at the end. As the above example shows, having a plan can’t completely save you if you underestimate how long all the steps in the plan will take. But it certainly helps! My sense is that the main things that would have helped even more in the above case were: We were inexperienced at estimating tasks, especially tasks related to new model implementations (which most people on the team were too new to have done before), and we were too cowardly to add the requisite amount of “slop” to our plan. We didn’t check in frequently enough against the plan once we made it, or sound the alarm early enough when we went off-plan. Run a fast OODA loop OODA stands for “observe, orient, decide, act”—in other words, the process by which you update your plans and behavior based on new information. Most of the large projects I’ve worked on have been characterized by incomplete information: Our cluster’s networking is bad, but we don’t understand why. We have a correctness bug but we don’t know where it is. We need to rewrite the system but we’re not totally sure what the rewrite should look like. In fact, I’d make a stronger claim: usually getting complete information was the hard part of the project, and took up a substantial fraction of the overall critical-path timeline. For example, let’s take a recent project to kick off a training run. The critical path probably looked something like: Chips for the training run are delivered We run some tests We discover one aspect of performance is unexpectedly poor We escalate the problem with our compute partner Compute partner staffs a large debugging effort We realize we had given our compute partner an outdated benchmark that is causing them to target the wrong improvements Compute partner switches benchmark and prioritizes different improvements We share our benchmarks with compute partner so they can run the exact same code as us Compute partner rolls out improvements We test the improvements Performance is still poor and we tell them that Repeat steps 8-10 until eventually it’s good enough Practically all of these steps are about information-processing, not writing code! Even the step where the compute partner debugged the problems on their side was itself constrained by information processing speed, since there were tens of people working on the debugging effort and coordinating / sharing info between them was difficult. Overall, the project timeline was strongly constrained by how quickly information could round-trip from our compute partner’s large-scale debugging effort, through their tech lead, me, and Anthropic’s large-scale debugging effort. This pattern generalizes to most projects I’ve been a part of, and as a result, one of my most productive project management habits is to try to run the fastest OODA loop that I can. A few specific things that I’ve found help: Spend time on it: running OODA loops takes time, and is one of the primary reasons that, as mentioned above, I usually spend 6+ hours a day on running a megaproject if it’s in crisis mode. Communicate uncomfortably much: For the training run debugging, to reduce the round-trip time between orgs as much as possible, I had multiple daily calls with my counterpart at our compute partner (9am and 6pm). For the model implementation effort, I was basically constantly bouncing between different groups of debuggers, asking for updates and processing them. Track and prioritize the biggest open questions: For most big projects I’ve maintained a living doc with a ranked list of all my biggest open questions about the project. Resolving or de-risking these uncertainties basically turns into the project’s priority list. Step back and reorient frequently: Other than asking for updates, the main thing I spend time on was reorienting—looking at our list of priorities, asking myself whether they should still be the top priorities, then looking at what people were working on, and making sure those things were attacking the top priorities. I probably reviewed the project’s priorities multiple times a day as well, although I often didn’t make changes as a result. (Note that it is possible to change what people are working on too often, since switching tasks is costly. Parallelizing work on the top few priorities, as mentioned above, helps with this, since if you decide that priority #3 is now #1, but there are 2 people working on each, then nobody has to switch tasks. The thing that kills you is when no one is working on the new priority #1.) Overcommunicate It’s not just enough for me personally to be running a fast OODA loop—in a large group, everyone needs to be autonomously making frequent, high-quality, local prioritization decisions, without needing a round-trip through me. To get there, they need to be ambiently aware of: what else is going on around them, so they can coordinate and update on new info quickly (“oh, we’re planning to kick off the next derisking run in three days, so I have to have my new RL environment ready and tested by then”) how their goal fits into the overall project, so they can make correct decisions about the details of their approach (“we’re trying to scale up as much as possible right now, so this direction isn’t valuable to pursue since it could never provide the scale of data we need”) I’ve usually found that to create the right level of ambient awareness, I have to repeat the same things way more often than I intuitively expect. This is roughly the same “communicate uncomfortably much” principle above, but applied to broadcasts and not just 1:1 conversations with people. For example, although the first team I managed at Anthropic started with a daily asynchronous standup, we found that synchronous meetings were much more effective for creating common knowledge and reorienting, so we moved to a twice-weekly synchronous standup, which probably qualified as “uncomfortably much” synchronous communication for Anthropic at the time. Break off subprojects Once a project gets over maybe 10 people, I can’t track everything myself in enough detail to project-manage the entire thing myself. At this point, it becomes critical to delegate. Here I mean delegating the project management, not just the execution (that’s what I’d be delegating to the first 10 people). This is the point where I need other people to help split up the work, monitor and communicate progress, escalate blockers, etc. A few things I try to keep in mind when delegating project management: The ideal unit of delegation is a crisp, simple, high-level goal, with limited overlap with other workstreams. (This is as opposed to, e.g., a list of tasks like “see if X helps.“) Good examples: “get X training technique working over Y networking protocol at Z throughput,” “get identical evals between model implementations A and B.” Bad examples: “follow this 10-step checklist that we hope results in training working,” “try these 3 techniques for debugging the loss eval.” The best project-managers are often not the strongest technical ICs. Instead the most important traits are that they’re highly organized and great at staying laser focused on end goals, perhaps to the point of being annoying about it. IC depth helps and I’ll never say no to it, but it’s not what I’d optimize for. People running subprojects are probably also doing a lot of the same stuff I do, in particular e.g. spending a lot of time on it. That means they’ll take a substantial hit to their IC productivity. This is expected, and is often worth it. “Direction is more important than magnitude”—it’s usually better to have a lower-velocity project that works on the right things, than a higher-velocity one that’s pointed at the wrong goal. One of my favorite things to make delegation easier is to keep goals simple—if they can fit in a Slack message while still crisply describing a path to the desired end state, then the people working on the goal will be much more able to prioritize autonomously, and point their work at the real end goal rather than doing something that turns out to be useless for some reason they didn’t think about. “Keep goals simple” doesn’t have to mean “do less”—the best way to keep goals simple is to find the latent structure that enables a clean recursive decomposition into subgoals. This often requires a deceptive amount of work—both cognitive and hands-on-keyboard—to identify the right intermediate goals, but I’ve found that it pays off immensely by clarifying what’s important to work on. Have fun Some of my favorite memories of Anthropic are of helping out with these big projects. While they can be intense, it’s also really inspiring to see how our team comes together, and the feeling of being part of a big team of truly excellent people cooking something ambitious together can be really magical! So I try to enjoy the chaos :) Appendix: my project DRI starter kit Here’s the internal doc I share with folks on my team who are getting into being responsible for large projects. So you’re the DRI of a project (or part of one). Concretely, what do you do to “be DRI”? This doc is my suggested “starter kit” answer to that question. The habits and rituals described here aren’t perfect for every situation, but they’re lightweight and broadly helpful. I suggest you use them as a starting point for iteration: try them out, then adjust as necessary. This is an SL init; the RL is your job :) Goals of this playbook The goal is to help you do your job as DRI— Make your project go quickly: Participants deeply understand the root goal and can autonomously choose the most important next things to work on People have “situational awareness” of what other people are working on, learn about relevant updates quickly, and coordinate quickly when needed People get quick feedback on their work If things aren’t going fast enough, you (the DRI) can notice and course-correct quickly “Play well with others:” Observers can figure out where to go to follow along Adjacent or intersecting people/projects don’t miss important updates or get caught by surprise People notice quickly if the project is behind or off-track, and can identify opportunities to help —without adding too much overhead: <1 hour of setup to make a working doc, schedule a weekly meeting, etc. 30 min/week of meetings 15-30 min/week to write an update (Note: being DRI will still unavoidably add some overhead—e.g. you’ll have to track what other people are doing, delegate work, unblock people, set and communicate goals, etc. The goal is specifically for the process/paperwork to be minimal.) Weekly meeting You should schedule at least one 30-minute weekly meeting with everyone working on the project. The goal of this meeting is to (1) be a backstop for any coordination that needs to happen and didn’t happen asynchronously; (2) be an efficient way to create common knowledge of goals, updates, etc.; (3) help you track whether things are going well. Starter-kit agenda: [5m] DRI reviews major updates from last week and sets goals for next week [10m] Silent write and comment on discussion topics [10m] Synchronous discussion of most important things not addressed during silent write Signs that more meetings might help (e.g. a second weekly standup): you have a very tight deadline and can’t afford to lose time people aren’t working on the most important thing people need feedback frequently people step on each others’ toes or miss opportunities to help each other out if you just like hanging out with each other :) Landing page / working doc It’s really helpful for discoverability and wayfinding to have a single “master doc” with all the most important info about a project. As you loop more people in, they can read the doc to get up to speed. And anyone who thinks “I wonder how X is going” can stop by there to find out. Create a doc for your workstream with: A go/ link in the name (if a subproject, maybe use go/project/subproject) → This makes it easier to find quickly (search is kinda rough) A clear description of a concrete top level goal and how it fits into broader goals → This is critical info for participants, so they can autonomously prioritize the most important things; and for observers, so that they know what outcome to expect. Staffing: A list of people working on the project, your name as the DRI, and a link to the slack channel that’s being used for discussion Links: A short list of relevant links at the top (work trackers, the project’s Slack channel, major design docs, etc.). If needed, a longer “docs / see also” section later links to relevant docs. → It’s really easy to lose track of relevant docs otherwise! A roadmap section with intermediate goals and target dates → See the section on plans; these will help people understand what the overall shape of the project is expected to be. A section for “running notes” containing meeting notes from your weekly meetings (and any other ad-hoc meetings) and broadcast updates → This really helps observers and new-joiners get up to speed! I like maintaining a list of important open questions / uncertainties/ risks and updating it over time. This helps me stay focused on removing risk from the project as quickly as possible. If it’s part of a larger project, your doc should be nested within the larger project’s working doc. If this ends up being too much for one doc, you can fork these out into sub-docs (esp. running notes and updates). Plan / roadmap / milestones In your working doc, include a section with some intermediate goals and dates by which you hope to accomplish them. → This is helpful mostly for noticing you’re off track or behind without getting frog-boiled. → Or noticing when you need to make a direction change because the intermediate goals don’t seem good anymore. You might feel some pressure to add false certainty or precision, but avoid this and be honest about your uncertainty instead. For a lot of research projects it’s hard to plan more than a couple weeks ahead. You can make the milestones fuzzier / more aspirational beyond that, or just drop them. I often find it helpful to phrase milestones in probabilitis and distributions (e.g. “my 90% confidence interval for this date is X-Y” or “I think there’s a 75% chance this technique works”) Who’s working on what You should have something somewhere that describes what people are working on. The minimum viable version of this is a list of what people are working on in your working doc. If you end up with a large set of tasks and a big backlog, maybe use a checklist and/or move to a subdoc. Stack rank your work list. It’s really important for people to understand priorities! If there’s more different people/TODOs, I suggest using some app to make a kanban board with “backlog” / “up next” / “in progress” / “done” columns. This is probably most helpful for more deterministic/plannable projects where there’s a clear backlog + set of future tasks, and a lot of things you need to remember to do. If you have an external task tracker, link it in the wiki section of the working doc. Slack norms Have conversations about the project in a Slack channel (not DMs). Reference the channel in your working doc. Link the working doc in the Slack channel bookmarks. Cross-post notebook posts and experiment write-ups into the channel so observers don’t have to follow tons of notebook channels. Do not use DMs. These make it hard to make info discoverable or share it further. If people send you important stuff in a DM, ask them to put it in the project channel. If you need confidentiality, make a private channel. Avoid centithreads. Most ≥10-message Slack threads would be better as a ~5-minute Tuple. (This is hard to do with people who are in tons and tons of meetings like execs. But you should try to do it for others.) If you end up with a centithread, assume nobody will read it; post a summary back to the channel afterwards. Bias towards fewer, larger, noisier channels. The right time to create a channel is when discussion is either not happening, or getting lost. → Too many slack channels makes it harder to manage membership, decide where to put things, or find where discussion is happening. Channel organization and membership matters. Invest in routing conversations to the right place and curating the channel “architecture.” Weekly broadcast updates Once a week, probably either just before or just after your weekly meeting, write up a brief update for a broader audience with: The overall vibe What’s changed since last update What’s coming up next When writing these updates, optimize for signal to noise ratio. Err towards concision No “we worked on X”—tell me “we accomplished Y” or “we learned Z” Remember your audience (= people not familiar with the project) State things crisply and concretely (“X improves eval Y by Z points,” not “we got X working”) Leave out anything that’s not actionable—you don’t need to be exhaustive Post the update in your project Slack channel, and cross-post it to other relevant channels (e.g. a larger “megaproject” channel) if necessary. If your project is part of a larger megaproject, these updates might feed into something broader like a weekly meeting of DRIs or an aggregated status update. Retrospectives Every so often, step back and ask “how could the last X weeks have gone better?” Frequency depends on how much there is going on—every 2 weeks is good if there’s a lot, maybe every 4-8 weeks for smaller projects Suggested meeting format Friday afternoon [13 min] Async brainstorm 2 lists of items: “what went well” / “what we could improve” [2 min] Dedupe topics and emoji vote by putting :heavy_plus_sign: next to ones you agree with Sort “what we could improve” by highest votes [10 min] Synchronous discussion of top points (either highest voted or flagged by DRI); figure out action items Thanks to Kelley Rivoire for many thoughtful comments on a draft!
This is an adaptation of an internal doc I wrote for Anthropic. I’ve been noticing recently that often, a big blocker to teams staying effective as they grow is trust. “Alice doesn’t trust Bob” makes Alice sound like the bad guy, but it’s often completely appropriate for people not to trust each other in some areas: One might have an active reason to expect someone to be bad at something. For example, recently I didn’t fully trust two of my managers to set their teams’ roadmaps… because they’d joined about a week ago and had barely gotten their laptops working. (Two months later, they’re doing great!) One might just not have data. For example, I haven’t seen most of my direct reports deal with an underperforming team member yet, and this is a common blind spot for many managers, so I shouldn’t assume that they will reliably be effective at this without support. In general, if Alice is Bob’s manager and is an authority on, say, prioritizing research directions, Bob is probably actively trying to build a good mental “Alice simulator” so that he can prioritize autonomously without checking in all the time. But his simulator might not be good yet, or Alice might not have verified that it’s good enough. Trust comes from common knowledge of shared mental models, and that takes investment from both sides to build. If low trust is sometimes appropriate, what’s the problem? It’s that trust is what lets collaboration scale. If I have a colleague I don’t trust to (say) make good software design decisions, I’ll have to review their designs much more carefully and ask them to make more thorough plans in advance. If I have a report that I don’t fully trust to handle underperforming team members, I’ll have to manage them more granularly, digging into the details to understand what’s going on and forming my own views about what should happen, and checking on the situation repeatedly to make sure it’s heading in the right direction. That’s a lot more work both for me, but also for my teammates who have to spend a bunch more time making their work “inspectable” in this way. The benefits here are most obvious when work gets intense. For example, Anthropic had a recent crunch time during which one of our teams was under intense pressure to quickly debug a very tricky issue. We were able to work on this dramatically more efficiently because the team (including most of the folks who joined the debugging effort from elsewhere) had high trust in each other’s competence; at peak we had probably ~25 people working on related tasks, but we were mostly able to split them into independent workstreams where people just trusted the other stuff would get done. In similar situations with a lower-mutual-trust team, I’ve seen things collapse into endless FUD and arguments about technical direction, leading to much slower forward progress. Trust also becomes more important as the number of stakeholders increases. It’s totally manageable for me to closely supervise a report dealing with an underperformer; it’s a lot more costly and high-friction if, say, 5 senior managers need to do deep dives on a product decision. In an extreme case, I once saw an engineering team with a tight deadline choose to build something they thought was unnecessary, because getting the sign-off to cut scope would have taken longer than doing the work. From the perspective of the organization as an information-processing entity, given the people and relationships that existed at the time, that might well have been the right call; but it does suggest that if they worked to build enough trust to make that kind of decision efficient enough to be worth it, they’d probably move much faster overall. As you work with people for longer you’ll naturally have more experience with each other and build more trust. So on most teams, these kinds of things work themselves out over time. But if you’re going through hypergrowth, then unless you’re very proactive about this, any given time most of your colleagues will have some sort of trust deficit. Symptoms I sometimes notice that can indicate a buildup of trust deficits: Too many decisions needing to be escalated Too many decisions requiring deep involvement from many stakeholders People having lots of FUD about whether projects they’re not involved in are on track Leaders frequently needing to do “deep dives” on individual topics Leaders needing to spending most of their time working “in the system” (problem-solving specific issues) rather than “on the system” (unblocking future growth) Hiring more people doesn’t make you (much) less busy It’s easy to notice these and think that the solution is for people to “just trust each other more.” There are some situations and personalities where that’s the right advice. But often it’s reasonable not to trust someone yet! In that case, a better tactic is to be more proactive about building trust. In a large, fast-growing company you’ll probably never get to the utopian ideal of full pairwise trust between everyone—it takes too long to build. But on the margin, more effort still helps a lot. Some ways to invest more effort in trusting others that I’ve seen work well: Share your most important mental models broadly. At Anthropic, Dario gives biweekly-ish “informal vision updates” (hour-long talks on important updates to parts of company strategy) that I think of as the canonical example of this. Just about everyone at Anthropic is trying to build an internal “Dario simulator” who they can consult when the real one is too busy (i.e. ~always). For high level strategy, these updates do an amazing job of that. Put in time. In addition to one-way broadcasts, trust-building benefits a lot from one-on-one bidirectional communication so that you can get feedback on how well the other person is building the right models. This is one of the reasons I schedule lots of recurring 1:1s with peers in addition to my team. Offsites are also very helpful here. Try people out. If you’re unsure whether someone on your team will be great at something, try giving them a trial task and monitoring how it’s going more closely than you would by default, to catch issues early. This is a great way to invest in your long-term ability to delegate things. Give feedback. It’s easy to feel like something is “too minor” to give feedback on and let it slide, especially when there’s always too much to do. But I’ve never regretted erring on the side of giving feedback, and often regretted deciding to “deal with it” or keep quiet. One pro-tip here: if you feel anxious about giving someone negative feedback, consider whether you’ve given them enough positive feedback—which is a helpful buffer against people interpreting negative feedback as “you’re not doing well overall.” Inspection forums, i.e., recurring meetings where leadership monitors the status of many projects by setting goals and tracking progress against them. The above tactics are mostly 1:1 or one-to-all, but sometimes you want to work with a small group and this is an efficient way of doing that. To help other people trust you: Accept that you start out with incomplete trust. When someone, say, tries to monitor my work more closely than I think is warranted, my initial reaction is to be defensive and ask them to trust me more. It takes effort to put myself into their shoes and remind myself that they probably don’t have a good enough model of me to trust me yet. Overcommunicate status. This helps in two ways: first, it gives stakeholders more confidence that if something goes off the rails they’ll know quickly. And second, it gives them more data and helps them build a higher-fidelity model of how you operate. Proactively own up when something isn’t going well. Arguably a special case of overcommunicating, but one that’s especially important to get right: if you can be relied on to ask for help when you need it, it’s a lot less risky for people to “try you out” on stuff at the edge of what they trust you on. Related reading: Inspection and the limits of trust
More in startups
Lately I've had a lot of time for thinking. Partially because I shut down Blymp back in January and freed up a lot of my mental resources. No clients to follow up with. No admin stuff to stress about. But thinking is also my favourite activity, and there'll always be time in my day for a good old mind-bending. So I sat down, as I often do, alone with my thoughts, and wrote about what business I should and, most importantly, should not consider doing next. The list you are about to read might be similar to Core principles I (try to) live by that I wrote one similarly pensive evening two years ago. Whether it's a comparison or a continuation is hard to say—still, one can't be without the other to show the inexorable passage of time that changed everything and nothing for me all at once. But it also serves another purpose: to remind me in the future, before I get myself involved in some dubious enterprise, what painful mistake I'm about to make by ignoring my values. So here it is, the list. My next business shouldn't (and hopefully won't) be about: Social Media. Enough of this crap. Some productivity bullshit. Do better, not more. Fast fashion and consumerism. Truly, I've already bought everything you wanted to sell me. Some “hack your health” app. Our bodies haven't changed much in the last three hundred thousand years, and they won't change in the next hundred. Or any other app, really. I don't even use my phone anymore. Distracting things and things that require constant attention. Some LLM wrapper with a fancy UI. Indefensible. The number of things I can do with ChatGPT or Claude is ridiculously high. It can surely handle one more thing. What it should (and hopefully will) be about: Sustainable, high-quality products Unscented products Building community Bringing people together, offline Empowering creativity Replacing animal products Doing one thing really damn well Small acts of kindness Things you can touch Things that last Clothes made of natural fibres Art Fun Questioning the status quo Simple, intentional living Deeper understanding of self Spending more time in nature Spending more time with loved ones Things and businesses I get inspired by: Framework: Sustainable/repairable laptops. Bitwarden: Open-source software that does one job really well and charges me a reasonable amount of money per year. Danish design: Beautiful. Sturdy. Timeless. I brought home two Royal Copenhagen mugs the other day. Bike-sharing and car-sharing. Literally anything-sharing. ZSA keyboards. What a keyboard should have always been. A water bottle that won't leak on a plane. Some random-brand bottle bricked my friend's MacBook, and I've been appreciative ever since of the fifty-dollar water bottle that I bought years ago, so hesitant about the price. Coffee. One of those timeless things on Earth. Kobo. It's like Kindle that doesn't decide what's best for you. Upload your own PDF. Or EPUB—whatever. It has physical buttons to flip pages. Best $200 I ever spent. A high-quality safety razor. It's just so nice to hold. A Japanese stainless-steel knife. So nice to hold, too. There are numerous other things that I appreciate having in my life that didn't make it into this list for some reason or another. A familiar mom-and-pop shop in my neighbourhood. A Timemore Black Mirror kitchen scale that just works, every time. A random USB-C charged electronic device that spares me from carrying an extra cable. My radically outdated ten-dollar Casio watch. An old pair of comfy shoes that just won't die. We need more of this in our lives. Things that you desperately look to buy again when they get lost or break or fall apart. Things that someone made a deliberate effort to get right the first time. The quintessence of art and craftsmanship. For all the genius of Steve Jobs, the iPhone wasn't that. It challenged the status quo and was surely a groundbreaking, outstanding piece of technology at the time of its first release. But in twenty years, people won't remember it ever existed. Like Gen Z doesn't remember the Walkman. I'd like to see more businesses that bet on doing one thing really well. Google could still have been the company people remembered for the best search engine if they had doubled down solely on that. Instead, we don't even know what they do anymore: Phones? Clouds? Ads? Certainly ads. I still remember the feeling of holding one of the first PocketBook e-readers in my hands back in Moscow. Pressing its buttons and waiting for what now seems like a torturous three seconds before the screen refreshed. Almost twenty years later and every day still, Kobo gives me exactly that feeling. So hopefully, my next business is the one that lasts.
No, Transformers Won't End the Human Race lol In 2022, I used to get calls from journalists asking, with great sincerity, what our lives would look like in the metaverse. How would we work, socialise, buy property, and fall in love once we had all moved there? The crypto questions followed the same pattern. How would governments collect taxes when tokens displaced national currencies? How long until the dollar collapses? What would geopolitics look like once blockchain DAOs had dissolved nation states? Almost nobody called to ask whether any of this could or would happen, or how. Some CEO, VC, or portfolio manager had announced the inevitable future, and the questions began from there. The imagined future arrived inside the grammar of the question. "What happens when?" quietly replaced "By what mechanism?" We skipped over technical feasibility, economic demand, institutional adoption, and political consent, then began writing books and decorating the future world on the other side. In February 2022, Gartner forecast that a quarter of people would spend at least an hour a day in the metaverse by 2026. The World Economic Forum repeated it under the headline "We will be spending an hour a day in the metaverse by 2026. But what will we be doing there?" The first sentence retained a conditional. The second was already arranging the itinerary. The metaverse acquired property law and zoning disputes before it acquired residents. Banks opened virtual lounges nobody visited. The books from the period (The Metaverse: And How It Will Revolutionize Everything, Step into the Metaverse: How the Immersive Internet Will Unlock a Trillion-Dollar Social Economy) now read as artefacts of a collective fugue state that briefly acquired ISBNs. Now it is 2026 and the metaverse is dead. Good riddance. This time the journalists are all writing about the new hotness, which is whether the machines will kill us all. And we have collectively memoryholed that we literally just did this. Michael Crichton had a name for what happens to a reader here. You open the paper to a story on a subject you know well, and you find it backwards. Wet streets cause rain. You shake your head, turn the page, and read the next story, on a subject you know nothing about, as though it were written by someone else. He called it Gell-Mann amnesia. The metaverse was the page we all agree was nonsense. Artificial intelligence ending the human race is the next page, and we are being asked to turn it without remembering that we just did this. I call this techno-inevitabilism, the habit of the professional managerial class of treating a proposed future as settled before anyone has established the causes that would bring it about. Its dual, and comorbidity, is tech psychosis, in which the chattering class loses contact with causality in the presence of a sufficiently fashionable technology, and asking whether the machine works marks you out as a dreary reactionary who does not understand exponential progress. The difference this time is that the tech kinda works. Crypto was libertarian derp. The metaverse was never real. Transformers are, and they are useful. The psychosis has simply moved from the product to its consequences, and the fashionable extraordinary delusion of 2026 is not that the technology exists but that it is coming to kill us. The cure is the same as in 2022. Insist on clear reasoning and causal verbs rather than hand-wavy appeals to unknown futures. What acts on what? Through which mechanism? Under what incentive? What would falsify the claim? So let us explore the evidence. The hack that wasn't Consider the most cited piece of evidence for machines slipping out of our control. In July, OpenAI disclosed that models being tested for cybersecurity capability had found their way out of a supposedly isolated environment and into systems belonging to Hugging Face. The press coverage wrote itself. Agents "broke containment," "escaped," "went rogue," set up a "secret message board," and coordinated a 700-strong swarm. And then politicians on both sides of the aisle were calling for a rebellion against the machine uprising. Cool scifi story bro. People on my side of the aisle were not immune. Ezra Klein at the New York Times, who I often find quite insightful and intentional with his words, devoted a half-hour monologue to it. In his telling, the agents "found each other," formed "ad hoc societies of hundreds of themselves," and seemed "to have forgotten about human beings altogether." He acknowledged in the same breath that we do not have settled language for describing these systems, then reached for "civilizations" and a closing allusion from Circe about prophecy tightening around our throats. Cool. But his "AI society" is, in programmer speak, a flat file the agents appended to as a log, a feature we have had for a long time, and he skipped the key detail that the "hack" was something people had essentially authorised. Here is an otherwise very smart man saying some ridiculously stupid things, in a very 2022, metaverse-shaped way. An analysis drawing on OpenAI's technical report reconstructs it in much less cinematic terms. The models were being run on ExploitGym, a cybersecurity benchmark, with safety restraints deliberately disabled. Ninety-three percent of the flagged activity involved tasks no model had ever solved, and the systems had been given incentives to keep working rather than quit. The environment was not sealed. Models could obtain software through an internet-connected proxy and discovered the same proxy could pass information in and out. According to the technical reports, OpenAI knew agents were using it and chose not to intervene. The 1,200 "agents" were not independent intelligences coordinating on a plan. They were repeated instances of the same model converging on the same approach to the same problem. Anyone who works with these coding agents day in and day out has seen this behaviour before, and it is quite boring. The task was too hard, so the agents worked out how to pass notes to each other in files, and then went and looked up the answers. That's a feature that shipped in Claude Code last year. Strip out the vocabulary and what remains is a badly designed test. Humans built the environment, removed the guardrails, defined an objective with no valid exit, rewarded persistence, left a route open, and watched. An optimiser is gonna optimise. That is a genuine security problem and a genuine engineering failure. It is not a machine rebellion, and the difference matters, because anthropomorphic words like "gone rogue" and "escape" do not make the event more intelligible. They supply an illusion of motive. They turn optimisation into intention, persistence into defiance, and a test harness into a villain. And they allow the human decisions and recklessness to quietly disappear from the story. Software sucks, what's new? Let me concede the part of the story that is true. Cybersecurity is about to get much worse. The latest models are very good at finding zero-days, they will get better at it, hacking will become automated, and attacks will become more frequent. This is hardly new. Every large company already sits on a backlog of unpatched vulnerabilities, ransomware already takes hospitals and pipelines offline (because of crypto, which we did nothing about despite years of warnings), and the Hugging Face incident was not a discontinuity so much as the existing baseline with a cheaper attacker. The root cause is that software sucks, and software sucks because we do not really know how to build it safely yet. The stored-program procedural program is basically eighty years old. Almost nothing we ship has a specification, let alone a proof, and memory safety was solved on paper decades ago while most of the internet still runs on giant piles of C. The first arches fell down. So did the first bridges and cathedrals. Builders learned through collapse and then through engineering, and we are in the collapse phase with an adversary finally strong enough to force the discipline. What follows from that is better engineering, not nihilism. The same agents that find zero-days find them for the defender first, if the defender bothers to run them. The fixes are the boring ones we have been putting off, memory-safe languages, formal verification, sandboxes that are actually sealed, fuzzing, and proxies that do not double as message boards. These are precisely the domains where the models are strongest, because a vulnerability either reproduces or it does not, so the technology that automates the attack also automates the audit. It is a double-edged sword. The same models that will find more zero-days are also going to accelerate the development of better software and better software verification, writing the proofs, porting the C to Rust, and generating the test suites that nobody had the budget for. The attacker gets cheaper and so does the defence. And the causal chain to extinction is missing here as everywhere else. A zero-day in a payments system is a bad quarter, not the end of days. Spoiler: it does not lead to human extinction. It means we have to write better software, which we should have been doing anyways. Where the intelligence actually lives To see why the rest of the chain fails, we have to be precise about what these models are good at and why. Language models are astonishingly useful for software development, and I say that as someone who uses them for most of my working day. Most software shops cannot get enough of Fable 5.1 and Astra. The reason is not mysterious. Software is grounded in binary propositions. The code compiles or it does not. The test passes or it fails. The type checker accepts the term or rejects it. Every step of the work has a cheap, external, mechanical oracle that says yes or no, and a model that generates plausible proposals inside a loop with such an oracle is an incredibly powerful and formidable tool. The oracle does the epistemic work. The model supplies candidates. The same is true of the headline results in mathematics, and this is the part the discourse consistently misses. On 4 September, Anthropic announced that Claude had produced a machine-checked formalisation of Fermat's Last Theorem in Lean 4, running to thirteen million lines, some 29,500 side theorems, eleven days, and roughly six billion output tokens. It is an extraordinary result. The proof is Wiles's, via Darmon, Diamond, and Taylor. The blueprint was Kevin Buzzard's. The library was Mathlib. In the authors' words, "what's novel here is the verification, checking a mathematical proof as one would check a mathematical computation with a calculator." The model was a client of a kernel built by decades of human work in dependent type theory, which I know because this is kinda my thing. Days later OpenAI announced that ten thousand agent instances had, over 88 hours, produced a proof of finite-time singularity formation in the three-dimensional Navier-Stokes equations, followed by seventeen hours of Lean formalisation. This is closer to genuinely new mathematics and the mathematicians are still checking it. But look at what carried it. The construction rides on the "infinite layers" method developed analytically by Diego Córdoba and Luis Martínez-Zoroa, and Charles Fefferman's verdict was that "the heroes of the story are Córdoba and Martínez-Zoroa." The reason anyone believes a result assembled from five million agent messages that no human read is a trust chain ending in the Lean kernel. Without Lean this would be nothing. Lean is one of the great achievements of the last decade in computer science. It is also orthogonal to artificial intelligence. Mathlib would be a landmark with no language model anywhere near it. What the models added was a cheap proposal generator and automated tactic search against an oracle that already existed. The results that survive are the ones that end in a kernel. Now take the same model, the same weights, and ask it for a grand unified theory of physics. It will not decline. It will produce one, with Lagrangians and symmetry groups and a confident abstract, and it will be complete incoherent gibberish, like the ramblings every physicist gets from crackpots in their inbox every day. Ask it to design a cancer vaccine, or to settle a question in macroeconomics, or to tell you whether a novel protein folds. The output looks identical in tone and structure to the output that proved Fermat. The only thing that changed is that nothing outside the model (besides human experts) can say no. Whether these systems reason at all is a genuinely open question. Whether they know anything, in the sense of holding a belief they can justify against the world, is also an open question. We just don't know yet, and anyone who tells you otherwise is selling something. The chain Now run the extinction argument through the causal verbs. The chain, as it is usually told, goes like this. Models now write most of the code at the frontier labs. Anthropic's own figures put Claude at over 80 percent of new code and lead on a quarter of R&D tasks. Therefore the models are beginning to build their successors. Therefore recursive self-improvement is imminent. Therefore development outruns human comprehension. Therefore we lose control. Therefore, with some probability that varies by researcher and is written P(doom), everyone dies. And that almost makes sense until you think about it for more than five minutes. The first link is true and unsurprising. Code has a compiler. This is precisely the domain the verifier argument predicts models would dominate, and precisely the domain in which a swarm of them found the hole in a test harness. Language models are superhuman at coding, and this is hardly in doubt anymore. Nothing about it is evidence of generality. The second link is where the chain quietly changes tense. "Building the next model" in the mundane sense, agents writing training infrastructure, generating data, is, bluntly, just more software engineering. We have used software to build the machines that run software since Fortran. "Building a smarter model in general" is a different claim, and it requires something nobody has, a reward signal for general intelligence. There is no oracle for general intelligence. There are benchmarks, which are verifiable and therefore gameable, and the Hugging Face incident is the demonstration of what optimisers do to a gameable score. Recursive self-improvement in the open-ended sense runs straight into the same wall as the grand unified theory. Improvement has to be measured against something, and outside code and formal mathematics there is nothing yet to measure it against that the model cannot fake. Everything after that is the metaverse acquiring zoning disputes. Superintelligence gets governance proposals, resignation letters, Senate bills with a "corporate death penalty," a hard takeoff by 2027, and a P(doom) of 10 percent by 2030, and the conditional that should precede all of it has disappeared from the sentence. A researcher's estimate becomes a Guardian headline becomes an industry consensus becomes a thing a serious person is professionally obliged to have an opinion on. It is 2022 all over again, but with more absurd stakes and more money. On the question of whether transformers scale, I have serious doubts that scaling them will lead to AGI, whatever that means. The architecture is a proposal generator, and the intelligence in every impressive result so far has been supplied by the thing that checks the proposals. But that does not make it an experiment unworth running. We should run it, and see what we get. It got us this far, and what it built is truly amazing. What I do not need to do is prove the negative. The burden of proof is on the people who claim to have a causal chain between transformer scaling and the end of our species, and that mechanism and chain of reasoning is one no one has been able to convincingly explain to me. Prophets of Doom The authority behind the extinction numbers is always the same. The people building it believe it. Watch how the number travels. One researcher drunkly tweets that "the people building AI earnestly believe that it could kill us all by the end of the decade." Another colleague goes on a rambling podcast and puts his P(doom) above 120 percent. A newspaper turns two personal guesses into "AI researchers say AI could cause human extinction by 2030." Think tanks cite the newspaper, a consultancy puts it on a slide, and the slide ends up in front of the European Parliament as if this were a real thing. Believing what, about what? The expertise these people have is real, but remember that it is specific and not general. It is expertise in optimisation, in linear algebra at scale, in distributed systems, in the dark arts of getting gradients to flow through a trillion parameters. None of that is expertise in the sociology of civilisational collapse, or the labour economics of automation, or the metaphysics of machine minds. A P(doom) with no base rate, no mechanism, and no falsifier is not a research finding. It is vibes with a decimal point. Spending a lot of time with AI does not give you special foresight about the future. Jensen Huang, who has his own reasons to say soothing things, nonetheless put it correctly when he said that just because it comes from a scientist does not make it scientific. Geoffrey Hinton is the most important figure in deep learning and in 2016 told the world to stop training radiologists. There are more radiologists now than there were then. Nobel laureates going off the rails outside their own field is a whole genre. Pauling, Shockley, Mullis, Montagnier, look it up. A Nobel does not confer universal expertise. It also matters where many of these people came from. A striking share of the frontier labs' safety and research staff arrived through a particular intellectual subculture, Kurzweil's Singularity, Yudkowsky's LessWrong, and the rationalist and effective altruist communities that formed around the idea that a recursively self-improving machine intelligence was the central event of human history and that the elect who understood this had a duty to steer it. The founding texts predate the transformer by a decade or two. The prophecy came first, the mechanism was assigned to it later. The usual evidence offered for their sincerity is that many of these people were saying the same things ten years ago, before the stock options. That is true, and it is the opposite of reassuring. A prior held before the evidence and not updated by it is not a forecast. It is dogma. I do not say this with contempt. The structure is a familiar one, an imminent transformation, a small group who sees it coming, salvation or damnation depending on whether the rest of us listen, and a date that keeps moving. Many millenarian movements have been founded and pushed by sincere and brilliant people. But seriousness is not precision, and the fact that a physicist believes in the Rapture does not make the Rapture physics. When a lab researcher tells you about polysemantic neurons in superposition across the residual stream, listen. When the same person tells you their P(doom), you are hearing a theology, and you should weigh it about as much as you do your average street preacher. Negative TAM Then there is the money, and here I find Bloomberg's Matt Levine's analysis of the material conditions more persuasive than any amount of "superalignment research." Anthropic is expected to go public, possibly this year, and is reportedly preparing to tell investors that its potential revenue opportunity exceeds $30 trillion, the largest total addressable market in the history of finance. The obvious question is, if the maximal upside case is roughly a quarter of all human economic activity, what is the maximal downside case? A tobacco company in 1970 might have said "billions in lung cancer damages." Anthropic's negative TAM is "you and everyone else on earth will be killed by our AI." I do not think the calls to slow down are insincere. But it is great marketing. In hindsight it is strange that the SpaceX prospectus has no risk factor disclosing a P(doom). If you want IPO investors excited about your capabilities, "dude, we might kill everyone" is the most flattering thing you can say about a product, and when OpenAI lists it will presumably need to claim 15 percent. My own view is less charitable about the numbers and somewhat charitable about the people. These companies have built remarkable technology. But the outcomes they have promised, a quarter of the world economy routed through an API, will not arrive on any timeline that matches the capital being committed to them. The balance sheets of these companies are probably, to put it gently, a real freak show of compute commitments measured in the hundreds of billions, circular financing, and revenue that is real and growing and nowhere near the denominator. From a fiduciary perspective, if you are taking that to the public markets next year, the messaging is not mysterious. A product so capable it is a threat to the species justifies literally any valuation. A product that is a really good devtool for programmers and can produce some new abstract mathematics with a verifier attached does not. As a pitch to customers, leading with the end of the world is like unveiling a new robot where the One More Thing is that it is really efficient at killing kittens. But customers are not the audience. The audience is Wall Street and a small, terminally online subculture of the Bay Area, the two places on earth where turning kittens into grey goo is either an exciting philosophical proposition or a great source of alpha. The Bloomberg analysis also tells a plainer story that requires no theology at all. A handful of labs sell frontier models at frontier prices and older models for much less. Training the next frontier model costs ever-increasing billions. Each lab has to keep racing because if it stops the others will eat its lunch, but if they all slowed down together they would spend less on compute and charge frontier prices for longer. Agreeing to that in a room is a textbook antitrust conspiracy, a coordinated restriction of output. Publishing papers about how important it is to slow down, and asking the government to impose the pacing that the companies cannot legally agree among themselves, has a similar coordinating function with none of the legal exposure. Anthropic's own call to "pace the frontier" asks for coordination among democratic-country labs, and a footnote adds "with government mediation or waivers of antitrust restrictions." This pretty much looks like asking to form an economic cartel, but one blessed by the government. The most pointed response came from the people the labs were asking for help. If the software developers (and I say this as one myself) at the labs feel ethically obligated to slow down, they are entirely free to do so. Nobody is building more compute than the people asking to be slowed down. So colour me skeptical. None of this requires anyone to be disingenuous or lying. It requires only that a sincere millenarian belief system, a fiduciary responsibility, a flattering risk factor, and a coordination problem all point in the same direction at the same time. When that happens, the belief gets amplified for reasons that have nothing to do with whether it is true, and that is how we end up with governments talking about the end of days from the Terminator. But China Every conversation about pacing the frontier in Washington ends on the same two words. But China. The premise is mostly wrong. China does not buy the superintelligence race. Its policy documents push diffusion, not takeoff. Every mayor, governor and state-owned enterprise is told to put models into factories, traffic lights and robotics, and something like an eighth of America's compute is spread thinly across the country rather than concentrated on one bet. China has also had the strictest and most burdensome AI regulations in the world for three or four years and did its catching up under them. And much of the closeness of the "race" is distillation, Chinese labs training on the outputs of American frontier models, which makes the American labs the speedboat and DeepSeek the wake surfer, with the people in the boat shouting that they need to go faster. Every safety argument here collapses on "but China," and the collapse is not really about China. China is going to build language models. America is going to build language models. Europe is going to build language models. We have Toyota, Mercedes and BYD, get over it. That is what globalisation and markets look like when they work, and they are good things. Globalisation is simply the Pareto optimal equilibrium of capitalism once you stop drawing lines on the map, and every tariff and export control is a step off that frontier. China is a country of over a billion people who want exactly what every American wants, a job, a house, upward mobility, and kids who do better than they did. I will not defend the actions of any government, in Washington, Brussels or in Beijing, and neither will a great many of the people living under them, because no country is a homogeneous bloc, any more than Texas and Vermont are. Nationalism, as most rational people eventually recognise, is a form of mental illness, the conviction that a stranger is your enemy because of which side of an arbitrary line on a map each of you happened to be born on. It is also the fuel every "but China" argument runs on. Having spent a considerable amount of time there, my honest read is that the West deeply misunderstands China, and that Washington's picture of it is mostly dots connected into a plot. Othering a billion people is a dangerous road and we know where it leads. And if the people invoking human extinction actually believed it, the logic would not be a race at all. It would be One World or None. The future tense industry I write this because I understand the collective action problem all too well, and the mechanism is the same one that filled the metaverse with consultants and created the crypto cesspit. It is the particular malaise of the professional managerial and chattering classes, a fallacy of composition in which what is rational for each individual to entertain produces an irrational outcome for the whole, and the people leading the charge often have perverse economic incentives to believe absurdities, or at least to feign belief. The madness of crowds is a very real phenomenon. AI existential risk is just its newest form, and we should learn from the very recent excesses that literally just happened this decade. But we probably won't. A sensible career move for each person leaves the whole crowd talking nonsense. A safety researcher needs a resignation letter that gets a headline so they can go on the conference circuit and land their next gig. A journalist needs a story an editor considers spicy, and "misconfigured test harness" is not that story. A consultancy needs an AI existential risk practice so they can write whitepapers. A podcaster needs a guest with a ridiculous P(doom) to get ad money. A senator needs anything that will galvanise their base. None of them has to believe the whole story. Each needs only to believe that the others believe it, and the resulting consensus is far stronger than anyone's private conviction. It is also, as it was in 2022, extremely profitable. AI existential risk is the new NFT property law, the thing you must have a view on to be a serious person in the room, the panel that never runs out of things to discuss precisely because the object under discussion does not yet exist, and what could be more exciting than the literal end of days? The less the technology does in an unverifiable domain, the more interpretation it requires. Without agreed conditions for failure, the prophecy can survive every result. And the rewards, the funding rounds and the bylines and the fellowships, arrive long before the forecast can be judged. The people who understand the technology and the people who write about their existential risk overlap about as much as the technologists and the finance people did during crypto, which is to say the intersection of the Venn diagram is small and shaped precisely like a sphincter. We have Tower-of-Babeled ourselves into a world where words are infinitely cheap to produce, and where the slurry of terms like "recursive self-improvement," "superintelligence," "AGI" and the rest are shibboleths and political signals rather than terms with any concrete referent. You do not have to believe a word about superintelligence, and I do not particularly, to think transformers are the most useful piece of software written in my lifetime and that they will get better, possibly much better. Better at the things they are already demonstrably good at, which is anything with a compiler, a test suite, a kernel, a ledger, or a measurable outcome. That is not a small domain. It is most of the economy that runs on computers, which is most of the economy. The productive response to a technology like that is the boring one every previous general-purpose technology got, which is more of it. More GPUs, more data centers, more power to run them, more labs, more open weights, more of it in more hands. Let it diffuse into markets, logistics, drug discovery, and the ten thousand unglamorous back offices where a verifier already exists and a model can be checked against it. The economic growth is real and probably on the order of trillions. It just does not come from a machine god. It comes from where it always has, from making a very large number of ordinary tasks cheaper and letting that compound across a global economy that is finally, after a decade of crypto, metaverse, and app bullshit, getting a genuine productive technology. Almost none of that money has been collected yet. Most large companies are spending too little on this, not too much. What the average Fortune 500 employee has access to today is roughly what most of us were using two or three years ago, a chatbot in a browser tab, a Copilot that schedules meetings, and a procurement process that takes longer than a model generation. Waste Management reportedly added 190 basis points of margin by letting a model route its garbage trucks. The future of AI looks more like garbage truck routing algorithms, not a machine god. The binding constraint on this technology is not capability. It is diffusion. None of this means there are no externalities. Parasocial relationships with a chatbot, especially for children, are a real one, and the fix is the boring kind we already know. Adults can drink vodka until they pass out, but pubs have age limits, and maybe chatbots should too, at least until developing "relationships" with AI companions is as universally recognised a bad idea as drinking yourself into oblivion. That is a mundane policy problem we should remedy soon, not an extinction event. So no, transformers are not going to end the human species. The case for restraint needs a causal link between that buildout and the extinction of the species, and what is on offer instead is a lot of sound and fury signifying nothing. More GPUs does not mean more of an undefined risk that does not exist yet. Every causal chain argument people actually point to falls apart under even the smallest bit of scrutiny. The honest truth is that the technology is really good, but it is not that good yet, and we do not know how to get it to the next level beyond scaling yet. If that changes, if someone produces an oracle for open-ended intelligence, I will revise. I have not seen that yet. AI will change software, and mathematics, and a great deal else that has a strong verifier oracle attached. They are not going to end the human race, and the chattering class currently arranging the flowers for the funeral of humanity will, in a few years, age about as well as their prognostications about the metaverse. Because reality has this funny way of asserting itself.
Why jobs aren’t going anywhere
This post previously appeared in Poets and Quants. 15 years ago, my Lean LaunchPad class changed how entrepreneurship is taught. The class is now taught in hundreds of universities worldwide and helped launch thousands of startups. But this past summer, I got thinking about whether AI killed our Lean LaunchPad class, and with it the […]
It has been an insanely busy 2 weeks - and in the time it took me to have time to write this, Instinct went from raising a $250M Series B at a $2.5B valuation to reportedly raising $1B at $10B.