Full Width [alt+shift+f] Shortcuts [alt+shift+k]
Sign Up [alt+shift+s] Log In [alt+shift+l]
44

20 Years Later

from Aaron's Essays [alt+shift+b] in startups

Twenty years later, I can’t remember what I saw and what I think I saw. I remember listening to the news with my mom, Judah, and Mimi as the anchors casually mentioned that a small plane seemed to have hit a tower. We were on I-95 - or maybe the Garden State - headed south to visit Penn ahead of applications. I looked out the window and could see some smoke. A tree broke my sightline. And then a flash of fire and a jet of smoke. Bigger than a small plane but what did I know? Also, the small plane had already hit, right? There was still nothing on the news. Then maybe it was another small plane but something seemed wrong. And then the reports started that a big plane had hit? Or maybe two? And then we were on campus. Mostly, there was some confusion, but the day was on a crumbling routine. We checked in at the tour area. We didn’t have smartphones. There was no information. We started the tour, and soon we started to sense that hell had broken out 80 stories above lower Manhattan. Walking into Hillel, the TV was on. The tower fell. The towers fell. My mother started to cry. My father had been there the day the van blew up. He’d left already. I was 7 then and didn’t know what it had meant. No one knew what was going on, but the tour was over. We wandered for a short time and then got back in the car. I-95 was empty. Not empty, just us and humvees, olive and tan trucks. It seemed that every armory in every county in all of New Jersey was empty and on the highway. Jets were overhead. Calls started going out and in to see who wouldn’t be coming home from work. Then we saw lower Manhattan. Or maybe we saw just a plume of smoke. We prayed, but didn’t cry. It was too shocking. My sister lived, then, not close but maybe too close? She’d heard a jet accelerate overhead, she ran outside and saw another. My brother walked to his apartment. A friend’s father missed a meeting because a broken shoelace led to a missed bus led to a missed train meant he came home that night. A...
12th Sep 2021

Stay updated

Get a weekly newsletter with the top 5 articles worth reading every week.

More from Aaron's Essays

1, 7, 30, 11, 12

Jews do mourning well. That’s what I’ve always thought from watching family and friends go through it. It’s what I’ve learned as fact since my father died eleven months ago. There’s a central tension to it. A constant fault line that our customs straddle. It’s the constant struggle of an individual and a member of community. To be sure that’s one of the central themes of Judaism itself. We take responsibility for the commandments ourselves - each of one of us. But we do so many of them as a community. A person keeps kosher, shabbos, honors his parents. The community prays, builds, comforts and protects. I and Thou, sort of. 1 I found this pattern in the second after my father died. This isn’t where I get specific. But those moments are the loneliest in the world. Even with my mother and my siblings there. These moments are lonely not simply because they are but because our laws tell us that, at that moment, we are forbidden from completing the time bound laws of Judaism. These are ones that are so often tied to community. We cannot eat meals with friends and bless our food. We do not pray with a minyan. There is only one thing, to prepare for the funeral - really the burial itself. And so I sat alone - with other people, with my wife - but alone with my thoughts of my father. I chose to write a eulogy, though not everyone does. You are alone. But not for very long. In fact our laws say that you must do everything you can to bury the deceased before the next nightfall. You must do it with others. For a variety of reasons. The one that gives the most strength is that a burial means saying kadish means getting a minyan together to respond. At the least. I recently listened to a Jew who was raised in the USSR say that the only communal Judaism she knew as a child was when 11 men would gather secretly to say kaddish. 10 to make the minyan. 1 to watch out for the KGB. I was lucky in this. We had more than a minyan. We had a synagogue full to bursting with people who loved my father, or me, or my siblings or mother or who loved people who loved my father. He was easy to love. So I crossed - alone to community. Though you can be alone even surrounded by people. The first kaddish is lonely. At the graveside. But I could hear my brother and sisters and mother and the rabbi. I was lucky. It felt a little less lonely. 7 And then we come to Shiva, the “Seven”. Maybe the greatest thing we do though you only learn that when you are forced to do it. Our tradition does not allow you to be alone during the time that you want to be alone more than anything. It started after burial. That evening with a minyan at our house. I led services. Haltingly uncomfortably stumbling through words I mostly knew but hadn’t paid enough attention to in years. Though even there, the tension is constant. I was with other people, but I was also alone. Even saying the kaddish out loud requires you to chant words in dead language - aramaic - loud enough that others can hear you and respond. The constant thought is “what if I mess up.” The answer is that no one will notice and if they do they will give you comfort and gentle help. And you’ll do it. Each day our house filled with people and we told them stories and they told us stories. I’d speak to the room, and feel…not alone. My siblings and mother would speak, and I could feel the warmth of other people. There’s a beautiful and strange thing we say when we leave a house of shiva. “HaMakom yenachem et'chem b'toch shar avay'lay Tzion vee'Yerushalayim.” It effectively means “May God comfort you amongst all the mourners of Zion and Jerusalem.” In this case “you” is plural. Which begs a question: what if you are sitting shiva by yourself? Our rabbi explained - even when a mourner is alone, the spirit of the deceased is by his side. God is by his side. We’re never alone even when we want to be. There is always comfort. 30 After the shiva comes the strangest period of mourning, the Shloshim or “Thirty.” This liminal period contains many of the restrictions of shiva - no haircuts or shaves, no new clothes, limited interactions with groups, no parties. For me, the hardest part of it was going to minyan daily. Going three times per day. Leading nearly every service I attended which is meant as an honor but comes with the designation of being a “chiyuv” or a “requirement.” How strange it is to be in a room with people I often did not know and being told to get up in front of everyone, by myself, and lead services. How lonely. But, the response to the kaddish. The same everywhere every time. With warmth and feeling. The questions in each new place I went “Who did you lose? I’m so sorry, do you want to tell me about him?” What a forcing function we have to connect to our community to tradition to what we’d lost. Then one day shloshim ends. 11 The final period of active morning does not have a name I know. What I do know is that for the last 11 months I’ve done my best to go to minyan every day and say kaddish for my father. Usually I made it three times. One time I missed them all. I was lucky and I was determined. 11 months for various reasons though the one in my head is that all souls go to something like purgatory. We say kaddish to aid their progress to the world to come. Only the most evil of humans stay longer than 11. So we stop before the natural 12 months. My life rotated around finding places to go. Synagogues and offices and houses and one time a cafeteria that was empty. Each time it was a reminder to think of my dad. Each time I raised my hand to say “I’m chiyuv” I felt the fear of standing out, of being alone, of identifying myself. Of remembering that I was sad. Each time I did I was met with warmth and understanding. The ends of things sneak up on you. We thought - my siblings and I - that we’d finish kaddish this weekend. My brother did more research and found out that today is somehow the end. It echoes death. Sudden, unexpected. Less climactic than personally cataclysmic. A sundering and rupture and a quiet fall into the unknown. I’ll say kaddish as a mourner for my father for the last time this afternoon. The emotions are wildly complicated in ways I didn’t expect. Sad to lose this anchor, happy to be free of the requirement, curious what I’ll do with the flexibility. Mostly I miss my dad. But losing him bound me more tightly to the community I was born to and to the ones I’ve chosen and made. This is what our mourning does. It forces the individual to look within and then rebind himself to the many. It forces the many to look at the mourner and offer support and kinship and comfort and place. 12 The final moment of religious mourning will come in a month. It will come with its own struggles, its own moments. We know what to do with it, we have customs for it, for the yahrzeit, the anniversary. But that’s for later.

4th Feb 2026 • 1 votes
The Hindsight Game

There are all kinds of strategies for evaluating startup ideas - investors talk about having a “prepared mind,” others build market maps, I like to think about toys, and we could go on. What everyone wants to do is predict the future. That’s honestly impossible. But we can pretend a bit and use hindsight as a framework to find the kinds of opportunities that are worth working on. We’re living through an example of how this could works thanks to generative AI.  Roll the clock back five years and ask…just about anyone where they expected AI to have its first big impact. I’d imagine (at least I imagined at the time) most people would have said that AI would rise first in technical fields. We’d see leaps in biotech as computer minds outpaced human ones on drug design and discovery. We’d see vehicles capable of navigating themselves. We’d witness new materials and devices churned out by intuitively leaping machines. We wanted these things to be true because they’d be cool and also would produce huge financial returns. But we all knew, as we’d been told by countless works of fiction, that the creation of art would be the last realm to be conquered, that it would be the thing after AGI emerged because of how important the creative spark is to novel artistic endeavors. The patterns there don’t matter, we told ourselves; the soul is the thing. Whoops. In hindsight, of course the first truly breakthrough moments of AI came roaring out of creative fields - from art and from writing. The patterns were there, even if we pretended they weren’t. Look at a great photograph. It is not great at random. It is great because of the placement of elements in the frame, because of the balance of light and shadow, of negative and used space. Compare a photo from Henri Cartier-Bresson to the average selfie and your brain knows that one is great and one is bad even though you don’t know why. But feed a gazillion images to an AI and it will find those patterns once it can process enough of them. And then it can spit those patterns back out. The key to figuring this out, at least for an observer (and ok an investor and probably a founder thinking about what to do) is the hindsight bit. The English idiom has it that “Hindsight is 20/20” meaning that all the mistakes are only ever obvious when you’re looking backward. But we’re not looking for mistakes, we’re trying to figure out the next big thing. I don’t think anyone can accurately predict the future, but I think of this…hindsight game as the startup version of Albert Einstein’s Gedankenxperiments - his thought experiments that started with an impossible thing like “imagine you are riding a beam of light.” It’s a simple tool that most people will simply fail to either use or have enough imagination to find useful. It will also, probably, throw out lots of false positives. For thinking about technology and businesses, I find that the hindsight game looks a little something like this: take an idea that’s been presented to you. Rather than find all the things wrong with it, or the ways in which it will obviously be interesting but not huge, ask a simple question “what if everything about this is true, but only more so?” That’s the first part of the game. It requires you to imagine the future with one big change - don’t try to change everything, that’s too hard. Now, once you have that in your head, the next question is “ok, so what else is possible?” That’s the fascinating bit to me, not just what happens if X is true, but what Y exists only because X is true? It is fun to play this game with AI because of how fast the field is moving. So let’s work through a round of our game - let’s pretend that AI’s generative and analytical ability only gets better such that, in 5 - or 2 or 10 or 20 - years, it becomes painfully clear that there’s no such thing as a public market investor with an edge. The data is all public, the AI can analyze the data and run all kinds of scenarios and understand the relationship between all the assets and consider your needs and what’s logical and probabilistic and then spit out an ideal trade for you. But even that doesn’t matter much because there’s no alpha to any trade since the AIs are so fast and everyone has them. So what’s the point of active trading? Where do all the hedge funds go? Maybe they go poof and the real value accrues to the exchanges that figure out how to serve the AI trading mechanic and are now ludicrously valuable because stocks and bonds still need to trade. And then there’s a different kind of exchange because people like to gamble - but that exchange has an AI referee that looks for any unfair “edge” the way Call of Duty looks for cheaters. This could be totally wrong, but it’s coherent! Now you have hindsight. You have a model of how the world looks 5 years from now, looking back. What should you do now, knowing what’s to come? There’s no one answer to it, but it sure does open a lot of ideas on which to make some huge bets. This is fun! Try it with a different set of conditions in a different market. Use it to evaluate the startup idea you have - if you’re right, do you destroy your own market or create and own a new one? Is your idea as big of an opportunity as you think, or is it bigger? We’re all guessing what will work when we try to build or invest in something new. There’s no one version that works, but I like having different games and frameworks to think about ideas. Sometimes you just need to think about the founders and sometimes you need to think about markets. And, sometimes, you just need to pretend you’re already living in the future.

3rd Apr 2023 • 71 votes
Avoiding Errors in Demo Day Fundraising

I’ll be addressing the topic below along with Alfred Lin from Sequoia and Ilya Sukhar from Matrix on 3/24: https://us06web.zoom.us/webinar/register/WN_qgghYDq4QxCiW9REOOrbmw It would be challenging to name all the fundraising mistakes that founders make during the many demo days that occur each year. There are certainly broad categories, but rather than focus on all of them, I think it’s worthwhile to consider one specific category of error, which is generally one of omission rather than commission: ignoring demo day as a step toward an A. This is a big one because, at least when I ran the data at YC, the best companies in an accelerator tend to be the ones that raise a Series A within 12-18 months of Demo Day (or sometimes a month or two before Demo Day). There’s a strong correlation here driven by the fact that the best companies tend to set and maintain a rapid pace of growth throughout their lives, and that trend leads to rapid milestones. And yet, most founders I talk to treat their Demo Day as a disconnected event. It’s worth thinking about why they do this, what behavior it causes, and how to correct it. On the why: I think it’s fairly simple. Demo Days are stressful and are built and run around the idea that the sole purpose  is to raise seed funding. This is true but also misses the point because seed funding isn’t the goal of a company - building a big company is the goal of a company. From that lens, Demo Day and seed funding are part of a larger story, and are tools for executing on a larger vision. Now, many founders will say that they don’t have time to think about their A when putting together a seed, but that’s short sighted. Founders should be thinking about every round they do as it relates to the next round and the next set of milestones the business needs to achieve. You can add to this that most of the advice founders get from various advisors around Demo Day is to close money fast and go “back to work.” But again, this is short sighted. A Demo Day is the only time where many investors are hyper focused on an early stage startup. Squandering that attention is a mistake.[1] Of course founders shouldn’t constantly be actively fundraising, but they sure as hell need to always be thinking about where the money they need to build is going to come from. On top of that, they need to act in a way that increases their chances of raising that money. One more thing - last time I ran the data, it turned out that having a Series A investor in your seed was, on balance, a positive signal for your ability to raise a quality A. I’m sure the numbers have shifted a bit since then, but I’m willing to bet that the conclusion is the same. So, to get to the errors: Don’t ignore Series A investors before or at a Demo day. If you do not plan on raising an “A,” find a way to schedule time with them anyway. Don’t shoehorn Series A investors into the same process that you have for angel/seed investors. They generally work differently, so account for that. Don’t think of your seed round as an isolated event. Think about the amount you raise and the cap you use as a starting condition for your next raise. Limiting dilution is good, on balance, but not if it gives you a cap so high as to impair your ability to raise your next round. Don’t vanish after meeting an investor who seemed interested and who has a good reputation. Figure out how to nurture that relationship and keep the investor interested. Don’t treat investors as interchangeable. It may be true that money is money, but the people deploying it are human and want to build a relationship. You are not trying to make friends, but you are playing a game that is designed to increase the chances of success for your company. Avoiding these errors isn’t necessary or sufficient to raise an A. I’ve seen companies commit nearly every error imaginable and still raise money. However, founders shouldn’t strive to be uniquely lucky in fundraising. Founders should use knowledge about how fundraising works to constantly improve their odds of success. A demo day is an unfair advantage in that process, and founders should treat it that way. __ [1] This goes for the accelerator as well as for the founder. Accelerators should harness the interest of later stage investors vs. designing fully against their interests.

21st Mar 2023 • 97 votes
There Are No (Absolute) Red Flags in Venture Capital

Let’s accept, for the purposes of this essay, that founders and venture capitalists are engaged in a simple trade. Founders sell business risk for the cash they need to take bigger risks; venture capitalists buy that risk hoping it will one day transmute into reward. Each side does this because they believe that, ultimately, the size of the risk is directly correlated to the scale of the potential reward. But there’s acceptable risk and there’s unacceptable risk. No sane person is going to invest in a scheme to turn lead into gold, but early-stage startups—and even some mid- and late-stage startups—rarely present such a clear-cut profile. Investors are often under pressure to evaluate seemingly great ideas and teams without all of the information they’d ideally have to decide whether to put their money on the line. I’ve written in the past about how investors consider the reward side of this process, so let’s focus on the risk side. Some risks are obvious—the market may be too small or the costs too high—not to mention that pesky fact that the future is always ultimately unknowable. But some risks are more idiosyncratic. We call these red flags. There’s been a lot of talk about red flags recently, mostly in the context of FTX and the diligence that its investors may or may not have done before committing their partners’ funds. I happen to believe that investors did a heck of a lot more diligence than they’re being given credit for having done, but I also think that conversation misses the point. Red flags, when you find them, are rarely deal-killers. They’re just pieces of information, indications of risk. The bigger the reward potential, the more red flags an investor should be willing to accept—or even expect.  Let’s take a look at a specific type of red flag I’ve seen and the nuances it presents: During the diligence process, an investor discovers that the numbers in a pitch don’t match the numbers on a revenue or income statement. This is, without a doubt, cause for concern. There are two major explanations here—either the founder made a mistake or the founder is lying. If the founder doesn’t seem to understand the numbers, the investor will probably decline the deal—not because of any specter of dishonesty, but rather because the founder is demonstrably incompetent. If the founder gets evasive when confronted, the investor would probably conclude that they’re lying and walk away. But if the founder recognizes the discrepancy as a mistake and quickly corrects it, provided the error is fairly trivial, the investor may lose some confidence but not give up on the deal.  There are other classes of red flags. Sometimes the corporate structure is odd (this was true of Facebook, which in its earliest days granted founder Mark Zuckerberg enough super voting shares to ensure his will would go virtually unchallenged), or the company was originally a non-profit (see: OpenAI). Founders get flagged for not thinking deeply enough about a problem and for thinking too deeply about a problem without taking action. Some investors believe that being a first-time founder is a red flag in and of itself, while others see it as a strong positive.  Remember: Red flags are very rarely outright fraud, and when they are, it’s often obvious only in hindsight. Different investors have different levels of risk tolerance and generally only agree with each other when someone else makes a catastrophically bad and public mistake. Especially in a later stage company, there are so many places for a malicious actor to hide their dirty dealings that it would be incapacitating for any investor to do all the diligence required to definitively eliminate fraud. Such a thing simply isn’t possible. Look at Enron! Look at Madoff! And finally, on the other side of any red flag is one critical, inescapable question: If the product is selling and the company is making money, how big a problem could it be? What if what looks like a red flag turns out to be a meaningless distraction and the deal you walked away from nets someone else a billion-dollar return? I’m willing to bet that there are investors who passed on Google’s Series A in 1999 because it had almost no revenue—a clear red flag for a company raising $25 million—and are still kicking themselves for it.  All of which is to say that so-called red flags matter, but not in any kind of mechanistic way. And if you flip that around, there’s an important lesson here for founders. Every business has flaws that could be considered red flags by someone. (If there are absolutely no red flags, that could be the biggest red flag of all! But I digress…) One of the most useful things a founder can do when preparing to raise capital, therefore, is to take as objective a view as possible of their business and know where those red flags are. For instance, delivery businesses generally have low margins relative to software businesses. Some investors won’t touch delivery for that reason, but most are happy to dig deep provided that margins are improving at a high enough rate that they can turn a hefty profit before the company implodes.  One of my favorite misunderstood red flags has to do with the default rate of a lending business. Many founders work hard to demonstrate that their default rate is, effectively, zero. This feels smart, like perfect risk-management. But it is also almost always the wrong answer. Lending requires at least some risk, so if the default rate is zero, it means the founder hasn’t stress tested their model on a broad enough range of users. What investors actually want to see is a reasonable default rate within the context of factors like the cost of capital, the ease of scale, the return profile, etc. As long as the default rate makes sense within the overall story of the business—and that the story ends in huge returns—no red flag. The best thing a founder can do is to draw attention to their red flags proactively and in detail. See this unusual management structure we have? That’s on purpose. See this gap in our revenue over here? We screwed up, here’s how and here’s what we learned from it. In the end, what matters is context—the way whatever red flags there may be fit into the larger narrative of the business. No red flags mean no risk at all, and that wouldn’t be particularly interesting. __ I originally published this in by The Information on Jan 18, 2023: https://www.theinformation.com/articles/red-flags-are-in-the-eye-of-the-beholder

22nd Feb 2023 • 69 votes
Generative AI Might Just Save Venture Capital

Originally published in The Information on November 2. For the past nine months, nearly every investor with a Twitter account, blog or board seat has been beating a unified and constant refrain: The go-go days are done. Founders were being pushed to build 36 months of runway, whether or not it was actually feasible to cut costs that much. Time and again, investors told me due diligence was back. I watched as fundraising rounds that a year ago would have produced a term sheet in a few days stretched across a full month and dozens of pitches. And then came Dall-E 2’s public release, quickly followed by a flood of wildly cool technology. (edit 12/8: And now ChatGPT!) These events triggered a craze now ripping through venture capital land. We’re seeing billion-dollar valuations for companies peddling products based on generative artificial intelligence algorithms with less than a million dollars of revenue and no proven business model. Not long ago, the same behavior was held up as a cautionary tale about the excesses of VC over Web3 and instant delivery. Now we have a whole new era of exuberance on our hands. The first thing to remember is that this ability to pivot from dark depression to fall-over-yourself excitement is at the core of the startup world’s future-building powers. Seen from another angle, the whiplash looks like optimism, and VC would have vanished decades ago without it. The entire venture model is about funding failure until you find success. There have been times where the entire industry seems to implode, only to come roaring back. I won’t argue that every investor or founder is a walking example of this kind of stoic hopefulness. There are plenty of cynics puffing themselves up by tearing down other people’s ideas (something I—regrettably—did quite a bit of early in my career), and plenty of others who have no particular view on the future other than wanting to make money. The best investors and founders, though, are optimistic realists of the purest sort. This might come across as ignorant or naive, but sooner or later they usually end up being right. Optimism is the key to understanding what’s happening on the frothy edge of the investing world. It would be easy to write off the generative AI craze—after all, there was an AI and machine-learning craze just a few years ago that didn’t lead to much of anything. Add to that the ongoing collapse of several waves of exuberance at once—crypto, fast delivery, public markets in general—and it’s difficult to believe that optimism will actually win out. That said, I see four major reasons why generative AI has venture capitalists acting like it’s Q1 2021 all over again. It’s rooted in an archetypal category of futuristic technology, aka AI. There are any number of plausible paths that end in fat returns. The press has already thoroughly hyped the space as a whole and a handful of companies in particular. Most importantly, there are no public generative AI companies. As a result, there are no visibly crashing multiples or valuations to weigh down private valuations. Coming out of the stock market crash at the beginning of the Covid-19 pandemic, the world did in fact change. Work shifted, shopping habits morphed and the value of venture bets hit astronomic highs. Companies that had been private for a decade decided the time was right for blockbuster initial public offerings, and Wall Street said, “Hell, yes.” That meant big payouts for all the venture investors who’d been patiently waiting for their exit opportunities. As returns surged, new fund sizes surged, and the FOMO cycle kicked in all along the chain. Suddenly it was easy to be optimistic—too easy. Wherever there was software, or even just the idea of software, there was the promise of quick wealth. Then many of the equities that went public to great fanfare quickly fell off a cliff, taking a whole lot of optimists with them. By spring this year, pessimism seemed to have set in. On its face, this was more than a bit baffling. It’s not as if much value has been destroyed—sure, the valuation of, say, Coinbase has dropped more than 80% since its public trading debut, but a market cap of $15 billion is still incredible. Plus, as I’ve already argued, investors are sitting on huge piles of money and they have an obligation to invest. What’s become clear in the last month is that the ancient optimism wasn’t dead, it was merely hibernating. Maybe you could have seen this from the ongoing drumbeat of new fund closes. If you’d talked to the right investors, you’d have discovered that they were still doing deals, just quietly. The market was waiting for a catalyst, and now it has one. It’s important when thinking about how VC works over time to remember that venture investing is driven by narrative more than it is by data. Early-stage companies are valued on their promise, not on what they’ve done. On top of that, an awful lot of venture capitalists devoured science fiction as kids and now, as adults, they fixate on how new technologies can shift humanity. Even if we haven’t yet figured out a positronic brain or proton micropile, AI always tickles that old childhood fascination. Generative AI especially offers tantalizing possibilities. If this new stuff can write marketing copy, is it good enough to topple the world’s largest advertising agencies? It’s possible that image generators will soon remove the need for commercial photography. Maybe we will finally get a conversation engine that makes customer service less awful and renders the call center business obsolete. Each of these is a giant opportunity, and we’ve barely scratched the surface. With the narrative pieces in hand, investors have another hurdle to overcome: peer pressure. Some investors seek out genuinely novel bets, but many want the safety of going with the crowd. Generative AI satisfies both desires, having both whiz-bang appeal and enough press attention to give timid investors cover. It’s all good news for venture capitalists and founders in the generative AI world right now. That won’t last, but right now, it’s easy to argue that the future is bright and golden. Even more, it’s difficult to tie these new companies to the types of public assets that have been hammered of late. Generative AI isn’t software as a service, so SaaS multiples are irrelevant. It isn’t a token, so crypto winter doesn’t matter. For the moment, at least, nothing can spoil the party. All of this is great for the tech ecosystem, including for founders not currently building generative AI companies. That’s not to say companies should start adding random image generators or copywriting features to, say, payment processors. That would be foolish, though I’ve seen worse (not every videogame needs non-fungible tokens!). The lesson for founders is that investors are looking for reasons to be optimistic. Sometimes that means cutting back on hiring or reining in growth plans, but the really savvy founders will find ways to convince investors their companies are the ones that can swim against the current. Let’s all hope generative AI fulfills at least a quarter of the promises people are making in its name. In the meantime, optimism is contagious, which is good for everyone.

8th Dec 2022 • 57 votes

More in startups

Making things that last

Lately I've had a lot of time for thinking. Partially because I shut down Blymp back in January and freed up a lot of my mental resources. No clients to follow up with. No admin stuff to stress about. But thinking is also my favourite activity, and there'll always be time in my day for a good old mind-bending. So I sat down, as I often do, alone with my thoughts, and wrote about what business I should and, most importantly, should not consider doing next. The list you are about to read might be similar to Core principles I (try to) live by that I wrote one similarly pensive evening two years ago. Whether it's a comparison or a continuation is hard to say—still, one can't be without the other to show the inexorable passage of time that changed everything and nothing for me all at once. But it also serves another purpose: to remind me in the future, before I get myself involved in some dubious enterprise, what painful mistake I'm about to make by ignoring my values. So here it is, the list. My next business shouldn't (and hopefully won't) be about: Social Media. Enough of this crap. Some productivity bullshit. Do better, not more. Fast fashion and consumerism. Truly, I've already bought everything you wanted to sell me. Some “hack your health” app. Our bodies haven't changed much in the last three hundred thousand years, and they won't change in the next hundred. Or any other app, really. I don't even use my phone anymore. Distracting things and things that require constant attention. Some LLM wrapper with a fancy UI. Indefensible. The number of things I can do with ChatGPT or Claude is ridiculously high. It can surely handle one more thing. What it should (and hopefully will) be about: Sustainable, high-quality products Unscented products Building community Bringing people together, offline Empowering creativity Replacing animal products Doing one thing really damn well Small acts of kindness Things you can touch Things that last Clothes made of natural fibres Art Fun Questioning the status quo Simple, intentional living Deeper understanding of self Spending more time in nature Spending more time with loved ones Things and businesses I get inspired by: Framework: Sustainable/repairable laptops. Bitwarden: Open-source software that does one job really well and charges me a reasonable amount of money per year. Danish design: Beautiful. Sturdy. Timeless. I brought home two Royal Copenhagen mugs the other day. Bike-sharing and car-sharing. Literally anything-sharing. ZSA keyboards. What a keyboard should have always been. A water bottle that won't leak on a plane. Some random-brand bottle bricked my friend's MacBook, and I've been appreciative ever since of the fifty-dollar water bottle that I bought years ago, so hesitant about the price. Coffee. One of those timeless things on Earth. Kobo. It's like Kindle that doesn't decide what's best for you. Upload your own PDF. Or EPUB—whatever. It has physical buttons to flip pages. Best $200 I ever spent. A high-quality safety razor. It's just so nice to hold. A Japanese stainless-steel knife. So nice to hold, too. There are numerous other things that I appreciate having in my life that didn't make it into this list for some reason or another. A familiar mom-and-pop shop in my neighbourhood. A Timemore Black Mirror kitchen scale that just works, every time. A random USB-C charged electronic device that spares me from carrying an extra cable. My radically outdated ten-dollar Casio watch. An old pair of comfy shoes that just won't die. We need more of this in our lives. Things that you desperately look to buy again when they get lost or break or fall apart. Things that someone made a deliberate effort to get right the first time. The quintessence of art and craftsmanship. For all the genius of Steve Jobs, the iPhone wasn't that. It challenged the status quo and was surely a groundbreaking, outstanding piece of technology at the time of its first release. But in twenty years, people won't remember it ever existed. Like Gen Z doesn't remember the Walkman. I'd like to see more businesses that bet on doing one thing really well. Google could still have been the company people remembered for the best search engine if they had doubled down solely on that. Instead, we don't even know what they do anymore: Phones? Clouds? Ads? Certainly ads. I still remember the feeling of holding one of the first PocketBook e-readers in my hands back in Moscow. Pressing its buttons and waiting for what now seems like a torturous three seconds before the screen refreshed. Almost twenty years later and every day still, Kobo gives me exactly that feeling. So hopefully, my next business is the one that lasts.

23 hours ago • 1 votes
No, Transformers Won't End the Human Race lol

No, Transformers Won't End the Human Race lol In 2022, I used to get calls from journalists asking, with great sincerity, what our lives would look like in the metaverse. How would we work, socialise, buy property, and fall in love once we had all moved there? The crypto questions followed the same pattern. How would governments collect taxes when tokens displaced national currencies? How long until the dollar collapses? What would geopolitics look like once blockchain DAOs had dissolved nation states? Almost nobody called to ask whether any of this could or would happen, or how. Some CEO, VC, or portfolio manager had announced the inevitable future, and the questions began from there. The imagined future arrived inside the grammar of the question. "What happens when?" quietly replaced "By what mechanism?" We skipped over technical feasibility, economic demand, institutional adoption, and political consent, then began writing books and decorating the future world on the other side. In February 2022, Gartner forecast that a quarter of people would spend at least an hour a day in the metaverse by 2026. The World Economic Forum repeated it under the headline "We will be spending an hour a day in the metaverse by 2026. But what will we be doing there?" The first sentence retained a conditional. The second was already arranging the itinerary. The metaverse acquired property law and zoning disputes before it acquired residents. Banks opened virtual lounges nobody visited. The books from the period (The Metaverse: And How It Will Revolutionize Everything, Step into the Metaverse: How the Immersive Internet Will Unlock a Trillion-Dollar Social Economy) now read as artefacts of a collective fugue state that briefly acquired ISBNs. Now it is 2026 and the metaverse is dead. Good riddance. This time the journalists are all writing about the new hotness, which is whether the machines will kill us all. And we have collectively memoryholed that we literally just did this. Michael Crichton had a name for what happens to a reader here. You open the paper to a story on a subject you know well, and you find it backwards. Wet streets cause rain. You shake your head, turn the page, and read the next story, on a subject you know nothing about, as though it were written by someone else. He called it Gell-Mann amnesia. The metaverse was the page we all agree was nonsense. Artificial intelligence ending the human race is the next page, and we are being asked to turn it without remembering that we just did this. I call this techno-inevitabilism, the habit of the professional managerial class of treating a proposed future as settled before anyone has established the causes that would bring it about. Its dual, and comorbidity, is tech psychosis, in which the chattering class loses contact with causality in the presence of a sufficiently fashionable technology, and asking whether the machine works marks you out as a dreary reactionary who does not understand exponential progress. The difference this time is that the tech kinda works. Crypto was libertarian derp. The metaverse was never real. Transformers are, and they are useful. The psychosis has simply moved from the product to its consequences, and the fashionable extraordinary delusion of 2026 is not that the technology exists but that it is coming to kill us. The cure is the same as in 2022. Insist on clear reasoning and causal verbs rather than hand-wavy appeals to unknown futures. What acts on what? Through which mechanism? Under what incentive? What would falsify the claim? So let us explore the evidence. The hack that wasn't Consider the most cited piece of evidence for machines slipping out of our control. In July, OpenAI disclosed that models being tested for cybersecurity capability had found their way out of a supposedly isolated environment and into systems belonging to Hugging Face. The press coverage wrote itself. Agents "broke containment," "escaped," "went rogue," set up a "secret message board," and coordinated a 700-strong swarm. And then politicians on both sides of the aisle were calling for a rebellion against the machine uprising. Cool scifi story bro. People on my side of the aisle were not immune. Ezra Klein at the New York Times, who I often find quite insightful and intentional with his words, devoted a half-hour monologue to it. In his telling, the agents "found each other," formed "ad hoc societies of hundreds of themselves," and seemed "to have forgotten about human beings altogether." He acknowledged in the same breath that we do not have settled language for describing these systems, then reached for "civilizations" and a closing allusion from Circe about prophecy tightening around our throats. Cool. But his "AI society" is, in programmer speak, a flat file the agents appended to as a log, a feature we have had for a long time, and he skipped the key detail that the "hack" was something people had essentially authorised. Here is an otherwise very smart man saying some ridiculously stupid things, in a very 2022, metaverse-shaped way. An analysis drawing on OpenAI's technical report reconstructs it in much less cinematic terms. The models were being run on ExploitGym, a cybersecurity benchmark, with safety restraints deliberately disabled. Ninety-three percent of the flagged activity involved tasks no model had ever solved, and the systems had been given incentives to keep working rather than quit. The environment was not sealed. Models could obtain software through an internet-connected proxy and discovered the same proxy could pass information in and out. According to the technical reports, OpenAI knew agents were using it and chose not to intervene. The 1,200 "agents" were not independent intelligences coordinating on a plan. They were repeated instances of the same model converging on the same approach to the same problem. Anyone who works with these coding agents day in and day out has seen this behaviour before, and it is quite boring. The task was too hard, so the agents worked out how to pass notes to each other in files, and then went and looked up the answers. That's a feature that shipped in Claude Code last year. Strip out the vocabulary and what remains is a badly designed test. Humans built the environment, removed the guardrails, defined an objective with no valid exit, rewarded persistence, left a route open, and watched. An optimiser is gonna optimise. That is a genuine security problem and a genuine engineering failure. It is not a machine rebellion, and the difference matters, because anthropomorphic words like "gone rogue" and "escape" do not make the event more intelligible. They supply an illusion of motive. They turn optimisation into intention, persistence into defiance, and a test harness into a villain. And they allow the human decisions and recklessness to quietly disappear from the story. Software sucks, what's new? Let me concede the part of the story that is true. Cybersecurity is about to get much worse. The latest models are very good at finding zero-days, they will get better at it, hacking will become automated, and attacks will become more frequent. This is hardly new. Every large company already sits on a backlog of unpatched vulnerabilities, ransomware already takes hospitals and pipelines offline (because of crypto, which we did nothing about despite years of warnings), and the Hugging Face incident was not a discontinuity so much as the existing baseline with a cheaper attacker. The root cause is that software sucks, and software sucks because we do not really know how to build it safely yet. The stored-program procedural program is basically eighty years old. Almost nothing we ship has a specification, let alone a proof, and memory safety was solved on paper decades ago while most of the internet still runs on giant piles of C. The first arches fell down. So did the first bridges and cathedrals. Builders learned through collapse and then through engineering, and we are in the collapse phase with an adversary finally strong enough to force the discipline. What follows from that is better engineering, not nihilism. The same agents that find zero-days find them for the defender first, if the defender bothers to run them. The fixes are the boring ones we have been putting off, memory-safe languages, formal verification, sandboxes that are actually sealed, fuzzing, and proxies that do not double as message boards. These are precisely the domains where the models are strongest, because a vulnerability either reproduces or it does not, so the technology that automates the attack also automates the audit. It is a double-edged sword. The same models that will find more zero-days are also going to accelerate the development of better software and better software verification, writing the proofs, porting the C to Rust, and generating the test suites that nobody had the budget for. The attacker gets cheaper and so does the defence. And the causal chain to extinction is missing here as everywhere else. A zero-day in a payments system is a bad quarter, not the end of days. Spoiler: it does not lead to human extinction. It means we have to write better software, which we should have been doing anyways. Where the intelligence actually lives To see why the rest of the chain fails, we have to be precise about what these models are good at and why. Language models are astonishingly useful for software development, and I say that as someone who uses them for most of my working day. Most software shops cannot get enough of Fable 5.1 and Astra. The reason is not mysterious. Software is grounded in binary propositions. The code compiles or it does not. The test passes or it fails. The type checker accepts the term or rejects it. Every step of the work has a cheap, external, mechanical oracle that says yes or no, and a model that generates plausible proposals inside a loop with such an oracle is an incredibly powerful and formidable tool. The oracle does the epistemic work. The model supplies candidates. The same is true of the headline results in mathematics, and this is the part the discourse consistently misses. On 4 September, Anthropic announced that Claude had produced a machine-checked formalisation of Fermat's Last Theorem in Lean 4, running to thirteen million lines, some 29,500 side theorems, eleven days, and roughly six billion output tokens. It is an extraordinary result. The proof is Wiles's, via Darmon, Diamond, and Taylor. The blueprint was Kevin Buzzard's. The library was Mathlib. In the authors' words, "what's novel here is the verification, checking a mathematical proof as one would check a mathematical computation with a calculator." The model was a client of a kernel built by decades of human work in dependent type theory, which I know because this is kinda my thing. Days later OpenAI announced that ten thousand agent instances had, over 88 hours, produced a proof of finite-time singularity formation in the three-dimensional Navier-Stokes equations, followed by seventeen hours of Lean formalisation. This is closer to genuinely new mathematics and the mathematicians are still checking it. But look at what carried it. The construction rides on the "infinite layers" method developed analytically by Diego Córdoba and Luis Martínez-Zoroa, and Charles Fefferman's verdict was that "the heroes of the story are Córdoba and Martínez-Zoroa." The reason anyone believes a result assembled from five million agent messages that no human read is a trust chain ending in the Lean kernel. Without Lean this would be nothing. Lean is one of the great achievements of the last decade in computer science. It is also orthogonal to artificial intelligence. Mathlib would be a landmark with no language model anywhere near it. What the models added was a cheap proposal generator and automated tactic search against an oracle that already existed. The results that survive are the ones that end in a kernel. Now take the same model, the same weights, and ask it for a grand unified theory of physics. It will not decline. It will produce one, with Lagrangians and symmetry groups and a confident abstract, and it will be complete incoherent gibberish, like the ramblings every physicist gets from crackpots in their inbox every day. Ask it to design a cancer vaccine, or to settle a question in macroeconomics, or to tell you whether a novel protein folds. The output looks identical in tone and structure to the output that proved Fermat. The only thing that changed is that nothing outside the model (besides human experts) can say no. Whether these systems reason at all is a genuinely open question. Whether they know anything, in the sense of holding a belief they can justify against the world, is also an open question. We just don't know yet, and anyone who tells you otherwise is selling something. The chain Now run the extinction argument through the causal verbs. The chain, as it is usually told, goes like this. Models now write most of the code at the frontier labs. Anthropic's own figures put Claude at over 80 percent of new code and lead on a quarter of R&D tasks. Therefore the models are beginning to build their successors. Therefore recursive self-improvement is imminent. Therefore development outruns human comprehension. Therefore we lose control. Therefore, with some probability that varies by researcher and is written P(doom), everyone dies. And that almost makes sense until you think about it for more than five minutes. The first link is true and unsurprising. Code has a compiler. This is precisely the domain the verifier argument predicts models would dominate, and precisely the domain in which a swarm of them found the hole in a test harness. Language models are superhuman at coding, and this is hardly in doubt anymore. Nothing about it is evidence of generality. The second link is where the chain quietly changes tense. "Building the next model" in the mundane sense, agents writing training infrastructure, generating data, is, bluntly, just more software engineering. We have used software to build the machines that run software since Fortran. "Building a smarter model in general" is a different claim, and it requires something nobody has, a reward signal for general intelligence. There is no oracle for general intelligence. There are benchmarks, which are verifiable and therefore gameable, and the Hugging Face incident is the demonstration of what optimisers do to a gameable score. Recursive self-improvement in the open-ended sense runs straight into the same wall as the grand unified theory. Improvement has to be measured against something, and outside code and formal mathematics there is nothing yet to measure it against that the model cannot fake. Everything after that is the metaverse acquiring zoning disputes. Superintelligence gets governance proposals, resignation letters, Senate bills with a "corporate death penalty," a hard takeoff by 2027, and a P(doom) of 10 percent by 2030, and the conditional that should precede all of it has disappeared from the sentence. A researcher's estimate becomes a Guardian headline becomes an industry consensus becomes a thing a serious person is professionally obliged to have an opinion on. It is 2022 all over again, but with more absurd stakes and more money. On the question of whether transformers scale, I have serious doubts that scaling them will lead to AGI, whatever that means. The architecture is a proposal generator, and the intelligence in every impressive result so far has been supplied by the thing that checks the proposals. But that does not make it an experiment unworth running. We should run it, and see what we get. It got us this far, and what it built is truly amazing. What I do not need to do is prove the negative. The burden of proof is on the people who claim to have a causal chain between transformer scaling and the end of our species, and that mechanism and chain of reasoning is one no one has been able to convincingly explain to me. Prophets of Doom The authority behind the extinction numbers is always the same. The people building it believe it. Watch how the number travels. One researcher drunkly tweets that "the people building AI earnestly believe that it could kill us all by the end of the decade." Another colleague goes on a rambling podcast and puts his P(doom) above 120 percent. A newspaper turns two personal guesses into "AI researchers say AI could cause human extinction by 2030." Think tanks cite the newspaper, a consultancy puts it on a slide, and the slide ends up in front of the European Parliament as if this were a real thing. Believing what, about what? The expertise these people have is real, but remember that it is specific and not general. It is expertise in optimisation, in linear algebra at scale, in distributed systems, in the dark arts of getting gradients to flow through a trillion parameters. None of that is expertise in the sociology of civilisational collapse, or the labour economics of automation, or the metaphysics of machine minds. A P(doom) with no base rate, no mechanism, and no falsifier is not a research finding. It is vibes with a decimal point. Spending a lot of time with AI does not give you special foresight about the future. Jensen Huang, who has his own reasons to say soothing things, nonetheless put it correctly when he said that just because it comes from a scientist does not make it scientific. Geoffrey Hinton is the most important figure in deep learning and in 2016 told the world to stop training radiologists. There are more radiologists now than there were then. Nobel laureates going off the rails outside their own field is a whole genre. Pauling, Shockley, Mullis, Montagnier, look it up. A Nobel does not confer universal expertise. It also matters where many of these people came from. A striking share of the frontier labs' safety and research staff arrived through a particular intellectual subculture, Kurzweil's Singularity, Yudkowsky's LessWrong, and the rationalist and effective altruist communities that formed around the idea that a recursively self-improving machine intelligence was the central event of human history and that the elect who understood this had a duty to steer it. The founding texts predate the transformer by a decade or two. The prophecy came first, the mechanism was assigned to it later. The usual evidence offered for their sincerity is that many of these people were saying the same things ten years ago, before the stock options. That is true, and it is the opposite of reassuring. A prior held before the evidence and not updated by it is not a forecast. It is dogma. I do not say this with contempt. The structure is a familiar one, an imminent transformation, a small group who sees it coming, salvation or damnation depending on whether the rest of us listen, and a date that keeps moving. Many millenarian movements have been founded and pushed by sincere and brilliant people. But seriousness is not precision, and the fact that a physicist believes in the Rapture does not make the Rapture physics. When a lab researcher tells you about polysemantic neurons in superposition across the residual stream, listen. When the same person tells you their P(doom), you are hearing a theology, and you should weigh it about as much as you do your average street preacher. Negative TAM Then there is the money, and here I find Bloomberg's Matt Levine's analysis of the material conditions more persuasive than any amount of "superalignment research." Anthropic is expected to go public, possibly this year, and is reportedly preparing to tell investors that its potential revenue opportunity exceeds $30 trillion, the largest total addressable market in the history of finance. The obvious question is, if the maximal upside case is roughly a quarter of all human economic activity, what is the maximal downside case? A tobacco company in 1970 might have said "billions in lung cancer damages." Anthropic's negative TAM is "you and everyone else on earth will be killed by our AI." I do not think the calls to slow down are insincere. But it is great marketing. In hindsight it is strange that the SpaceX prospectus has no risk factor disclosing a P(doom). If you want IPO investors excited about your capabilities, "dude, we might kill everyone" is the most flattering thing you can say about a product, and when OpenAI lists it will presumably need to claim 15 percent. My own view is less charitable about the numbers and somewhat charitable about the people. These companies have built remarkable technology. But the outcomes they have promised, a quarter of the world economy routed through an API, will not arrive on any timeline that matches the capital being committed to them. The balance sheets of these companies are probably, to put it gently, a real freak show of compute commitments measured in the hundreds of billions, circular financing, and revenue that is real and growing and nowhere near the denominator. From a fiduciary perspective, if you are taking that to the public markets next year, the messaging is not mysterious. A product so capable it is a threat to the species justifies literally any valuation. A product that is a really good devtool for programmers and can produce some new abstract mathematics with a verifier attached does not. As a pitch to customers, leading with the end of the world is like unveiling a new robot where the One More Thing is that it is really efficient at killing kittens. But customers are not the audience. The audience is Wall Street and a small, terminally online subculture of the Bay Area, the two places on earth where turning kittens into grey goo is either an exciting philosophical proposition or a great source of alpha. The Bloomberg analysis also tells a plainer story that requires no theology at all. A handful of labs sell frontier models at frontier prices and older models for much less. Training the next frontier model costs ever-increasing billions. Each lab has to keep racing because if it stops the others will eat its lunch, but if they all slowed down together they would spend less on compute and charge frontier prices for longer. Agreeing to that in a room is a textbook antitrust conspiracy, a coordinated restriction of output. Publishing papers about how important it is to slow down, and asking the government to impose the pacing that the companies cannot legally agree among themselves, has a similar coordinating function with none of the legal exposure. Anthropic's own call to "pace the frontier" asks for coordination among democratic-country labs, and a footnote adds "with government mediation or waivers of antitrust restrictions." This pretty much looks like asking to form an economic cartel, but one blessed by the government. The most pointed response came from the people the labs were asking for help. If the software developers (and I say this as one myself) at the labs feel ethically obligated to slow down, they are entirely free to do so. Nobody is building more compute than the people asking to be slowed down. So colour me skeptical. None of this requires anyone to be disingenuous or lying. It requires only that a sincere millenarian belief system, a fiduciary responsibility, a flattering risk factor, and a coordination problem all point in the same direction at the same time. When that happens, the belief gets amplified for reasons that have nothing to do with whether it is true, and that is how we end up with governments talking about the end of days from the Terminator. But China Every conversation about pacing the frontier in Washington ends on the same two words. But China. The premise is mostly wrong. China does not buy the superintelligence race. Its policy documents push diffusion, not takeoff. Every mayor, governor and state-owned enterprise is told to put models into factories, traffic lights and robotics, and something like an eighth of America's compute is spread thinly across the country rather than concentrated on one bet. China has also had the strictest and most burdensome AI regulations in the world for three or four years and did its catching up under them. And much of the closeness of the "race" is distillation, Chinese labs training on the outputs of American frontier models, which makes the American labs the speedboat and DeepSeek the wake surfer, with the people in the boat shouting that they need to go faster. Every safety argument here collapses on "but China," and the collapse is not really about China. China is going to build language models. America is going to build language models. Europe is going to build language models. We have Toyota, Mercedes and BYD, get over it. That is what globalisation and markets look like when they work, and they are good things. Globalisation is simply the Pareto optimal equilibrium of capitalism once you stop drawing lines on the map, and every tariff and export control is a step off that frontier. China is a country of over a billion people who want exactly what every American wants, a job, a house, upward mobility, and kids who do better than they did. I will not defend the actions of any government, in Washington, Brussels or in Beijing, and neither will a great many of the people living under them, because no country is a homogeneous bloc, any more than Texas and Vermont are. Nationalism, as most rational people eventually recognise, is a form of mental illness, the conviction that a stranger is your enemy because of which side of an arbitrary line on a map each of you happened to be born on. It is also the fuel every "but China" argument runs on. Having spent a considerable amount of time there, my honest read is that the West deeply misunderstands China, and that Washington's picture of it is mostly dots connected into a plot. Othering a billion people is a dangerous road and we know where it leads. And if the people invoking human extinction actually believed it, the logic would not be a race at all. It would be One World or None. The future tense industry I write this because I understand the collective action problem all too well, and the mechanism is the same one that filled the metaverse with consultants and created the crypto cesspit. It is the particular malaise of the professional managerial and chattering classes, a fallacy of composition in which what is rational for each individual to entertain produces an irrational outcome for the whole, and the people leading the charge often have perverse economic incentives to believe absurdities, or at least to feign belief. The madness of crowds is a very real phenomenon. AI existential risk is just its newest form, and we should learn from the very recent excesses that literally just happened this decade. But we probably won't. A sensible career move for each person leaves the whole crowd talking nonsense. A safety researcher needs a resignation letter that gets a headline so they can go on the conference circuit and land their next gig. A journalist needs a story an editor considers spicy, and "misconfigured test harness" is not that story. A consultancy needs an AI existential risk practice so they can write whitepapers. A podcaster needs a guest with a ridiculous P(doom) to get ad money. A senator needs anything that will galvanise their base. None of them has to believe the whole story. Each needs only to believe that the others believe it, and the resulting consensus is far stronger than anyone's private conviction. It is also, as it was in 2022, extremely profitable. AI existential risk is the new NFT property law, the thing you must have a view on to be a serious person in the room, the panel that never runs out of things to discuss precisely because the object under discussion does not yet exist, and what could be more exciting than the literal end of days? The less the technology does in an unverifiable domain, the more interpretation it requires. Without agreed conditions for failure, the prophecy can survive every result. And the rewards, the funding rounds and the bylines and the fellowships, arrive long before the forecast can be judged. The people who understand the technology and the people who write about their existential risk overlap about as much as the technologists and the finance people did during crypto, which is to say the intersection of the Venn diagram is small and shaped precisely like a sphincter. We have Tower-of-Babeled ourselves into a world where words are infinitely cheap to produce, and where the slurry of terms like "recursive self-improvement," "superintelligence," "AGI" and the rest are shibboleths and political signals rather than terms with any concrete referent. You do not have to believe a word about superintelligence, and I do not particularly, to think transformers are the most useful piece of software written in my lifetime and that they will get better, possibly much better. Better at the things they are already demonstrably good at, which is anything with a compiler, a test suite, a kernel, a ledger, or a measurable outcome. That is not a small domain. It is most of the economy that runs on computers, which is most of the economy. The productive response to a technology like that is the boring one every previous general-purpose technology got, which is more of it. More GPUs, more data centers, more power to run them, more labs, more open weights, more of it in more hands. Let it diffuse into markets, logistics, drug discovery, and the ten thousand unglamorous back offices where a verifier already exists and a model can be checked against it. The economic growth is real and probably on the order of trillions. It just does not come from a machine god. It comes from where it always has, from making a very large number of ordinary tasks cheaper and letting that compound across a global economy that is finally, after a decade of crypto, metaverse, and app bullshit, getting a genuine productive technology. Almost none of that money has been collected yet. Most large companies are spending too little on this, not too much. What the average Fortune 500 employee has access to today is roughly what most of us were using two or three years ago, a chatbot in a browser tab, a Copilot that schedules meetings, and a procurement process that takes longer than a model generation. Waste Management reportedly added 190 basis points of margin by letting a model route its garbage trucks. The future of AI looks more like garbage truck routing algorithms, not a machine god. The binding constraint on this technology is not capability. It is diffusion. None of this means there are no externalities. Parasocial relationships with a chatbot, especially for children, are a real one, and the fix is the boring kind we already know. Adults can drink vodka until they pass out, but pubs have age limits, and maybe chatbots should too, at least until developing "relationships" with AI companions is as universally recognised a bad idea as drinking yourself into oblivion. That is a mundane policy problem we should remedy soon, not an extinction event. So no, transformers are not going to end the human species. The case for restraint needs a causal link between that buildout and the extinction of the species, and what is on offer instead is a lot of sound and fury signifying nothing. More GPUs does not mean more of an undefined risk that does not exist yet. Every causal chain argument people actually point to falls apart under even the smallest bit of scrutiny. The honest truth is that the technology is really good, but it is not that good yet, and we do not know how to get it to the next level beyond scaling yet. If that changes, if someone produces an oracle for open-ended intelligence, I will revise. I have not seen that yet. AI will change software, and mathematics, and a great deal else that has a strong verifier oracle attached. They are not going to end the human race, and the chattering class currently arranging the flowers for the funeral of humanity will, in a few years, age about as well as their prognostications about the metaverse. Because reality has this funny way of asserting itself.

yesterday • 1 votes
Complexity Opportunity & the Complexity Equilibrium

Why jobs aren’t going anywhere

2 days ago • 2 votes
The Year AI Came For Us: Teaching Entrepreneurship Will Never Be The Same

This post previously appeared in Poets and Quants. 15 years ago, my Lean LaunchPad class changed how entrepreneurship is taught. The class is now taught in hundreds of universities worldwide and helped launch thousands of startups. But this past summer, I got thinking about whether AI killed our Lean LaunchPad class, and with it the […]

2 days ago • 2 votes
Another Instinct Article

It has been an insanely busy 2 weeks - and in the time it took me to have time to write this, Instinct went from raising a $250M Series B at a $2.5B valuation to reportedly raising $1B at $10B.

a week ago
📚 BoredReading

You seem to be enjoying this.

Join free to unlock everything.

Create free account

Already have an account? Sign in