More from Pluralistic: Daily links from Cory Doctorow
Today's links Unpermissioned research: Fighting Trump means preserving the internet, which means scraping. Hey look at this: Delights to delectate. Object permanence: Leaked advanced PalmOS device specs; Stalwart workers; "I, Rowboat"; SMS uprising; Facebook v switching costs; Capitalism of fools. Upcoming appearances: Dąbrowa Górnicza, Warsaw, Brighton, London, Budapest, South Bend, Hudson, Victoria, Vancouver. Recent appearances: Where I've been. Latest books: You keep readin' em, I'll keep writin' 'em. Upcoming books: Like I said, I'll keep writin' 'em. Colophon: All the rest. Unpermissioned research (permalink) After half a century of neoliberalism, we are all drenched in capitalism's established religion, the worship of property rights. We are so marinated in property worship that even capitalism's critics frame their critiques in "property talk," to the exclusion of other, more important rights, like human rights, labor rights and privacy rights. To do this is to surrender before the battle even starts. Critics lose when they allow oligarchs and their apologists to choose a battlefield where they have a nearly unbeatable advantage. Take privacy: privacy is a human right, not a property right. Human rights aren't for sale. You can't sell yourself into slavery, you can't sell your kidneys to make the rent. If privacy is a property right – one that can be traded away – then Facebook's industrial-scale privacy invasions are actually fine, since you "traded" your privacy to Mark Zuckerberg in exchange for the privilege of talking to your friends. Some self-styled critics of tech monopolists say that the answer to Facebook's privacy invasions is to force the company to pay for your privacy with cash, rather than services: https://www.wired.com/story/opinion-andrew-yangs-plan-to-pay-you-for-your-data-doesnt-add-up/ This is ideological capture in its purest form: the "data dividend" that Facebook would owe you under this system amounts to a few dollars per year. For wealthy people, the sums would be trivial, while working people, who've been on the downward leg of every K-shaped recovery for a quarter century, who've maxed out their credit cards and re-mortgaged their homes and drive Uber on the weekends to make rent, would have to subject themselves to ongoing surveillance. That surveillance is already used to determine the highest price those working people will pay – companies like Plexure inform fast food places when you've just gotten paid so they can tack an extra dollar onto your breakfast burrito in the app: https://pluralistic.net/2026/04/30/something-must-be-done/#there-ive-done-something Being forced to sell your privacy doesn't just raise the prices you pay, it also lowers the wages you earn. The same people who can't afford this "pay or privacy" system have their private data used to calculate the lowest wage they'll accept for each ride on those weekend Uber shifts: https://pluralistic.net/2024/12/18/loose-flapping-ends/#luigi-has-a-point In other words: not being able to afford privacy will result in you having even less disposable income, which will mean that you'll have to sell even more of your privacy. Lather, rinse, repeat. But even the wealthy people who can afford to forego the pittances Facebook and others offer in exchange for their private information will find privacy elusive. That's because private information isn't a "rival good" – a thing only one person can own at a time. The fact that your mother is your mother "belongs" to both you and her, as well as your grandparents, your father, your siblings and your kids. The fact that you don't sell your family tree to a tech company won't stop all those other people from selling it on – as anyone whose foolish relations handed their genome over to 23andme can attest: https://www.npr.org/2025/03/24/nx-s1-5338622/23andme-bankruptcy-genetic-data-privacy In the property religion, the way you can tell if something is valuable is if it has a high price. Property cultists insist that the problem with privacy is that our privacy is being sold too cheaply. They're wrong: private information isn't "mispriced" – it shouldn't be priced. Human beings are the most valuable things in our world and they are literally priceless. Murder isn't "theft of life." Rape isn't "theft of sex." While insurers and civil courts have ways of calculating the "price" of an injury or violation, great care has been taken over the centuries to ensure that this does not turn human beings into commodities. You can't buy a "murder offset" that lets you kill people provided you pay into a fund that saves a human somewhere else: https://pluralistic.net/2021/04/14/for-sale-green-indulgences/#killer-analogy Human beings are too valuable to be priced. We have an entire, sui generis way of balancing the conflicting interests of human rights. My daughter and wife have rights over me, I have rights over them, and when those rights come into conflict – say, if my daughter believes I can no longer care for myself and wants to put me in a care home – the process for resolving that conflict isn't an auction: https://www.theguardian.com/technology/2008/feb/21/intellectual.property Your kids aren't your property. In fact, all the most important relationships in your life are non-market. Doctors have patients, not customers. Any time a doctor calls you a "customer" they are demoting you. A doctor doesn't sell you health. You have rights as a patient that far exceed the rights accruing to a mere customer. Same goes for other professions: Teachers have pupils, librarians have patrons, lawyers have clients. "Customer" is a demotion from all of these. As every "user agreement" you've ever clicked through demonstrates, Big Tech loves to have everything defined in property terms – and so does all big business. Take the fight over scraping for AI. You might think that this is a fight over the economic rights of creative workers – certainly, my fellow creative workers treat it as such. But because this debate is being framed in terms of property rights, rather than labor rights, this is a fight that workers are set up to lose. The tell here is how the media companies – who have been eroding the wages of creative workers for decades as they consolidated into a curdled, inbred oligopoly – describe the AI companies' scraping: as an unlicensed taking. Mitch Glazier, the $1.4m/year CEO of the Recording Industry Association of America issues press releases decrying AI training for image generators without negotiating a license fee first: https://pluralistic.net/2026/03/03/its-a-trap-2/#inheres-at-the-moment-of-fixation Who's Mitch Glazier? Oh, just a former Congressional staffer who was drummed out of the Capitol Building after he snuck a clause into must-pass legislation that would have transferred hundreds of millions of dollars from musicians to record labels, who was then immediately hired as the CEO of the record industry's largest lobbying group: https://www.eff.org/deeplinks/2013/12/tpps-attack-artists-termination-rights Mitch Glazier – and the businesses he represents – aren't opposed to AI replacing artists. They're opposed to AI replacing media companies. Remember the Hollywood writers' strike? The proposal to replace screenwriters with chatbots didn't come from OpenAI, it came from Disney, Warner, Universal and other companies who claim that AI training is "theft." If AI training is "theft," then it can be cured by making a purchase, something that the AI companies can easily afford, thanks to the hundreds of billions of dollars they have been given by the world's richest investors, who are the high priests and cardinals of the property religion. The Hollywood writers are the only workers in the world who have successfully beaten back the use of AI in their workplace, and they didn't do it by making recourse to property rights. The Writers Guild is a union and it enjoys a weak form of "sectoral bargaining" (where all the workers in a field bargain with all the businesses at once) called "multi-employer bargaining": https://pluralistic.net/2023/10/01/how-the-writers-guild-sunk-ais-ship/ The Hollywood writers' strike was an unqualified victory for the writers, who defended their labor rights to co-determination when it came to the use of new tools on their jobsite. Under the terms of their hard-fought contract, screenwriters don't have to use AI, but they can if they want. For example, writers on a long-running sitcom might train an AI with every script in the series' history, so they can ask a chatbot continuity questions as they beat out a new season of the show. But they don't have to do this if they don't want to, and even if they do, neither their wages nor their headcount can be reduced. The media companies insist that scraping is a copyright violation, that it's "theft." As a matter of law, this is far from obvious or settled: the process of making transient copies of many works, performing mathematical analysis on them, and then publishing that analysis as software is not obviously a copyright violation, and anyone who claims otherwise doesn't understand copyright: https://pluralistic.net/2023/02/09/ai-monkeys-paw/#bullied-schoolkids Worse: by demoting a labor rights issue to a mere property rights issue, AI critics are setting workers up to fail. Say the issue with AI training really is mere copyright. If that's so, the media companies who want nothing better than to pauperize creative workers can amend their standard contracts so that any worker who does business with them must irrevocably transfer their "AI training rights" to the company. Then, that company will absolutely, 100% license those rights to an AI company to create a model designed to replace that worker. The company will get paid for the training, and the resulting model will come with "guardrails" to stop other media companies from using proprietary data to compete with it. This is the story of the past 50 years of copyright expansion: every new copyright we've created "to help artists" was scooped up by their bosses, who grew more powerful and were able to demand more concessions from those artists, who were therefore poorer and thus needed more copyrights to help them (lather, rinse, repeat): https://pluralistic.net/2026/08/18/enron-corpus/#sign-here If creative workers' AI fight is merely a copyright fight, then that fight can only determine whether media companies or tech companies will get the biggest portion when those workers are devoured by corporations. Only a labor rights fight can take creative workers off the menu altogether. Treating AI training as "theft" creates harms whose blast radius extends well beyond creative workers' livelihoods. Scraping is a hugely beneficial activity. If scraping – taking a vast corpus of copyrighted works without permission – is theft, then every search engine is a crime, unless it can afford to license "search indexing rights" from every site on the internet. There's exactly one company that could pull that off: Google, a rapacious tech monopolist that is – not coincidentally – one of the leaders of the movement to beggar every creative worker. We will not improve the world, the internet, or creative workers' lives by ensuring that the last search engine anyone ever creates is Google. Remember our earlier discussion of how privacy violations are weaponized to make poor people even poorer, by depressing their wages and raising prices based on inferences about their economic desperation? Our best weapon for fighting this practice is scraping, because that's how we catch corporations changing prices and wages based on surveillance data: https://pluralistic.net/2023/09/17/how-to-think-about-scraping/ Scraping is how we produce evidence of the changes that powerful people are making to the world around us. Do you want to know whether Mark Zuckerberg or Elon Musk are downranking content critical of Trump and Big Tech and pumping racist and conspiratorial posts into the resulting void? You'd better hope you can scrape the feeds they cram into billions of people's eyeballs. Same goes for keeping track of genocide apologists, data-center astroturfers and ICE cheerleaders who've flooded Tiktok ever since Trump stole it and handed it over to his creepy billionaire pal Larry Ellison. Making copies of that stuff isn't theft. It's not a copyright violation. Not even if you do it to billions of works. Not even if it's bad for the companies whose feeds you're capturing. Not even if it's bad for the dark money groups who funded the content. Sure, if you do this carelessly or recklessly, you can end up violating someone's labor rights, or privacy rights, or human rights. And because those frameworks aren't based on the sanctity of property rights, they can be used to protect these important rights without giving corporate America the right to have you fined or arrested for documenting their takeover of the America. The people who keep track of this stuff are worried about being fined or arrested. Ethan Zuckerman, one of America's foundational internet scholars, has just accepted Canadian government funding to move his lab from UMass to McGill in Montreal: https://ethanzuckerman.com/2026/08/27/my-personal-contribution-to-the-us-canada-trade-war/ Zuckerman studies platform power: "using data to answer hard questions about social media, search engines and AI tools." He leads a team that is documenting exactly, precisely how tech companies collude with authoritarians to spy on us, manipulate us, and control us. And his methodology is something called "unpermissioned research," which is what academics call scraping: https://www.techpolicy.press/ai-companies-threaten-independent-social-media-research/ "Unpermissioned research" seeks to circumvent limits that platforms establish specifically to stop outsiders from learning how they operate. When you're doing unpermissioned research, you try to get around rate limits, query throttles, and other measures that platforms use to block others from mapping their extent and documenting their conduct. "Unpermissioned research" isn't a free-for-all. Universities have ethical rules designed to protect the privacy rights and other human rights of research subjects, and because these aren't property rights, they can be balanced against the socially beneficial outcomes of research. Universities can get this wrong, of course, but when they do, it's not theft. It's a human rights violation, a privacy violation, a labor violation. If you want to know how AI companies are trying to destroy creators' livelihoods, you have to scrape the AI companies. You can't ask companies for permission to gather information that might be used to destroy them – they'll just say no. If taking information off the internet without permission is "theft," then gathering information by scraping AI companies is also theft. Sometimes a tech company will set up a "research portal" that supposedly obviates the need to scrape by putting all the relevant information in one convenient place. That's what Facebook did in the wake of the 2016 election, when it was widely condemned for publishing paid political disinformation. But Facebook's official research portal omitted vast amounts of paid political disinformation, something we only know because NYU set up a scraping project called Ad Observer that documented the discrepancy: https://pluralistic.net/2021/08/06/get-you-coming-and-going/#potemkin-research-program Facebook used legal threats to kill Ad Observer, and then…they killed their official research portal, too: https://pluralistic.net/2021/07/15/three-wise-zucks-in-a-trenchcoat/#inconvenient-truth Zuckerman is one of dozens of leading US academics who are relocating their labs and teams to Canadian universities, citing fear of political interference from the Trump regime: https://vancouver.citynews.ca/2026/08/27/canada-recruits-dozens-of-foreign-scientists-researchers-poaching-many-from-u-s/ The Canadian government has committed $504m to the project. Some of that research will help Canada develop new green energy, and some of it will help Canada make important medical breakthroughs. But Zuckerman's research has a special place in the portfolio of Canadian research projects, because – thanks to scraping – it is a leading source of information about how Trump's tech companies are waging war on the American people and the world. Scraping isn't theft of data, just like murder isn't theft of life. Scraping can be harmful, and we can create laws and social regimes and ways of talking about those harms that don't give authoritarian governments and vast multinational corporations the right to decide who can document and analyze their conduct. Take Wikipedia: the project exists solely to organize and disseminate information, for free, to everyone in the world. Wikipedia is among the most important parts of the internet, and one of the most positive developments of the 21st century. The entire project is licensed under a generous Creative Commons license that encourages unlimited commercial re-use of its contents. Even if you think scraping copyrighted works is theft, scraping Creative Commons Attribution 4.0 works is unquestionably not theft. But Wikipedia is being hammered by AI scrapers, which are operating so aggressively that they threaten the project's ability to keep its servers online. Wikipedia has an AI problem, but that AI problem isn't "theft" – it's denial of service, the aggressive act of intentionally or recklessly flooding a server with so much traffic that it crashes. If you've been lured into a cultlike worship of property rights, this seems like a contradiction. But once you relegate the relatively unimportant matter of property rights to its correct station, you can see – and reason about – the universe of rights that are far more important than mere property. All it takes is realizing that there are far worse things you can do with information than "stealing" it. (Image: Bearas, CC BY-SA 4.0, modified) Hey look at this (permalink) Meta's $17 Billion Settlement is a Bad Deal for Teens and All Social Media Users https://www.eff.org/deeplinks/2026/09/metas-17-billion-settlement-bad-deal-teens-and-all-social-media-users New Twitter launches, says Musk’s X gave up the name https://arstechnica.com/tech-policy/2026/08/new-twitter-launches-says-musks-x-gave-up-the-name/ For The Economist, Mentioning Workers’ Interests Is Heresy https://jacobin.com/2026/08/acemoglu-economics-ai-automation-working-class How Youth and Educators Can Fight Enshittified Tech https://clalliance.org/blog/how-youth-and-educators-can-fight-enshittified-tech/ Stalking the Wily Hacker: 40 years later – Cliff Stoll https://www.youtube.com/watch?v=656058JxTM0 Object permanence (permalink) #25yrsago NYT says ebooks don't exist, fails to mention thriving ebook pirate scene https://www.nytimes.com/2001/08/28/business/forecasts-of-an-e-book-era-were-it-seems-premature.html #25yrsago Parking tickets waived in exchange for written apologies https://web.archive.org/web/20010826013513/http://www.thesmokinggun.com/doc_o_day/lewiston1.shtml #20yrsago "I, Row-Boat" https://web.archive.org/web/20060000000000*/http://www.flurb.net/1/doctorow.htm #20yrsago Filipino students use SMS to organize mass demonstrations https://web.archive.org/web/20060902160514/http://blog.wired.com/sterling/index.blog%3Fentry_id%3D1545927 #20yrsago Spam pump-and-dumps work http://news.bbc.co.uk/2/hi/technology/5284618.stm #25yrsago Leaked: Handspring's next PalmOS device https://web.archive.org/web/20020824213501/http://www.palmstation.com/view_article.asp?article=4614 #15yrsago “Stalwart Workers”: neglected backbone of the firm https://web.archive.org/web/20110920155246/http://blogs.hbr.org/hbsfaculty/2011/08/stop-ignoring-the-stalwart-wor.html #5yrsago Facebook's war on switching costs https://pluralistic.net/2021/08/28/talking-hard-work-blues/#hostage-takers #5yrsago The "work ethic" is a dirty trick we play on ourselves https://pluralistic.net/2021/08/28/talking-hard-work-blues/#work-will-set-you-free #1yrago The capitalism of fools https://pluralistic.net/2025/08/28/strew-deal/#neither-fish-nor-fowla Upcoming appearances (permalink) Dąbrowa Górnicza, Kongres Regeneracja! to interdyscyplinarna, Sep 5 https://regeneracja.plse.org.pl/ Warsaw: Romana i Jana Podoskich, Sep 6 https://wydarzenia.phub.pl/events/0ee0e198-f843-423a-890f-c84ff50a46c0 Brighton: The Reverse Centaur's Guide to Life After AI with Carole Cadwalladr (Brighton Dome), Sep 8 https://brightondome.org/whats-on/LSC-cory-doctorow-the-reverse-centaurs-guide-to-life-after-ai/ London: The Reverse Centaur's Guide to Life After AI with Riley Quinn (Foyle's Picadilly), Sep 9 https://www.foyles.co.uk/events/enshittification-cory-doctorow-riley-quinn Budapest: Brain Bar, Sep 17 https://brainbar.com/munkatars/cory-doctorow South Bend: An Evening With Cory Doctorow (Notre Dame), Oct 6 https://franco.nd.edu/events/2026/10/06/an-evening-with-cory-doctorow/ Hudson, OH: Hudson Library, Oct 7 https://engagedpatrons.org/EventsExtended.cfm?SiteID=3850&EventID=596952&PK= Victoria: Munro's Books, Oct 20 https://www.munrobooks.com/events/6113620261020 Vancouver: BC Policy Solutions Gala, Nov 12 https://bcpolicy.ca/gala/ Recent appearances (permalink) How Tech Platforms Took Over the Economy (Dystopia Now) https://sites.libsyn.com/566555/enshittification-and-reverse-centaurs-cory-doctorow-on-how-tech-platforms-took-over-the-economy Hope, AI, Fixing the Internet and the Reverse Centaur of it all (Wilosophy) https://podcastaddict.com/everyone-relax/episode/231414816 Deflating the AI Bubble (Do Not Pass Go) https://www.donotpassgo.ca/p/deflating-the-ai-bubble-with-cory Technofeudal Enshittification (Fucking Cancelled) https://www.fuckingcancelled.com/p/technofeudal-enshittification-with Who The Machine Serves (EFF) https://archive.org/details/effecting-change-who-the-machine-serves Latest books (permalink) "The Reverse-Centaur's Guide to AI," a short book about being a better AI critic, Farrar, Straus and Giroux, June 2026 https://us.macmillan.com/books/9780374621568/thereversecentaursguidetolifeafterai/ "Canny Valley": A limited edition collection of the collages I create for Pluralistic, self-published, September 2025 https://pluralistic.net/2025/09/04/illustrious/#chairman-bruce "Enshittification: Why Everything Suddenly Got Worse and What to Do About It," Farrar, Straus, Giroux, October 7 2025 https://us.macmillan.com/books/9780374619329/enshittification/ "Picks and Shovels": a sequel to "Red Team Blues," about the heroic era of the PC, Tor Books (US), Head of Zeus (UK), February 2025 (https://us.macmillan.com/books/9781250865908/picksandshovels). "The Bezzle": a sequel to "Red Team Blues," about prison-tech and other grifts, Tor Books (US), Head of Zeus (UK), February 2024 (thebezzle.org). "The Lost Cause:" a solarpunk novel of hope in the climate emergency, Tor Books (US), Head of Zeus (UK), November 2023 (http://lost-cause.org). "The Internet Con": A nonfiction book about interoperability and Big Tech (Verso) September 2023 (http://seizethemeansofcomputation.org). Signed copies at Book Soup (https://www.booksoup.com/book/9781804291245). "Red Team Blues": "A grabby, compulsive thriller that will leave you knowing more about how the world works than you did before." Tor Books http://redteamblues.com. "Chokepoint Capitalism: How to Beat Big Tech, Tame Big Content, and Get Artists Paid, with Rebecca Giblin", on how to unrig the markets for creative labor, Beacon Press/Scribe 2022 https://chokepointcapitalism.com Upcoming books (permalink) "The Post-American Internet," a geopolitical sequel of sorts to Enshittification, Farrar, Straus and Giroux, 2027 "Unauthorized Bread": a middle-grades graphic novel adapted from my novella about refugees, toasters and DRM, FirstSecond, April 20, 2027 "Enshittification, Why Everything Suddenly Got Worse and What to Do About It" (the graphic novel), Firstsecond, 2027 "The Memex Method," Farrar, Straus, Giroux, 2027 Colophon (permalink) Today's top sources: Currently writing: “Once Is Enemy Action,” a science fiction novel about the origins of modern technofascism. Today's words: 527 (10843 total). "The Post-American Internet," a sequel to "Enshittification," about the better world the rest of us get to have now that Trump has torched America. Fourth draft completed. Submitted to editor. A Little Brother short story about DIY insulin PLANNING This work – excluding any serialized fiction – is licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net. https://creativecommons.org/licenses/by/4.0/ Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution. How to get Pluralistic: Blog (no ads, tracking, or data-collection): Pluralistic.net Newsletter (no ads, tracking, or data-collection): https://pluralistic.net/plura-list Mastodon (no ads, tracking, or data-collection): https://mamot.fr/@pluralistic Bluesky (no ads, possible tracking and data-collection): https://bsky.app/profile/doctorow.pluralistic.net Medium (no ads, paywalled): https://doctorow.medium.com/ Tumblr (mass-scale, unrestricted, third-party surveillance and advertising): https://mostlysignssomeportents.tumblr.com/tagged/pluralistic "When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla READ CAREFULLY: By reading this, you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ISSN: 3066-764X
Today's links Why businesses lie about AI: Humoring the boss all the way into bankruptcy. Hey look at this: Delights to delectate. Object permanence: Vinge x NYT; Syklarov x publishers; Human hair castles; Gingrich's bot army; Accessibility v Web DRM; David Byrne x WinXP; Furries don't fuck in fursuits; AI's pogo-stick grift. Upcoming appearances: Edinburgh, Sydney, Melbourne, Brighton, London, South Bend. Recent appearances: Where I've been. Latest books: You keep readin' em, I'll keep writin' 'em. Upcoming books: Like I said, I'll keep writin' 'em. Colophon: All the rest. Why businesses lie about AI (permalink) Neoclassical economics assumes rationality. The corollary of, "If you're so smart, why aren't you rich?" is "you're rich, so you must be very smart!" Thus it is that many people assume that if powerful, well-compensated CEOs insist that "AI is changing everything," well then, AI must be changing everything. But the evidence for this "changing everything" thesis is thin on the ground. Despite a global mania that has reduced the real, pressing need for digital sovereignty to the imaginary need to create "sovereign AI," no one can really articulate the case for "sovereign AI." If Donald Trump ordered Big Tech to turn off all of your country's chatbots tomorrow, nothing would change. Every one of your country's ministries and corporations would chug on with nary a hitch. Households, too, though perhaps a few of the younger members of those families would have to do their own homework again. (Contrast this with what would transpire if Trump directed his tech giants to switch off your country's Office 365 access, or to brick your Android and iOS phones, or to killswitch your John Deere tractors. Your country would effectively cease to exist. If "digital sovereignty" means anything, it means doing something about this urgent fact): https://pluralistic.net/2026/06/18/their-trillions-our-billions/#eyes-on-the-prize The world is full of people who insist that "AI is changing everything" but who – when pressed – have to admit that what they mean is that they're pretty sure that AI will change everything. Eventually. After we allow it to consume all the planet's energy, carbon, water and financial resources. Maybe. (They're pretty sure.) One person who's had a lot of opportunity to observe the shear between the stated business/AI situation and the real business AI situation is Nikhil Suresh from Hermit Tech, a consulting firm of "radically ethical data wizards" (that is, tech consultants). Suresh reports on his experience talking with hundreds of executives (and, more importantly, their subordinates) about what (if anything) AI is doing for business in an essay entitled "AI Mania Is Eviscerating Global Decisionmaking": https://hermit-tech.com/blog/ai-mania-is-eviscerating-global-decisionmaking Suresh has a good track record of writing trenchant, frank criticism of AI. You may know him from his 2024 essay, "I Will Fucking Piledrive You If You Mention AI Again": https://ludic.mataroa.blog/blog/i-will-fucking-piledrive-you-if-you-mention-ai-again/ Or possibly from his "Contra Ptacek's Terrible Article On AI," a stinging rebuttal to Thomas Ptacek's widely read "My AI Skeptic Friends Are All Nuts": https://ludic.mataroa.blog/blog/contra-ptaceks-terrible-article-on-ai/ While those are important pieces of critical AI realpolitik, none of them have the heft or urgency of "AI Mania Is Eviscerating Global Decisionmaking," whose thesis can be summed up with this passage from halfway through this 6,000-word article: [W]e’re facing a coordination problem around executives being honest around the AI gains they’ve witnessed – if they co-operate, they keep their jobs. If they defect, they will possibly be fired by their embarrassed peers (who have now been implicitly called liars, cowards, or incompetents) and then replaced with someone that will toe the line anyway. If they could all admit the truth at once there might be some hope, but there is no way to coordinate that event. In other words, corporate leadership is starting from the premise that AI has (or will) radically change the business, and they're working backwards from that premise to find the evidence to support this article of faith. In support of this thesis, Suresh cites "hundreds" of conversations with execs and employees who spoke to him on the condition that he would "file the serial numbers" off their stories. These, combined with his own experience consulting for large, multi-billion-dollar companies make it clear that "AI mania" is an absolutely justifiable label for the state of AI in corporate circles. Here are a few highlights from this morning's read – moments where I had to look away from my screen and read out a passage to my wife so that we could share a "holy shit" moment. A person worked for a division that "pivoted" to re-engineer its software to create interfaces that support AI agents. When it became apparent that only ten users had touched this expensive new technology, they "pivoted" again to support "agentic workflows." Why did they double down on AI agents after discovering such yawning market indifference for "agentic"? "Because every company has to do something agentic now." Suresh describes this as a literal religious mania. In the 500+ employee businesses Suresh studied, the only people who were promoted – or even spared from being fired – were people who professed "religious declarations of faith" about "the transformative power of AI." Employees who voiced honest, informed objections to AI in the workplace were passed over for promotions or targeted for layoffs. This has created a situation in which everyone – "boards, executives, employees, vendors, consultants" – has a strong incentive to lie about how much AI is delivering for their companies. Suresh says he's seen announcements from publicly traded companies about their AI triumphs that he knows for a fact never took place. Suresh says he's never seen a successful enterprise AI project: "Every single one – we have seen 0% success in a year and a half." Not one of their clients would face a business challenge if OpenAI went out of business tomorrow. The problem most companies struggle with is that they're "terminally bad at running software projects effectively." Adding AI to the mix doesn't solve this problem – it just adds a whole new range of ways that software deployment can fail. Chatbots don't help. The internally facing chatbot that's supposed to help employees figure out how to navigate the business sucks because it is only as good as its training date – the business's documentation of its own processes. Businesses suck at documenting their processes. Customer-facing chatbots also suck. They either can't solve your problem, or, when they seem to solve your problem, the "solution" goes nowhere. Suresh recounts his sole positive customer service chatbot experience: a Mitsubishi chatbot with a natural sounding, responsive voice politely took all the details of an automotive failure and promised him a callback. That callback never came, but Suresh is certain that Mitsubishi has logged this as a chatbot success story, even though the experience convinced him not to buy a Mitsubishi car. Suresh and his team at Hermit Tech now have a policy of not even asking about ongoing AI projects. They've learned that by the time an AI project has begun, no one will discuss it honestly until it reaches a crisis point. Suresh says he frequently encounters people who reflexively utter the AI catechism: "AI is changing everything." But when he presses these people for details, they admit that their organization "does not currently use LLMs for anything, and indeed, that they cannot name a single thing that has changed other than they get some use out of ChatGPT." This shear ("AI is changing everything"/"Well, OK, we're not using AI for anything") is so extreme that Suresh once met an exec who confessed to crafting an AI-centered AI strategy for a $2b/year business, even though that exec "had never even used ChatGPT or any AI tool in their life." Some people have privately admitted to Suresh that they've embraced AI in order to earn a career-boosting corporate reputation for "thought leadership." But many other people (especially nontechnical people) sincerely believe that AI is about to "change everything." As Suresh says, if you're in business with a liar, you might be able to reason with them in private – but you can't reason with a true believer. The true believers are in charge. Suresh points out that it would be very weird for the CEO of an engineering firm or a hospital to mandate "specific procedures or building techniques without explicit agreement from the professionals on staff." But when it comes to AI, business leaders will confidently demand that the skilled professionals who perform the business's core functions use AI, even if those professionals don't think it will help. As an aside: I remember the dotcom era, when the business press was full of articles about the conflict between CEOs and a new workforce that demanded the right to use the web on the job. Today, the business press is full of articles about the conflict between the workforce and CEOs who demand that they use AI. Suresh describes workers who feel they have to "AI wash" their work: "They just do the work, the same way they have for decades, and say Claude did it." To add verisimilitude to this sham, they write circular processes in which one chatbot prompts another, and then the process repeats itself in reverse, for the sole purpose of consuming AI tokens to score a high rank on corporate "token leaderboards." How to account for this wildly, expensively irrational corporate leadership? Suresh places the blame in the hypnotizing, mesmerizing power of the AI demo. For example: Hermit Tech is often engaged to set up a database product called Snowflake for its customers. Snowflake has a useless, expensive AI bolt-on called Cortex, that Snowflake itself describes as being 92% accurate under ideal circumstances (that is, at least 8% of the time, it will mislead you, perhaps very badly). Suresh describes sales meetings with execs who were lukewarm on the idea of retooling with Snowflake, but who were very interested in Cortex. Against their better judgment, Suresh and his team provided them with a Cortex demo, carefully explaining that this AI tool could not satisfy their requirements. Without fail, this resulted in the previously lukewarm customers insisting that they be allowed to purchase Cortex immediately. Sales prospects who'd been unmoved by a pitch for new technology that would result in millions in savings were hypnotized by demos of a product that was described as unsuitable and unreliable. To their credit, Hermit Tech refused to sell these customers Cortex, and stopped doing Cortex demos altogether. Suresh describes the experience of "the total 180°, that shift from ice-cold to red-hot buying frenzy" as "deeply unsettling." What's more, the Cortex demos that Suresh and co performed were, by his account, pretty uninspiring. The thing that these demos had going for them is that they showed AI actually doing something marginally useful, to execs who'd already spent millions on AI without having anything to show for their money. The spectacle of AI that does something galvanizes corporate leaders who feel like they're the only bosses who can't find a revolutionary use for AI in their businesses. This is the situation up and down the corporate org-chart. Suresh has a reader whose title is "Head of AI" at a billion-dollar firm who tells him "their job is totally fraudulent but it was the only promotion pathway remaining at the organisation." This exec is hardly alone. They're part of a cohort of executives at companies that have publicly announced "100x" productivity gains, but who confessed to Suresh that nothing of the sort has happened. Why did these companies make these claims? Because their customers were making the claims. How could you hope to sell to a company that had 100x'ed its productivity with AI unless you, too had 100x'ed your productivity? If, as a vendor, you walked into a boardroom and said that this wasn't a plausible claim, you'd be calling your sales prospect a liar, with real consequences: "getting enterprise contracts cancelled because you wanted to opine on something that doesn’t really matter to your organisation’s mission is a great way to get fired." With the state of the industry dominated by froth, lies and mutual destruction pacts, it's no wonder that companies are deploying "totally gameable metrics such as 'money spent on AI'" as a means of evaluating employees and divisions. Between true believers and people who must find ways to plausibly tout their AI usage, there is now a gigantic market for "AI solutions." At best these are just traditional tech consulting contracts, like migrating a database from Oracle to Snowflake, with some kind of ornamental AI usage around the edges so that the person who commissions the work can claim to be "procuring AI-enabled services" for the business. This isn't a harmless frippery: contracts are delayed and work is put off until the work can be made "sufficiently AI" to attain the minimum degree of buzzword compliance. Worse: every fake AI project that produces real results (because it's not really AI) adds credibility to the AI true believers, who view these projects as proof that AI can do anything, and therefore demand to know why everything isn't being done by AI. Suresh ends his essay with a long section on how to "navigate AI mania" – advice for how to smile and nod politely when you're confronted with AI bullshit, while steering clear of the worst consequences and avoiding needless fights. This looks like very sound advice for anyone in a corporate environment, but thankfully, that isn't me. Rather than summarize that advice, I want to reflect a little on two questions that Suresh's essay raises but doesn't answer. The first is why? Why are people in power such easy converts to this religious mania? I have my own theory. The most important discomfort that powerful people experience is having ego-shattering conflicts with subordinates who know how to do things they do not know how to do. The fact that you're "in charge" is hard to reconcile with the fact that the people you're nominally in charge of tell you that all your ideas are impossible, illegal, immoral, or lethal: https://pluralistic.net/2026/01/05/fisher-price-steering-wheel/#billionaire-solipsism Take that Cortex demo. Sure, Cortex is an expensive, unreliable way to address a Snowflake database. But (unlike Snowflake) Cortex is controlled via conversational, plain-language commands. With Cortex, a boss doesn't need to ask an underling to retrieve information from the company Snowflake system, an interaction that might come with unsolicited feedback about the technical or commercial incoherence of the boss's request. Cortex is the underling, except that unlike a human underling, Cortex never back-sasses you about your foolish questions. The fact that it grossly misleads you 8% of the time is a small price to pay for a life untroubled by uppity pismires who insist that your ideas be connected to base reality as they understand it. The other question Suresh implicitly raises is, "How can you reconcile the failure of AI in the enterprise with the individual claims of skilled technologists who insist that AI is helping them do great work?" The answer is that these AI users are "centaurs" – experienced workers who are assisted by automation on terms that they set for themselves: https://pluralistic.net/2025/09/11/vulgar-thatcherism/#there-is-an-alternative Thanks to their skill and experience, these workers possess discernment, the ability to tell good code from bad, and (more importantly) good uses of code-generation tools from bad. They demonstrate the adage that worker-driven automation improves quality, while capital-driven automation improves throughput: https://pluralistic.net/2026/07/28/hitl-ers/#ai-ai-oh An automation technique that requires close supervision by skilled and experienced workers isn't going to be a raw productivity powerhouse. You don't "100x" your code this way, at least, not in the sense of firing 99 of your coders and having the remaining programmer pick up all their work. Rather, an automation tool that requires the continuous and conscientious exercise of discernment will let individual practitioners improve their work in extremely satisfying and useful ways. It's a way to spend more on operations in order to produce better outputs. It's not a way to cut your workforce, realize a gigantic savings, and still produce comparable goods and services at a far lower cost. That is why some individual coders report such delight with their AI tools. They engage with those tools on their own terms, to improve their work in the ways that they, in their expert judgment, consider beneficial. No one ranks them on a "token-maximization" scoreboard. No one tells them they can't do a project if it isn't "sufficiently AI." When they set out to do a project, no one makes them prove that it couldn't be "done by AI." As ever, the most important fact about a given technology isn't "what it does," but "who it does it for" and "who it does it to." All the pathologies Suresh observes and documents so well in this piece are hypertrophied versions of the buzzword-compliance dysfunctions from previous bubbles, but at a scale never before seen. Quantity has a quality all its own. These businesses aren't just wasting billions – they're replacing skilled workers with defective chatbots. As I've written before, AI is the asbestos we're shoveling into the walls of our technological society. Our descendants will spend generations digging it out again, and the longer the bubble goes on without popping, the longer it will take to repair the damage. Hey look at this (permalink) The rent was already high. Then came the $200 work-from-home fee https://finance.yahoo.com/real-estate/articles/rent-already-high-then-came-174555709.html Families in London temporary housing told they cannot use in-built air conditioning https://www.theguardian.com/society/2026/jul/27/homeless-families-london-temporary-housing-air-conditioning The New Defcon Badges Pack a Unique Open Source Chip That Doubles as a Security Key https://www.wired.com/story/defcon-34-badge-baochip-andrew-bunnie-huang/ US government map of Africa mislabels every country at global conference https://www.theguardian.com/us-news/2026/jul/30/government-map-mislabels-african-countries?CMP=Share_AndroidApp_Other EFF Guide to Recording Law Enforcement https://www.eff.org/deeplinks/2026/07/eff-guide-recording-law-enforcement Object permanence (permalink) #25yrsago Vernor Vinge in the NYT https://www.nytimes.com/2001/08/02/technology/a-scientist-s-art-computer-fiction.html #25yrsago Why publishers should thank Syklarov https://web.archive.org/web/20011023092940/http://www.zdnet.com/zdnn/stories/comment/0,5859,2800985,00.html #25yrsago David Byrne track to be bundled with WinXP https://web.archive.org/web/20010804040357/http://www.ananova.com/news/story/sm_365899.html?menu=news.technology #20yrsago Five things about blogs that no one ever needs to say again https://web.archive.org/web/20060813090449/http://www.stevenberlinjohnson.com/2006/08/five_things_all.html #15yrsago Castles made from human hair https://inhabitat.com/artist-uses-human-hair-to-construct-a-castle-of-3000-bricks/ #15yrsago Wisconsin Democratic voters targeted with Koch-funded absentee ballot notices advising them to vote 2 days after the recall election https://www.politico.com/blogs/david-catanese/2011/08/afp-wisconsin-ballots-have-late-return-date-037977?showall #15yrsago Gingrich’s million Twitter followers: “80% dummy accounts, 10% paid followers” https://web.archive.org/web/20110812100159/https://gawker.com/5826645/most-of-newt-gingrichs-twitter-followers-are-fake #15yrsago Missouri State business-school professor leads successful campaign to ban Slaughterhouse-Five from local schools https://www.theguardian.com/books/2011/jul/29/slaughterhouse-five-banned-us-school #10yrsago Australian media accessibility group raises red flag about DRM in web standards https://hotelsantalya.net/accessiq/news/news/2016-p/08-p/concerns-raised-for-assistive-technology-development-as-w3c-debates-encrypted/ #10yrsago Reminder: the GOP has been attacking veterans and their families for years https://web.archive.org/web/20160803203106/https://crookedtimber.org/2016/08/02/trumps-indecent-proposal/ #10yrsago Isis joins Donald Trump in denouncing Khizr Khan https://web.archive.org/web/20160802161454/https://theintercept.com/2016/08/02/donald-trump-and-islamic-state-agree-no-room-for-people-like-khizr-khan/ #10yrsago Furries don’t have sex in fursuits https://www.ohjoysextoy.com/fursuits-grey-white/ #5yrsago Machine learning sucks at covid https://pluralistic.net/2021/08/02/autoquack/#gigo #1yrago AI's pogo-stick grift https://pluralistic.net/2025/08/02/inventing-the-pedestrian/#three-apis-in-a-trenchcoat Upcoming appearances (permalink) Virtual: EFFecting Change: Who the Machine Serves, Aug 12 https://www.eff.org/event/effecting-change-who-machine-serves Edinburgh International Book Festival with Jimmy Wales, Aug 17 https://www.edbookfest.co.uk/events/the-front-list-cory-doctorow-and-jimmy-wales Sydney: The Festival of Dangerous Ideas, Aug 23-24 https://festivalofdangerousideas.com/program/ Melbourne: Enshittification at the Wheeler Centre, Aug 25 https://www.wheelercentre.com/events-tickets/season-2026/cory-doctorow-enshittification Brighton: The Reverse Centaur's Guide to Life After AI with Carole Cadwalladr (Brighton Dome), Sep 8 https://brightondome.org/whats-on/LSC-cory-doctorow-the-reverse-centaurs-guide-to-life-after-ai/ London: The Reverse Centaur's Guide to Life After AI with Riley Quinn (Foyle's Picadilly), Sep 9 https://www.foyles.co.uk/events/enshittification-cory-doctorow-riley-quinn South Bend: An Evening With Cory Doctorow (Notre Dame), Oct 6 https://franco.nd.edu/events/2026/10/06/an-evening-with-cory-doctorow/ Vancouver: BC Policy Solutions Gala, Nov 12 https://bcpolicy.ca/gala/ Recent appearances (permalink) F@#$ the AI Overlords (On The Media) https://www.wnycstudios.org/podcasts/otm/articles/f-the-ai-overlords Why AI Won't Replace Workers, But Will Crash The Economy (Smart Cookies) https://www.youtube.com/watch?v=rRRmUuxJolY AI and the Enshittification Era (The Weekly Show with Jon Stewart) https://www.youtube.com/watch?v=-dAIJRjb-Bw AI is not inevitable (Betakit) https://www.youtube.com/watch?v=DbiTVkq1WHo A Conversation with Lina Khan (Law and Economy Student Network) https://www.youtube.com/live/7Ak5LZllqwE Latest books (permalink) "The Reverse-Centaur's Guide to AI," a short book about being a better AI critic, Farrar, Straus and Giroux, June 2026 https://us.macmillan.com/books/9780374621568/thereversecentaursguidetolifeafterai/ "Canny Valley": A limited edition collection of the collages I create for Pluralistic, self-published, September 2025 https://pluralistic.net/2025/09/04/illustrious/#chairman-bruce "Enshittification: Why Everything Suddenly Got Worse and What to Do About It," Farrar, Straus, Giroux, October 7 2025 https://us.macmillan.com/books/9780374619329/enshittification/ "Picks and Shovels": a sequel to "Red Team Blues," about the heroic era of the PC, Tor Books (US), Head of Zeus (UK), February 2025 (https://us.macmillan.com/books/9781250865908/picksandshovels). "The Bezzle": a sequel to "Red Team Blues," about prison-tech and other grifts, Tor Books (US), Head of Zeus (UK), February 2024 (thebezzle.org). "The Lost Cause:" a solarpunk novel of hope in the climate emergency, Tor Books (US), Head of Zeus (UK), November 2023 (http://lost-cause.org). "The Internet Con": A nonfiction book about interoperability and Big Tech (Verso) September 2023 (http://seizethemeansofcomputation.org). Signed copies at Book Soup (https://www.booksoup.com/book/9781804291245). "Red Team Blues": "A grabby, compulsive thriller that will leave you knowing more about how the world works than you did before." Tor Books http://redteamblues.com. "Chokepoint Capitalism: How to Beat Big Tech, Tame Big Content, and Get Artists Paid, with Rebecca Giblin", on how to unrig the markets for creative labor, Beacon Press/Scribe 2022 https://chokepointcapitalism.com Upcoming books (permalink) "The Post-American Internet," a geopolitical sequel of sorts to Enshittification, Farrar, Straus and Giroux, 2027 "Unauthorized Bread": a middle-grades graphic novel adapted from my novella about refugees, toasters and DRM, FirstSecond, April 20, 2027 "Enshittification, Why Everything Suddenly Got Worse and What to Do About It" (the graphic novel), Firstsecond, 2027 "The Memex Method," Farrar, Straus, Giroux, 2027 Colophon (permalink) Today's top sources: Currently writing: "The Post-American Internet," a sequel to "Enshittification," about the better world the rest of us get to have now that Trump has torched America. Fourth draft completed. Submitted to editor. A Little Brother short story about DIY insulin PLANNING This work – excluding any serialized fiction – is licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net. https://creativecommons.org/licenses/by/4.0/ Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution. How to get Pluralistic: Blog (no ads, tracking, or data-collection): Pluralistic.net Newsletter (no ads, tracking, or data-collection): https://pluralistic.net/plura-list Mastodon (no ads, tracking, or data-collection): https://mamot.fr/@pluralistic Bluesky (no ads, possible tracking and data-collection): https://bsky.app/profile/doctorow.pluralistic.net Medium (no ads, paywalled): https://doctorow.medium.com/ Tumblr (mass-scale, unrestricted, third-party surveillance and advertising): https://mostlysignssomeportents.tumblr.com/tagged/pluralistic "When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla READ CAREFULLY: By reading this, you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ISSN: 3066-764X
Today's links Technocarcinization: Enshittification is the great leveler. Hey look at this: Delights to delectate. Object permanence: Grampa's backyard Disneyland; Elizabeth Warren on monopolies; Spotify v Apple (antitrust edn); Exxon lobbyist confesses; "When the Sparrow Falls." Upcoming appearances: London, Edinburgh, Sydney, Melbourne, Brighton, London, South Bend. Recent appearances: Where I've been. Latest books: You keep readin' em, I'll keep writin' 'em. Upcoming books: Like I said, I'll keep writin' 'em. Colophon: All the rest. Technocarcinization (permalink) "Carcinization" is a curious biological phenomenon: given enough time, across many environments, many species will evolve into crabs. The body-type of a crab, with its low center of gravity, sideways gait (useful for evading predators), ease of concealment and protected organs is suitable to many different environments: https://en.wikipedia.org/wiki/Carcinisation Lately, I've watched the American Big Tech platforms as they underwent their own form of technocarcinization, which is when every tech company turns into Facebook. For a long time, it seemed to me that you could make sense of the tech platforms by placing them into one of four quadrants on a 2×2 grid, in which one axis denoted "control freakishness" and the other, "surveillance." Each quadrant had its own canonical company. The most surveillant/least controlling company (top left) was Google. They would let you roam the whole wide internet and exert no control over your conduct, but would spy on you wherever you went. The least surveillant/most controlling company was Apple, who imprisoned you in its manicured walled garden, but promised never to spy on you. The non-spying/non-controlling option is free/open source tech (of course), which doesn't care what you do, and doesn't watch you do it. And the most spying, most controlling company was Facebook, a company whose products did everything they could to imprison you within their virtual walls, from which vantage they could effect maximal surveillance. I've used this comparison many times over the years. I included in my 2023 book The Internet Con, along with the joke that Tiktok's position on the grid was so far up and to the right (maximum surveillance and control) that we'd had to put its logo on the back cover. Enough people took this joke seriously and wrote in to complain that they'd gotten a misprint without the logo that we added it to the paperback: https://www.versobooks.com/products/3035-the-internet-con The grid was useful, until technocarcinization started to push all the tech companies into that top right quadrant. Apple is no longer the company that protects you from surveillance – they're the company that spies on you, having secretly added a total surveillance system to the iPhone to target ads to you: https://pluralistic.net/2022/11/14/luxury-surveillance/#liar-liar Apple can't even claim to protect you from third-party surveillance. Sure, they block Facebook from spying on you, but they have barred ICE Block, an app that tells you if there are ICE chuds hunting in your neighborhood, looking to kidnap you and send you to a concentration camp. Apple declared ICE mercenaries to be a "protected class": https://pluralistic.net/2025/10/06/rogue-capitalism/#orphaned-syrian-refugees-need-not-apply And thanks to Apple's control-freakery – which prevents you from overriding Apple's decisions about your own devices – once Apple decides to spy on you or sell you out to fascist goons, there's nothing you can do about it: https://locusmag.com/feature/cory-doctorow-neofeudalism-and-the-digital-manor/ Then there's Google, the company that ran a free-range livestock operation in which you could roam wherever you liked, because they could always find you when it was time for the slaughter. For years now, Google has been moving inexorably to the kind of control-freak nonsense that you used to only find in one of Apple's crystal prisons. For example, every year or two, Google floats a proposal to use secure hardware in your device to rat you out if you've got an ad-blocker, privacy blocker, or other aftermarket add-on that lets you choose how you experience the digital world: https://pluralistic.net/2023/08/02/self-incrimination/#wei-bai-bai It's an idea they just can't quit, despite the fact that it's fucking abominable and everyone hates it: https://pluralistic.net/2026/06/12/compelled-speech/#quishing Google used to pride itself in its ability to send you to the open web, viewing search as a conduit to other peoples' resources. Now, with AI search summaries, Google is harvesting the open web and then eating the seed corn, keeping searchers inside of Google's walled garden: https://pluralistic.net/2026/06/29/arsonist-firefighters/#im-feeling-lucky Google also took the idea of a free/open browser and ran with it, rehabilitating some discarded Apple code and turning it into Chrome, the internet's most dominant browser – by far. Now, Google is nerfing that browser's plug-in architecture in a way that blocks all kids of user-tunable options, including and especially ad-blocking: https://protonprivacy.substack.com/p/google-is-finally-killing-ublock And Google has also announced that they're going to turn Android into an iPhone, making it both technically challenging and radioactively illegal for you to install software of your choosing on your own property: https://arstechnica.com/gadgets/2025/08/google-will-block-sideloading-of-unverified-android-apps-starting-next-year/ Google is adopting every one of Apple's worst practices, and Apple is adopting all of Google's worst practices, and so they're both turning into Facebook: technocarcinization! What's driving this technocarcinization? Well, the obvious answer is that the more Facebooklike a company becomes, the more ways there are for it to rip you off. Surveillance can be monetized by selling your data, by ad targeting, and by surveillance-based pricing and wage-suppression: https://pluralistic.net/2026/01/21/cod-marxism/#wannamaker-slain Control lets platforms block competing products, extract massive junk fees to the businesses they connect you to, and control repair and end-of-life, forcing you to replace hardware by blocking parts and independent service: https://pluralistic.net/2026/01/10/markets-are-regulations/#carney-found-a-spine It turns out that "if you're not paying for the product, you're the product" is only half-right. The other half is, "even if you pay for the product, you're the product." Pay, don't pay: companies will productize anyone they can. And thanks to our enshittogenic policy environment – where the worst ideas of the worst people make the most money – you can always be productized: https://pluralistic.net/2025/09/10/say-their-names/#object-permanence This is independent of the kind of person running the company. Facebook is run by Mark Zuckerberg, a cringe halfwit whose only successful idea was to offer Harvard bros a way of nonconsensually rating the fuckability of female undergrads. Everything he's done since was an acquisition (Whatsapp, Insta) or a flop (metaverse, Libra), or both (Oculus). Zuck owns the majority of the voting stock in the company, which means he has total control over its actions. He can ignore or fire his board members at will. He is the move fast/break things guy, whose every foolish whim can become policy that impacts billions of people. By contrast, Google and Apple are no longer run by their flamboyant founders, who were every bit as prone to folly as Zuck. They were constrained by their shareholders, which meant that the blast-radius of Steve Jobs's worst ideas (like treating his otherwise curable cancer with green juice) were confined to his own person. Today, Apple and Google are run by bloodless business sociopaths who go to enormous lengths to project an air of sober adulthood. And yet, these people – who would never be caught dead bow-hunting their own livestock or climbing into an MMA cage – have steered their companies into Facebook's quadrant on our enshittification 2×2. I think this shows just how much the enshittification of tech is a matter of the policy environment, not the personalities of the people involved. Sure, the worst people imaginable run these companies, but the reason they're able to yield to their most venal impulses and succeed is because the world has been re-arranged to make sociopathy and greed into fitness factors. We get technocarcinization because the most fit organism for a landscape without consequences is a zuckerbergian techno-crab: https://pluralistic.net/2023/07/28/microincentives-and-enshittification/ What can we do about it? Well, we're going to have to remake the landscape to punish (rather than reward) enshittification: https://pluralistic.net/2026/01/01/39c3/#the-new-coalition And in the meantime, there is one inhabitant of the 2×2 that hasn't drifted up and to the right: free and open source software. It's still snugly nestled in the low-surveillance/low-control box, and if you live in that box, your life will be much, much better for it. There's no better time to make the switch: with RAM and storage prices through the ceiling and OSes growing ever-more bloated with AI and spyware (but I repeat myself), this is the moment to rehabilitate that old computer with Linux: https://www.fosslinux.com/158206/linux-on-older-hardware-revival-guide.htm The alternative is to be tormented by crabs no matter what you're trying to do or where you're trying to get to. Hey look at this (permalink) How the AI bubble could pop and take down the global economy, according to the BIS https://www.theregister.com/ai-and-ml/2026/06/29/how-the-ai-bubble-could-pop-and-take-down-the-global-economy-according-to-the-bis/5263793 To Decarbonize Quickly, Think Beyond Electrification https://jacobin.com/2026/06/climate-electrification-homes-cars-decarbonization-tech Ireland is big tech’s lapdog – and that compromises its EU presidency https://www.theguardian.com/commentisfree/2026/jun/30/ireland-big-tech-lapdog-eu-presidency-digital-sovereignty Beyond Denial How Oil Execs Shaped a Landmark Climate Study https://www.propublica.org/article/wedges-climate-research-bp-fossil-fuel-princeton US Supreme Court just blew up EU-US Data Transfers https://noyb.eu/en/us-supreme-court-just-blew-eu-us-data-transfers Object permanence (permalink) #15yrsago Print-on-demand and donations - report on DIY publishing business models https://www.publishersweekly.com/pw/by-topic/columns-and-blogs/cory-doctorow/article/47858-with-a-little-help-heuristics.html #15yrsago Brazil rises up for free speech in 40 national demonstrations https://globalvoices.org/2011/06/30/brazil-freedom-march/ #10yrsago Grandad builds miniature backyard Disneyland https://abcnews.com/Lifestyle/grandpa-builds-disneyland-inspired-backyard-theme-park-grandkids/story?id=40276633 #10yrsago Elizabeth Warren on monopolies in America, including Apple, Google, and Amazon https://washingtonmonthly.com/2016/06/30/elizabeth-warrens-consolidation-speech-could-change-the-election/ #10yrsago White House plan to use data to shrink prison populations could be a racist dumpster fire https://www.wired.com/2016/06/white-house-mission-shrink-us-prisons-data/ #10yrsago Even if Moore's Law is "running out," there's still plenty of room at the bottom https://www.technologyreview.com/2016/05/13/245938/moores-law-is-dead-now-what/ #10yrsago Black-hat hacker handles are often advertisements https://www.wired.com/beyond-the-beyond/2016/07/web-semantics-modern-german-black-hat-hacker-handles/ #10yrsago Spotify threatens to report Apple to competition regulators over App Store rejection https://web.archive.org/web/20160630220301/https://www.recode.net/2016/6/30/12067578/spotify-apple-app-store-rejection #10yrsago Researchers find over 100 spying Tor nodes that attempt to compromise darknet sites https://www.defcon.org/html/defcon-24/dc-24-speakers.html#Noubir #5yrsago Exxon lobbyist confesses to his crimes https://pluralistic.net/2021/07/01/basilisk-tamers/#exxonknew #5yrsago When the Sparrow Falls https://pluralistic.net/2021/07/01/basilisk-tamers/#rage-against-the-machine Upcoming appearances (permalink) London: Idler Festival, Jul 11 https://www.idler.co.uk/festival/ Edinburgh International Book Festival with Jimmy Wales, Aug 17 https://www.edbookfest.co.uk/events/the-front-list-cory-doctorow-and-jimmy-wales Sydney: The Festival of Dangerous Ideas, Aug 23-24 https://festivalofdangerousideas.com/cory-doctorow/ Melbourne: Enshittification at the Wheeler Centre, Aug 25 https://www.wheelercentre.com/events-tickets/season-2026/cory-doctorow-enshittification Brighton: The Reverse Centaur's Guide to Life After AI with Carole Cadwalladr (Brighton Dome), Sep 8 https://brightondome.org/whats-on/LSC-cory-doctorow-the-reverse-centaurs-guide-to-life-after-ai/ London: The Reverse Centaur's Guide to Life After AI with Riley Quinn (Foyle's Picadilly), Sep 9 https://www.foyles.co.uk/events/enshittification-cory-doctorow-riley-quinn South Bend: An Evening With Cory Doctorow (Notre Dame), Oct 6 https://franco.nd.edu/events/2026/10/06/an-evening-with-cory-doctorow/ Recent appearances (permalink) Lawfare Daily https://www.youtube.com/watch?v=T1KIwaYRs1g How to Think About AI (Organized Money) https://www.organizedmoney.fm/p/how-to-think-about-ai-with-cory-doctorow Breaking Points https://www.youtube.com/watch?v=VJmUbkRqXeE A.I. Enshittifies Everything (Slate) https://slate.com/podcasts/what-next-tbd/2026/06/cory-doctorow-thinks-a-i-is-overvalued-and-overrated-and-still-a-threat A World That Just Might Work https://aworldthatjustmightwork.com/2026/06/cory-doctorow-ai-use-it-dont-buy-the-hype-dont-feed-the-bubble/ Latest books (permalink) "The Reverse-Centaur's Guide to AI," a short book about being a better AI critic, Farrar, Straus and Giroux, June 2026 https://us.macmillan.com/books/9780374621568/thereversecentaursguidetolifeafterai/ "Canny Valley": A limited edition collection of the collages I create for Pluralistic, self-published, September 2025 https://pluralistic.net/2025/09/04/illustrious/#chairman-bruce "Enshittification: Why Everything Suddenly Got Worse and What to Do About It," Farrar, Straus, Giroux, October 7 2025 https://us.macmillan.com/books/9780374619329/enshittification/ "Picks and Shovels": a sequel to "Red Team Blues," about the heroic era of the PC, Tor Books (US), Head of Zeus (UK), February 2025 (https://us.macmillan.com/books/9781250865908/picksandshovels). "The Bezzle": a sequel to "Red Team Blues," about prison-tech and other grifts, Tor Books (US), Head of Zeus (UK), February 2024 (thebezzle.org). "The Lost Cause:" a solarpunk novel of hope in the climate emergency, Tor Books (US), Head of Zeus (UK), November 2023 (http://lost-cause.org). "The Internet Con": A nonfiction book about interoperability and Big Tech (Verso) September 2023 (http://seizethemeansofcomputation.org). Signed copies at Book Soup (https://www.booksoup.com/book/9781804291245). "Red Team Blues": "A grabby, compulsive thriller that will leave you knowing more about how the world works than you did before." Tor Books http://redteamblues.com. "Chokepoint Capitalism: How to Beat Big Tech, Tame Big Content, and Get Artists Paid, with Rebecca Giblin", on how to unrig the markets for creative labor, Beacon Press/Scribe 2022 https://chokepointcapitalism.com Upcoming books (permalink) "The Post-American Internet," a geopolitical sequel of sorts to Enshittification, Farrar, Straus and Giroux, 2027 "Unauthorized Bread": a middle-grades graphic novel adapted from my novella about refugees, toasters and DRM, FirstSecond, April 20, 2027 "Enshittification, Why Everything Suddenly Got Worse and What to Do About It" (the graphic novel), Firstsecond, 2027 "The Memex Method," Farrar, Straus, Giroux, 2027 Colophon (permalink) Today's top sources: Currently writing: "The Post-American Internet," a sequel to "Enshittification," about the better world the rest of us get to have now that Trump has torched America. Fourth draft completed. Submitted to editor. A Little Brother short story about DIY insulin PLANNING This work – excluding any serialized fiction – is licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net. https://creativecommons.org/licenses/by/4.0/ Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution. How to get Pluralistic: Blog (no ads, tracking, or data-collection): Pluralistic.net Newsletter (no ads, tracking, or data-collection): https://pluralistic.net/plura-list Mastodon (no ads, tracking, or data-collection): https://mamot.fr/@pluralistic Bluesky (no ads, possible tracking and data-collection): https://bsky.app/profile/doctorow.pluralistic.net Medium (no ads, paywalled): https://doctorow.medium.com/ Tumblr (mass-scale, unrestricted, third-party surveillance and advertising): https://mostlysignssomeportents.tumblr.com/tagged/pluralistic "When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla READ CAREFULLY: By reading this, you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ISSN: 3066-764X
Today's links Molly Crabapple's 'Here Where We Live Is Our Country': An essential book for this moment and for the moments that led to it. Hey look at this: Delights to delectate. Object permanence: Home chemistry sets in danger; Every pirate wants to be an admiral; Painful computer workarounds; JPEG patent invalidated; UBS whistleblower v USA (x USA); David Foster Wallace x tennis; Who cares about "bandwidth hogs?" Upcoming appearances: London, Kansas City, LA, Menlo Park, Toronto, NYC, Edinburgh, South Bend. Recent appearances: Where I've been. Latest books: You keep readin' em, I'll keep writin' 'em. Upcoming books: Like I said, I'll keep writin' 'em. Colophon: All the rest. Molly Crabapple's 'Here Where We Live Is Our Country' (permalink) Molly Crabapple's Here Where We Live Is Our Country is one of the most important, timely and salient works of history I've ever read. It's a history of the Jewish Labor Bund, a socialist, internationalist organization that once dominated Jewish political identity: https://www.penguinrandomhouse.com/books/646320/here-where-we-live-is-our-country-by-molly-crabapple/ In the late 19th and early 20th centuries, there were hundreds of thousands of Bund members, both in the Pale of Settlement (the rural regions of the Russian empire that the Tsar confined most Jews to) and in diasporic centers like New York City. The Bund played an important role in the Russian Revolution and in the resistance to the rise of European fascism, and fought valiantly in the antifascist underground guerrilla bands in Nazi-occupied territories. Despite this faded prominence, the Bund is all but unknown today. I was only vaguely aware of it, even though I attended seven years' worth of Yiddish classes at the Workmen's Circle, a Bund-originated socialist fraternal organization, and was bar-mitzvahed at a Workmen's Circle hall. It wasn't until I read about the Bund in Naomi Klein's essential 2023 book Doppelganger that I first caught a glimmer of its significance: https://pluralistic.net/2023/09/05/not-that-naomi/#if-the-naomi-be-klein-youre-doing-just-fine The thesis of Doppelganger is that the world is full of "mirror world" pairs with opposite political valences. For example, the mirror world version of the health justice movement is MAHA. Both MAHA and health justice share many commonalities (such as a skepticism of Big Pharma and its captured regulators), but arrive at totally different conclusions. Health justice demands universal access to medical care, compulsory licenses and patent reform for life-saving medicines, and systemic interventions to address discrimination against gender minorities, women, and racialized people. MAHA starts from the same diagnosis, but arrives at a totally different prescription: "eating clean," buying unregulated supplements from grifters, rejecting vaccines, attributing chronic health problems to personal moral failings, along with a conspiratorial rejection of life-saving medication. Mirror worlds are everywhere. One chapter of Klein's work deals with the "mirror worlds" of Jewish identity and what radical Jews once called "the Jewish question": https://ernestmandel.org/english/works/Jewish-Question-Since-World-War-II In the 19th century, antisemitism was often described as "the socialism of fools." In the real world, we observe the dominance of parasitic finance capital over productive labor and embark upon a great class struggle to seize the means of production. In the mirror world, antisemites observe this same fact, combine it with the fact that some of these bankers are Jewish, and embark on a genocidal program of antisemitic violence. But antisemites weren't the only mirror-world pairing with a view on "the Jewish question." Early 20th century Jews also lived on either side of the political looking-glass. On one side, you had the Bundists, whose motto (and the title of Crabapple's book) was "Here, where we live, is our country." For Bundists, Jews belonged everywhere Jews were. As the Jewish socialist Meyer London wrote, "Thousands of Jewish boys and girls pray to God not to lead them again out of Egypt, but to help them free Egypt." The Bund saw its struggle as just one aspect of the universal struggle for liberation. They understood that persecuted minorities everywhere labored under the double bind of racist and class oppression (and further, that women labored under gender oppression), but they also understood that these identity markers were tactical facts about how these workers should set about freeing themselves. They didn't mistake identity for a strategic difference: the goal was always universal liberation, and the reason to consider identity-based oppression was to ensure that every comrade was brought along in the struggle. As Crabapple writes, the Bund more-or-less invented intersectional analysis, and they practiced it with an eye to all the struggles of the world. Bund newspapers (even those published by the Bund underground in the Warsaw Ghetto) closely tracked the struggles of Black workers in the Jim Crow south, just as the Black radical press of the day reported closely on antisemitic lynchings in Europe. The Bund underground even managed to send telegrams of support to Gandhi from Nazi-occupied Poland. On the other side of the Jewish mirror was (of course) Zionism. Zionism and the Bund were founded in the same year, in response to the same events. The Bund was founded in secret by exiled radical Jews in Vilna whom the Tsar had banished for their resistance activities. Zionism was founded in Geneva by Theodor Herzl, who sheltered Jews who had fled Tsarist Russia to escape antisemitic violence. Where the Bund called for universalism and solidarity with all workers to keep Jews safe in every place where Jews lived, Zionists dreamed of a Jewish homeland, a stronghold to which Jews could retreat from the world. Where the Bund fought antisemites who would banish or exterminate Jews, Zionist leaders were willing to align themselves with antisemites, finding common cause in the idea that European Jewry should abandon Europe in favor of Palestine. Indeed, the Balfour Declaration – which established a plan for the UK handing over its occupied territories in Palestine to create a Jewish homeland – was fomented by vicious antisemites as part of a plan to ethnically cleanse the UK of all Jews: https://www.palestine-studies.org/en/node/232119 As Crabapple documents in detail, in the ensuing decades of struggle that followed, Zionist leaders repeatedly entered into alliances with antisemitic politicians, even those who presided over (and sometimes directed) campaigns of racist terror against Jews. Despite their mutual hatred, they shared a common goal: terrorizing Europe's Jews out of Europe and into Palestine. Meanwhile, Bundists never wavered from their rejection of antisemites. In the Bundists' socialist, internationalist program, the pursuit of a Jewish homeland merely dangled the possibility of Jewish liberation – at the expense of Palestinians, and without having anything to offer to all the other oppressed peoples of the world. While I discovered the Bund through reading Naomi Klein, many others learned about it from Crabapple's widely circulated 2018 New York Review of Books article, "My Great-Grandfather the Bundist": https://archive.is/20260518010455/https://www.nybooks.com/online/2018/10/06/my-great-grandfather-the-bundist/ Predictably, Crabapple's article provoked attacks from Zionists who told Crabapple they blamed the Bund for its own extermination. In their telling, the Bund's stubborn refusal to confront antisemitism as "history's oldest hatred" was a suicidal delusion that led their members into the Nazis' mass graves. But for many Jews, Crabapple's article was a revelation about a different way to be Jewish, an identity that rejected the Apartheid state of Israel (South African Apartheid and the state of Israel share a birth year, and Apartheid South Africa and Israel carried on a robust program of mutual trade in arms and surveillance tools): https://imeu.org/resources/key-issues/fact-sheet-an-overview-apartheid-south-africa-israel/275 This revelation only gained salience and prominence after October 7, 2023, when Israel responded to a massacre perpetrated by Hamas by embarking on a years-long program of genocide and extraterritorial aggression. Zionists have defended these crimes against humanity as inseparable from Jewish identity and the only plausible answer to "the Jewish question." Israel's defenders insist that even naming the genocide in Palestine (let alone opposing it) is inherently antisemitic. Ironically, Israel's loudest cheerleaders are the millions antisemitic evangelical Christian Zionists who vastly outnumber Jewish Zionists, who support Israel in hopes of bringing about a Biblical prophecy in which Christ returns and every Jew is cast down to Hell. In the years since, Crabapple's work to revive the Bund has only gained adherents, especially among Jews who refuse to accept that their safety can only be secured through mass slaughter and imperial conquest. Crabapple's response to this burgeoning movement is this book, a massive, heroic, brilliant, and pitiless history of the Bund that proposes its own answer to "the Jewish question." Beyond its political importance, Here Where We Live Is Our Country is a remarkable scholarly and artistic achievement. Crabapple taught herself to speak and read Yiddish so that she could consume primary sources, and she crisscrossed the globe to see and research the key sites of Jewish oppression and the Jewish liberation struggle. It's a monumental book. Thanks to Crabapple's voluminous research, Here Where We Live delivers a blow-by-blow look at the Bund's rise and its triumphs, but even more importantly, the tactical disagreements, factional disputes, and personal animus that too often snatched defeat from the jaws of victory for these committed revolutionaries. At times, Crabapple's tick-tock of these fights seems to embody the wry maxim: "Two Jews, three arguments." But the point of all this nuanced, textured detail isn't to rehash the tittle-tattle of the previous century, nor is it to show off Crabapple's prowess as a researcher. Rather, in rehearsing these fights, Crabapple shows how reasonable these disputes seemed at the time, and how terrible the consequences were for all concerned. In this mode, Crabapple manages the admirable achievement of being both sympathetic and pitiless. Crabapple, after all, is a veteran political activist who has traveled extensively to active war-zones to document atrocities and offer mutual aid to those fighting for justice. She's endured every failure that radical politics can manifest, sat through every kind of bad meeting, and she recognizes in these disputes the same personalities and personal failings that have broken her heart a hundred times. She understands why these people are this way – but she can also see, with perfect hindsight, the ghastly horrors that followed, which swamp any matter of principle these people might have stood on. There's plenty of this sympathetic pitilessness to go around, and it's not just the Bund or Jews who come in for it. Every factionalist blunder in pre-Revolutionary Russia, in the Soviet Union, in interwar Poland, and in occupied Poland comes in for examination – as do every imprisonment, maiming, rape and death that these blunders opened the door to. Crabapple's heroes are principled, but they are imperfect, and sometimes foolish, and sometimes self-deluding (for example, the Palestinian leader who insists that his rank-and-file fighters want to establish a multi-ethnic democracy, despite the undeniable presence in their number of people who want to banish all Jews from Palestine). The twentieth century was a charnel house, and so the cost of these mistakes is high. Often, these mistakes lead to mass graves, with these mistake-makers tangled among the bodies. They never had the chance to learn from their mistakes. But, through Crabapple's work, we might. It is in the postscript to this book that its true message lands. After 480 pages, we arrive at Crabapple's conclusion. In reflecting on these people, who died in their millions and whose memory was all but erased, she asks, "Did the Bund fail?" Her answer is a resounding no. The Bund lost, but it did not fail. The Bund was failed, as were the Zionists, the Roma, European socialists, disabled and queer people – everyone the Nazis burned, gassed, or buried alive. These people cried out to the rest of the world – to America, to Canada, to the UK, to all the places that were not under Nazi occupation – and begged for help, for safe passage, for rescue. The world slammed its doors. Even after they joined the war, they refused to admit Jews and other victims of Nazi genocide. They refused visas, closed borders, turned back boats of escapees, sometimes sending them back to occupied Europe to be slaughtered. In his review in the New York Review of Books, historian Adam Hochschild writes: Imagine that the United States had not passed the Immigration Act of 1924, which essentially slammed the door on almost all newcomers for more than forty years. Without it, Jewish immigration to the US would surely have soared during the 1920s and 1930s. Some 2.5 million Jews, most of them hoping for a better life than they had in tsarist Russia, had already come here between 1880 and 1924. Then, even in the decade before Hitler took power, Jews still had many reasons to leave Europe. Poland, whose Jewish population of 2.8 million was the continent’s largest, was a cauldron of antisemitism between the wars, with outbreaks of deadly violence, segregated seating and de facto quotas in many universities, and numerous other humiliations. https://www.nybooks.com/articles/2026/05/28/a-dream-of-a-socialist-commonwealth-the-jewish-bund/ No one who's paid attention during this century's xenophobic policies and attacks on refugees can fail to see the parallels. And no one who's paid attention to the genocide in Gaza and the official response in the "free" world to Palestinian solidarity movements can fail to see those parallels, either. For the Jews who are told – by Zionists, including the millions of American gentile Zionists who outnumber Jewish Zionists 30:1 – that all this is being done for us, that our continued existence requires it, Crabapple's history of the Bund shows us what's on the other side of the mirror. As NYT editor Max Strasser writes in his review of Here Where We Live: [The Bund was] the kind of movement leftists today dream about — political party, social movement, mutual aid group — with tens of thousands of members. The Bund published newspapers and ran soup kitchens and summer camps; its athletes competed in a socialist version of the Olympics. Bund activists organized across Eastern Europe and beyond — they helped elect a congressman on the Lower East Side. https://www.nytimes.com/2026/04/06/books/review/here-where-we-live-is-our-country-molly-crabapple.html The politics we dream of isn't a fantasy. It's the politics our grandparents lived – a politics that wasn't lost, but rather, erased. Erased by Nazis and Stalinists, who committed wholesale slaughter of Bundists. But that politics was also erased by Zionists, who swept through the Displaced Persons' camps of post-war Europe, imposing a draft on the Jews who'd been penned in those stinking camps by a world that refused to welcome Jews, even after the horrors of the death-camps were widely known. Zionists bullied and coerced these Jews – including Bundists who rejected their cause – to serve as foot-soldiers in the Israeli army, even beating elderly parents until their sons and daughters agreed to fight. Bundists always rejected all forms of ethno-nationalism. As Jews, they had lived in the violence and oppression that always attended every ethno-nationalist program. They never imagined that Israel would escape this fate. As the Bundist leader Henryk Erlich wrote in 1933: "We are not a chosen people. Our nationalism is just as ugly, just as harmful as the nationalisms of all the other nations." Crabapple has done heroic and important work in excavating this history. She has vindicated the sacrifices made by the Bundist archivists who smuggled their papers out of Nazi occupation and gave their lives to ensure that some day their story could be told. In so doing, she has also vindicated her own great-grandfather, Sam Rothbort, a Bundist who fled the Pale of Settlement for New York City, whose art-practice traveled to Crabapple through her mother, who is also a painter. It wasn't just the art-practices that traveled – it was also the art, and it was one of Rothbort's paintings ("Itka, the Bundist," depicting a girl throwing a rock through a window) that set her on this journey. This volume is also graced by Crabapple's own art, stark monochrome ink-washes in her characteristic style, which bring these long-dead people to vivid life. They're a reminder of the role that culture plays in every radical movement, of the ways that the Bund welcomed its members to live a radical life through sport and song and picnics, and not just meetings and street-demonstrations. Even before this book, Crabapple had made a mark through her paintings and writings. But with Here Where We Live Is Our Country, Crabapple has given us a magnum opus, a book that might help us turn the tide of history. Hey look at this (permalink) What Is a Dickover? https://daringfireball.net/2026/05/what_is_a_dickover Inventing ELIZA: How the First Chatbot Shaped the Future of AI https://sites.google.com/view/elizaarchaeology/book Inside Graham Platner’s Plan To Wield Power https://www.levernews.com/graham-platners-power/ mcmodernslopcore https://www.tumblr.com/mcmansionhell/817896092499869696/mcmodernslopcore Locus Award for Best Non-fiction https://en.wikipedia.org/wiki/Locus_Award_for_Best_Non-fiction Object permanence (permalink) #20yrsago Sign a letter supporting the BBC’s online archive https://web.archive.org/web/20060704182401/http://www.freeculture.org.uk/letters/CreativeArchiveLetter #20yrsago Home chemistry under assault https://web.archive.org/web/20060603021709/http://wired.com/wired/archive/14.06/chemistry_pr.html #20yrsago Cliches to avoid when writing about women and video-games https://web.archive.org/web/20060704223941/http://www.richardcobbett.co.uk/codex/clicktoread/filingcabinet/writing_a_girls_in_games_article/ #20yrsago JPEG patent invalidated https://web.archive.org/web/20060613015757/http://www.pubpat.org/Chen672Rejected.htm #20yrsago SF story about AI-human love https://www.salon.com/2006/05/30/perfect_man/ #15yrsago Sensation: Acerbic novel about pop culture and popular madness as functions of parasitic manipulation https://memex.craphound.com/2011/05/30/sensation-acerbic-novel-about-pop-culture-and-popular-madness-as-functions-of-parasitic-manipulation/ #15yrsago Every Pirate Wants to Be an Admiral: why less copyright gets you more culture https://www.theguardian.com/commentisfree/video/2011/may/30/internet-piracy-cory-doctorow #15yrsago Social incentives vs economic incentives in crowdsourced work https://web.archive.org/web/20110602184500/https://blog.crowdflower.com/2011/05/designing-incentives-for-crowdsourcing-workers/ #15yrsago Painful workarounds from computer novices https://www.reddit.com/r/AskReddit/comments/hmlmd/what_is_the_most_painful_way_you_have_seen_your/ #10yrsago To imagine the ocean of the future: picture a writhing mass of unkillable tentacles, forever https://web.archive.org/web/20160530145354/https://arstechnica.com/science/2016/05/octopuses-may-indeed-be-your-new-overlords/ #10yrsago When Brad Birkenfeld blew the whistle on UBS, the US government paid him $104M and sent him to jail https://web.archive.org/web/20160602152611/http://fullmeasure.news/news/politics/the-whistleblower-05-23-2016 #10yrsago The last time there were this many unsold $100M+ homes on the market, the world economy imploded https://web.archive.org/web/20160529040314/https://www.nytimes.com/2016/05/29/business/a-worrisome-pileup-of-100-million-homes.html #10yrsago David Foster Wallace’s essays on tennis, finally collected between one set of covers https://www.csmonitor.com/Arts-Culture/Books/2016/0530/String-Theory-gathers-the-brainy-witty-tennis-writing-of-David-Foster-Wallace #10yrsago United Arab Emirates hacked UK journalist https://citizenlab.ca/research/stealth-falcon/ #10yrsago Internet economics 101: “bandwidth hogs” considered harmless https://web.archive.org/web/20160530155601/https://arstechnica.com/tech-policy/2016/05/should-broadband-data-hogs-pay-more-isp-economics-say-no/ #20yrsago JPEG patent invalidated https://web.archive.org/web/20060613015757/http://www.pubpat.org/Chen672Rejected.htm #20yrsago SF story about AI-human love https://www.salon.com/2006/05/30/perfect_man/ #15yrsago Sensation: Acerbic novel about pop culture and popular madness as functions of parasitic manipulation https://memex.craphound.com/2011/05/30/sensation-acerbic-novel-about-pop-culture-and-popular-madness-as-functions-of-parasitic-manipulation/ #10yrsago To imagine the ocean of the future: picture a writhing mass of unkillable tentacles, forever https://web.archive.org/web/20160530145354/https://arstechnica.com/science/2016/05/octopuses-may-indeed-be-your-new-overlords/ #10yrsago When Brad Birkenfeld blew the whistle on UBS, the US government paid him $104M and sent him to jail https://web.archive.org/web/20160602152611/http://fullmeasure.news/news/politics/the-whistleblower-05-23-2016 #10yrsago The last time there were this many unsold $100M+ homes on the market, the world economy imploded https://web.archive.org/web/20160529040314/https://www.nytimes.com/2016/05/29/business/a-worrisome-pileup-of-100-million-homes.html #10yrsago David Foster Wallace’s essays on tennis, finally collected between one set of covers https://www.csmonitor.com/Arts-Culture/Books/2016/0530/String-Theory-gathers-the-brainy-witty-tennis-writing-of-David-Foster-Wallace #10yrsago United Arab Emirates hacked UK journalist https://citizenlab.ca/research/stealth-falcon/ #10yrsago Internet economics 101: “bandwidth hogs” considered harmless https://web.archive.org/web/20160530155601/https://arstechnica.com/tech-policy/2016/05/should-broadband-data-hogs-pay-more-isp-economics-say-no/ Upcoming appearances (permalink) SXSW London, Jun 2 https://www.sxswlondon.com/session/how-big-tech-broke-the-internet-b3c4a901 Kansas City: Facing the Future (Woodneath Library Center), Jun 10 https://www.mymcpl.org/events/119655/facing-future-cory-doctorow LA: The Reverse Centaur's Guide to Life After AI with Brian Merchant (Skylight Books), Jun 19 https://www.skylightbooks.com/event/skylight-cory-doctorow-presents-reverse-centaurs-guide-life-after-ai-w-brian-merchant Menlo Park: The Reverse Centaur's Guide to Life After AI with Angie Coiro (Kepler's), Jun 21 https://www.keplers.org/upcoming-events-internal/cory-doctorow-2026 Toronto: TBA, Jun 23 NYC: The Reverse Centaur's Guide to Life After AI with Jonathan Coulton (The Strand), Jun 24 https://www.strandbooks.com/cory-doctorow-the-reverse-centaur-s-guide-to-life-after-ai.html Philadelphia: The Reverse Centaur's Guide to Life After AI with David Williams (Fitler Club/Philadelphia Citizen), Jun 25 https://www.eventbrite.com/e/cory-doctorow-book-event-tickets-1990110326559 Chicago: The Reverse Centaur's Guide to Life After AI with Rick Perlstein (Exile in Bookville), Jun 26 https://exileinbookville.com/events/50628 Edinburgh International Book Festival with Jimmy Wales, Aug 17 https://www.edbookfest.co.uk/events/the-front-list-cory-doctorow-and-jimmy-wales South Bend: An Evening With Cory Doctorow (Notre Dame), Oct 6 https://franco.nd.edu/events/2026/10/06/an-evening-with-cory-doctorow/ Recent appearances (permalink) On Enshittification – and what can be done about it (Re:publica) https://www.youtube.com/watch?v=KhINQgPMVSI EFFecting Change: How to Disenshittify the Internet (EFF, with Wendy Liu) https://archive.org/details/effecting-change-enshittification The “Enshittification” of Everything (Bioneers) https://bioneers.org/cory-doctorow-enshittification-of-everything-zstf2605/ Enshittification (99% Invisible) https://99percentinvisible.org/episode/666-enshittification/ Artificial Intelligence: The Ultimate Disruptor, with Astra Taylor and Yoshua Bengio (CBC Ideas) https://www.cbc.ca/listen/live-radio/1-23-ideas/clip/16210039-artificial-intelligence-the-ultimate-disruptor Latest books (permalink) "Canny Valley": A limited edition collection of the collages I create for Pluralistic, self-published, September 2025 https://pluralistic.net/2025/09/04/illustrious/#chairman-bruce "Enshittification: Why Everything Suddenly Got Worse and What to Do About It," Farrar, Straus, Giroux, October 7 2025 https://us.macmillan.com/books/9780374619329/enshittification/ "Picks and Shovels": a sequel to "Red Team Blues," about the heroic era of the PC, Tor Books (US), Head of Zeus (UK), February 2025 (https://us.macmillan.com/books/9781250865908/picksandshovels). "The Bezzle": a sequel to "Red Team Blues," about prison-tech and other grifts, Tor Books (US), Head of Zeus (UK), February 2024 (thebezzle.org). "The Lost Cause:" a solarpunk novel of hope in the climate emergency, Tor Books (US), Head of Zeus (UK), November 2023 (http://lost-cause.org). "The Internet Con": A nonfiction book about interoperability and Big Tech (Verso) September 2023 (http://seizethemeansofcomputation.org). Signed copies at Book Soup (https://www.booksoup.com/book/9781804291245). "Red Team Blues": "A grabby, compulsive thriller that will leave you knowing more about how the world works than you did before." Tor Books http://redteamblues.com. "Chokepoint Capitalism: How to Beat Big Tech, Tame Big Content, and Get Artists Paid, with Rebecca Giblin", on how to unrig the markets for creative labor, Beacon Press/Scribe 2022 https://chokepointcapitalism.com Upcoming books (permalink) "The Reverse-Centaur's Guide to AI," a short book about being a better AI critic, Farrar, Straus and Giroux, June 2026 (https://us.macmillan.com/books/9780374621568/thereversecentaursguidetolifeafterai/) "Enshittification, Why Everything Suddenly Got Worse and What to Do About It" (the graphic novel), Firstsecond, 2026 "The Post-American Internet," a geopolitical sequel of sorts to Enshittification, Farrar, Straus and Giroux, 2027 "Unauthorized Bread": a middle-grades graphic novel adapted from my novella about refugees, toasters and DRM, FirstSecond, April 20, 2027 "The Memex Method," Farrar, Straus, Giroux, 2027 Colophon (permalink) Today's top sources: Currently writing: "The Post-American Internet," a sequel to "Enshittification," about the better world the rest of us get to have now that Trump has torched America. Third draft completed. Submitted to editor. "The Reverse Centaur's Guide to AI," a short book for Farrar, Straus and Giroux about being an effective AI critic. LEGAL REVIEW AND COPYEDIT COMPLETE. "The Post-American Internet," a short book about internet policy in the age of Trumpism. PLANNING. A Little Brother short story about DIY insulin PLANNING This work – excluding any serialized fiction – is licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net. https://creativecommons.org/licenses/by/4.0/ Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution. How to get Pluralistic: Blog (no ads, tracking, or data-collection): Pluralistic.net Newsletter (no ads, tracking, or data-collection): https://pluralistic.net/plura-list Mastodon (no ads, tracking, or data-collection): https://mamot.fr/@pluralistic Bluesky (no ads, possible tracking and data-collection): https://bsky.app/profile/doctorow.pluralistic.net Medium (no ads, paywalled): https://doctorow.medium.com/ Tumblr (mass-scale, unrestricted, third-party surveillance and advertising): https://mostlysignssomeportents.tumblr.com/tagged/pluralistic "When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla READ CAREFULLY: By reading this, you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ISSN: 3066-764X
More in AI
When I finished learning how to build an LLM from scratch, I was left with a mystery: my own models were not as good as OpenAI's original GPT-2 models, despite being based on the same architecture. My models all had 163M parameters, and followed the design from Sebastian Raschka's book "Build a Large Language Model (from Scratch)". That meant that they were pretty much the same as the setup for the OpenAI GPT-2 "small" instance, except that they did not use weight-tying or bias on the QKV matrices. Weight-tying means that you re-use the initial embedding matrix as the output head at the end, and using it means that GPT-2 small saved quite a few parameters -- it was 124M rather than 163M -- at, at least in my own experiments, a cost in quality; similarly, while I found that QKV bias made a tiny improvement in loss terms, I'd felt it was likely within the noise. But GPT-2 small consistently beat my models on an instruction fine-tuning (IFT) task -- also adapted from Raschka's book. That test fine-tunes the model on a subset of the Alpaca dataset, until validation loss starts rising, and then runs a test set through the resulting model. The responses to the test set questions are stored, and then I run all of the responses from all of the models under test past GPT 5.5 in one go to get an aggregate score; more details here. GPT-2 small always did better than any of my models on this. Additionally, it did surprisingly well on a simpler eval -- one that just measured the cross entropy loss it got on a test set. It scored close to my own best models, and better than many of them. What made this result particularly interesting was that the test set in question was a split of my own training data; my models would not have seen it when training (at least, in theory), but it seems likely that it would be much more similar to their own training data than it was to OpenAI's. I've checked two things while probing this mystery: It seems very likely that the GPT-2 models were overtrained by modern standards; would overtraining my own models get them closer? It turned out that no, it probably didn't help with the IFT eval (though there might have been some signal there). It did help quite a lot with the test loss eval, though. The way I was handling dropout in the IFT test might have been unduly benefiting some models while working against others. I decided to standardise on not using dropout during this eval, as (counter-intuitively for me) it seemed to harm the results of most models, even those that had been pre-trained with dropout. In particular, the OpenAI weights were harmed by using dropout, and making a change that benefited them (along with some of my own models) seemed the most conservative approach to take in investigating this. The next thing I wanted to look into was the training data. The exact dataset that the various GPT-2 models were trained on has never been released; all we know about it is from the paper, where they say: [W]e created a new web scrape which emphasizes document quality. To do this we only scraped web pages which have been curated/filtered by humans. Manually filtering a full web scrape would be exceptionally expensive so as a starting point, we scraped all outbound links from Reddit, a social media platform, which received at least 3 karma. This can be thought of as a heuristic indicator for whether other users found the link interesting, educational, or just funny. They called it "WebText". There is an OpenWebText that tries to replicate it, but although they tried to follow the same procedure as the original, there's no guarantee that it is all that similar. By comparison, I'd normally been training against FineWeb. While this is a general web-scraping dataset, without the "curation" provided by using only stuff that was linked from upvoted Reddit posts, it has been refined to remove any obvious junk. I had felt that it was pretty much equivalent. But what if I were wrong about that? I decided to see if I could get better models by using better data. The starting point Here's a table of all of the models I've been comparing to date. The "Test loss" column shows how well the model in question did on that held-back cross entropy loss evaluation. The "IFT epochs" column shows how many epochs of fine-tuning the model needed before its validation loss started rising, the "IFT score" the score that GPT 5.5 gave the model's responses to the test set of my Alpaca data, and the "IFT rank" the model's rank in terms of that score. The OpenAI small model is in there in bold, and I've also included the OpenAI medium model for comparison purposes. Test loss IFT epochs IFT score IFT rank OpenAI weights: medium 3.231442 2 43.75 1 JAX, overtrained one long epoch 3.324953 3 19.77 4 JAX, overtrained two normal epochs 3.326482 4 19.72 5 JAX, with MHA bias, no dropout 3.418784 4 18.69 6 JAX, no MHA bias, no dropout 3.420089 5 21.46 3 JAX, no MHA bias, with dropout 3.476802 5 13.22 15 OpenAI weights: small 3.499677 2 26.00 2 1xrtx3090-stacked-interventions 3.538161 4 13.77 14 8xa100m40-stacked-interventions-1 3.577761 4 10.76 18 Cloud FineWeb, 8x A100 40 GiB 3.673623 3 17.72 7 1xrtx3090-baseline 3.683835 4 15.74 8 8xa100m40-baseline 3.691526 3 14.19 13 Cloud FineWeb, 8x H100 80 GiB 3.724507 4 14.33 12 Cloud FineWeb, 8x A100 80 GiB 3.729900 3 11.34 17 Cloud FineWeb, 8x B200 160 GiB 3.771478 4 14.67 11 Local FineWeb train 3.943522 5 12.31 16 Local FineWeb-Edu extended train 4.134991 5 15.04 9 Local FineWeb-Edu train 4.166892 5 14.99 10 You can see that the OpenAI small model did pretty well in terms of the test loss, when you consider that it has 39M fewer weights than my models and was being tested against a dataset that differs more from its likely training data than it does from my own models'. Additionally, the specific models that did better than OpenAI's small one were all trained with JAX rather than PyTorch -- my hypothesis for that is that it's a result of the JAX ones getting better initial weights by pure chance. But the big difference was in the IFT score. In the specific run that gave the results in this table, the OpenAI small model got 26.00 -- the closest of my own models was more than 4.5 points lower, at 21.46. This difference was consistent over all of my other test runs. The GPT-2 small model was always ahead of mine. (GPT-2 medium, of course, beat GPT-2 small and all of my models, but given that it is twice the size of mine, that's not a big surprise.) Now, quite some time ago, I had tried looking into data quality as a lever to pull for model performance. At the bottom of the table, with the worst test loss of all models, you can see two models: "Local FineWeb-Edu train" "Local FineWeb-Edu extended train" These two were (as you might guess from the names) trained on the FineWeb-Edu dataset, which includes just the most "educational" data from FineWeb. They scored very badly on the test loss score. Given that the test dataset is from FineWeb, that's not a big surprise -- as I've written previously: If you train a model on Jane Austen and then evaluate against Chuck Tingle, then you're not going to get amazing results. But again, GPT-2 had the same issue, and did perfectly well on the test loss eval. On the other hand, while these FineWeb-Edu models' performance on the IFT eval wasn't stellar -- there are plenty of my other models ahead of them -- they did seem to punch above their weight. Consistently across all of the IFT evals I've done, they have scored higher than many of the others -- despite their poor loss on the test eval. Additionally: they were amongst the first models that I trained, before I'd spent time learning about how to optimise my hyperparameters and training loop. They did not use gradient clipping, they did use dropout, their batch size was just "whatever I could squeeze into the GPU", and I didn't set the learning rate to the right kind of value or schedule it over the course of the training run. So maybe a new training run on FineWeb-Edu plus my training improvements would help? And maybe some other tweaks to the training data would be worth looking into? The plan I decided to see what would happen if I trained some models with better-quality data. Specifically, I would train models with my current optimised loop and hyperparameters on four different datasets: FineWeb-Edu -- essentially the same as "Local FineWeb-Edu train" but with a better training setup. This would test the "more educational -> better" hypothesis. A 50:50 split of FineWeb and FineWeb-Edu. I've read that LLMs can be helped by having a decent amount of lower-quality data in their training loop, as it helps them to generalise. Perhaps having some FineWeb in there in addition to the FineWeb-Edu stuff would improve that test loss score while also helping the IFT test? A "curated" dataset containing 45% of its contents from FineWeb, 45% from FineWeb-Edu, and 10% from the Simple English Wikipedia. The full Wikipedia is huge, and full of obscure facts -- while the Simple English one is small and hopefully richer in useful information on a per-token basis. And conveniently, Answer.ai have made a snapshot of it available on Hugging Face Hub. Might deliberately putting a bunch of encyclopaedic data into the training set make the model better at the IFT eval (which has lots of factual questions in it, like "who wrote Pride and Prejudice")? OpenWebText. Even though I was unsure how well it matched the original WebText, given that it was there, it seemed silly to not try training something on it and see how it matched up. I would train each model on 3.2B tokens of the chosen dataset; that's the Chinchilla-optimal amount for my 163M-parameter models. If there were any interesting results, then I might consider doing overtrained models later on. I decided to be at least vaguely scientific about this, and to pre-register some predictions: The FineWeb-Edu-only model would do pretty badly on the test loss, but better than my older FineWeb-Edu models (90%). It would also punch above its weight on the IFT eval (90%). The 50:50 split: I expected it to do worse on the test eval than my JAX FineWeb-only models (70%), but better than the FineWeb-Edu one (90%). I wasn't sure about how it would do on the IFT eval, but thought it might be somewhere in between the two groups (60%). The curated dataset I had high hopes for in terms of the IFT eval -- let's say 80% chance of it being the best of all of my models. For the test loss eval, I expected it to do about as well as the 50:50 split, maybe a little bit worse (70%). I had no idea how the OpenWebText eval would do! Could be worse, could be better. Here's how things turned out. The FineWeb-Edu model I already had a dataset based on FineWeb-Edu ready to go, from when I trained those two original models. It is just the 10B-token sample of the original dataset at the time I generated it last December, formatted appropriately for my training script (details on the dataset card). I kicked off a training run with my JAX code (which I've been using for the other posts in this series): giles@poppy:~/Dev/jax-gpt2-from-scratch (main)$ XLA_PYTHON_CLIENT_MEM_FRACTION=0.95 uv run train.py full-llm-full-train-with-mha-output-bias-fineweb-edu datasets/ 2026-09-11 18:11:47.991583 Downloading dataset Fetching 4 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 1772.93it/s] Download complete: : 0.00B [00:00, ?B/s] | 0/4 [00:00<?, ?it/s] 2026-09-11 18:11:48.226273 Loading dataset into RAM Download complete: : 0.00B [00:00, ?B/s] 2026-09-11 18:16:29.507646 Creating model 2026-09-11 18:16:33.042509 Creating optimizer 2026-09-11 18:16:34.138990 Start train 0%| | 0/33165 [00:00<?, ?it/s] 2026-09-11 18:17:38.486288 Saving checkpoint 1%|▌ | 173/33165 [13:22<39:17:03, 4.29s/it, loss=6.897, tps=21,201] ...and just less than 40 hours later, I had a model: Training complete in 142,912.226 seconds 2026-09-13 09:58:26.437276 Tokens seen: 3,260,252,160 2026-09-13 09:58:26.437284 Throughput: 22,813 tokens/second 2026-09-13 09:58:26.437302 Final train loss: 3.342 2026-09-13 09:58:26.437309 Done I converted the saved JAX safetensors file from the last checkpoint into a format that would be compatible with my PyTorch eval code, and ran my smoke test: how would it complete the sentence "Every effort moves you"? Every effort moves you closer to God’s Kingdom, and even closer to Him. As we can see in That was nice and coherent -- if unusually religious! -- so that was promising. I ran the test eval: giles@perry:~/Dev/ddp-base-model-from-scratch (main)$ uv run test_loss.py datasets/ ../jax-gpt2-from-scratch/runs/full-llm-full-train-with-mha-output-bias-fineweb-edu/model.json ../jax-gpt2-from-scratch/runs/full-llm-full-train-with-mha-output-bias-fineweb-edu/checkpoints/latest/pytorch-model.safetensors Fetching 4 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 2758.50it/s] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3200/3200 [03:52<00:00, 13.74it/s] Loss against our test dataset: 3.632900 That was pretty good, putting it at a better test loss than all of the models I had trained without optimised hyperparameters, and worse than all of the ones I had trained on FineWeb with optimised hyperparameters. So that fit in with my prediction that it would be better than the old FineWeb-Edu models; the fact that it was also better than the non-optimised training runs with FineWeb seemed sensible enough that I felt silly for not having predicted that it would have fallen exactly there :-) I decided to leave the IFT eval until the end so that I could check all of the models from these experiments together, so it was time to upload this one to Hugging Face, and move on to the next model. 50:50 FineWeb to FineWeb-Edu I put together a new repo with a script to prepare datasets specifically for my training setup. You provide it with config that specifies some source datasets along with information about how to process them and how to mix them together, and it uploads a new dataset to Hugging Face Hub with the required characteristics. For example, for the 50:50 FineWeb to FineWeb-Edu split, the config looked like this: { "seed": 42, "tokens_desired": 10000000000, "upload_dataset_name": "gpjt/fw-fwedu-5050-gpt2-tokens", "sources": [ { "name": "FineWeb", "hf_id": "HuggingFaceFW/fineweb", "hf_name": "sample-10BT", "hf_split": "train", "item_field": "text", "weight": 50 }, { "name": "FineWeb-Edu", "hf_id": "HuggingFaceFW/fineweb-edu", "hf_name": "sample-10BT", "hf_split": "train", "item_field": "text", "weight": 50 } ] } The way the script works is pretty simple: it works out (based on those weights and the tokens_desired) how many tokens it wants from each source dataset, shuffles the items in the sources, then it loops until it has the desired number of tokens or more stored in an output. In the loop, it works out which source is currently most under-represented, grabs an item from it, tokenises it, and adds it to the output. Running it with that 50:50 config seemed to work fine: giles@perry:~/Dev/prepare-llm-training-dataset (main)$ uv run prepare-dataset.py runs/fw-fwedu-5050/ Resolving data files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 27468/27468 [00:00<00:00, 89875.56it/s] Loading dataset shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 102/102 [00:00<00:00, 133.75it/s] Resolving data files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 2410/2410 [00:00<00:00, 87461.48it/s] Loading dataset shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 98/98 [00:00<00:00, 200.09it/s] 2026-09-13 20:13:22.000187: Generating dataset; per-source counts 2026-09-13 20:13:22.000217: FineWeb: 5,000,000,000 2026-09-13 20:13:22.000221: FineWeb-Edu: 5,000,000,000 FineWeb: 100%|████████████████████████████████████████████████████████████████████████████████████████████████▉| 4999999705/5000000000 [1:01:33<00:00, 1353639.33token/s] FineWeb-Edu: 5000000363token [1:01:33, 1353639.47token/s] 2026-09-13 21:14:55.747239: Done generating tokens 2026-09-13 21:14:55.748480: FineWeb: 4,999,999,705 / 5,000,000,000 (1.000, 1 iterators) 2026-09-13 21:14:55.748487: FineWeb-Edu: 5,000,000,363 / 5,000,000,000 (1.000, 1 iterators) 2026-09-13 21:14:55.748489: Total: 10,000,000,068 2026-09-13 21:14:55.748491: Catting... 2026-09-13 21:16:29.565152: Catted into a tensor of shape torch.Size([10000000068]) 2026-09-13 21:16:29.566663: Saving... 2026-09-13 21:16:36.006267: Saved 2026-09-13 21:16:36.009413: Uploading to gpjt/fw-fwedu-5050-gpt2-tokens Processing Files (1 / 1) : 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 20.0GB / 20.0GB, 117MB/s New Data Upload : 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 14.6GB / 14.6GB, 98.1MB/s ...du-5050/train.safetensors: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 20.0GB / 20.0GB 2026-09-13 21:17:59.545875: Done So we had almost-perfect 50:50 balance between the datasets, and it saved this dataset on Hugging Face. I ran a script to double-check that it looked sane, and it did, so it was time to spin up a training run: giles@perry:~/Dev/jax-gpt2-from-scratch (main)$ XLA_PYTHON_CLIENT_MEM_FRACTION=0.90 uv run train.py full-llm-full-train-with-mha-output-bias-fw-fwedu-5050 datasets/ 2026-09-13 21:20:59.880918 Downloading dataset Fetching 2 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [01:13<00:00, 36.70s/it] Download complete: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20.0G/20.0G [01:13<00:00, 1.24GB/s] 2026-09-13 21:22:13.521745 Loading dataset into RAM Download complete: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20.0G/20.0G [01:13<00:00, 272MB/s] 2026-09-13 21:22:33.787720 Creating model 2026-09-13 21:22:35.501063 Creating optimizer 2026-09-13 21:22:36.043837 Start train 0%| | 0/33165 [00:00<?, ?it/s] 2026-09-13 21:23:11.437206 Saving checkpoint 0%| | 26/33165 [02:20<38:07:05, 4.14s/it, loss=9.308, tps=18,246] That was running on perry, my normal workstation, and I kicked it off in parallel with the "curated" model training run below on poppy my training box, but I'll keep the runs separate for the purposes of this writeup. When this had been running for an hour or so, our power went out. My guess is that having the tumble dryer running, the car charging, the kettle boiling, the electric hob switched on, and two machines doing training runs is a bit too much for our electrics... which might be a problem in the future, especially if (as planned) I make poppy a multi-GPU machine. However, as things stand, I was able to kick it off again after switching the circuit breaker back on, and things held up. Again, about 40 hours later: Training complete in 136,060.457 seconds 2026-09-15 12:05:26.432638 Tokens seen: 3,227,516,928 2026-09-15 12:05:26.432642 Throughput: 23,721 tokens/second 2026-09-15 12:05:26.432650 Final train loss: 3.793 2026-09-15 12:05:26.432653 Done (Note that the numbers reported at the end of a restarted run like this only include what happened after the restart.) I converted it to PyTorch-compatible tensors, and did the smoke test: Every effort moves you on to other options—in fact, it’s not even worth that effort. Just make Looking good! Time for the loss test: giles@perry:~/Dev/ddp-base-model-from-scratch (main)$ uv run test_loss.py datasets/ ../jax-gpt2-from-scratch/runs/full-llm-full-train-with-mha-output-bias-fw-fwedu-5050/model.json ../jax-gpt2-from-scratch/runs/full-llm-full-train-with-mha-output-bias-fw-fwedu-5050/checkpoints/latest/pytorch-model.safetensors Fetching 4 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 1192.07it/s] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3200/3200 [03:53<00:00, 13.72it/s] Loss against our test dataset: 3.462454 That was almost in keeping with my prediction that it would do worse than the JAX FineWeb-only models, except that it was better than the worst of those, "JAX, no MHA bias, with dropout": it was actually better than I predicted. So, a promising model. Time to upload it to Hugging Face -- and now let's move on to the next one. The "curated" dataset With my dataset-preparation script, this was easy enough to set up: { "seed": 42, "tokens_desired": 10000000000, "upload_dataset_name": "gpjt/fw-fwedu-simplewiki-gpt2-tokens", "sources": [ { "name": "FineWeb", "hf_id": "HuggingFaceFW/fineweb", "hf_name": "sample-10BT", "hf_split": "train", "item_field": "text", "weight": 45 }, { "name": "FineWeb-Edu", "hf_id": "HuggingFaceFW/fineweb-edu", "hf_name": "sample-10BT", "hf_split": "train", "item_field": "text", "weight": 45 }, { "name": "Simple English Wikipedia", "hf_id": "answerdotai/simplewiki", "hf_name": "articles", "hf_split": "train", "item_field": "md", "weight": 10 } ] } Running that worked nicely: giles@perry:~/Dev/prepare-llm-training-dataset (main)$ uv run prepare-dataset.py runs/fw-fwedu-simplewiki/ Resolving data files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 27468/27468 [00:00<00:00, 90196.13it/s] Loading dataset shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 102/102 [00:00<00:00, 358.90it/s] Resolving data files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 2410/2410 [00:00<00:00, 88254.11it/s] Loading dataset shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 98/98 [00:00<00:00, 589.23it/s] 2026-09-13 18:59:04.106327: Generating dataset; per-source counts 2026-09-13 18:59:04.106387: FineWeb: 4,500,000,000 2026-09-13 18:59:04.106407: FineWeb-Edu: 4,500,000,000 2026-09-13 18:59:04.106422: Simple English Wikipedia: 1,000,000,000 FineWeb: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████▉| 4499997964/4500000000 [59:41<00:00, 1256362.56token/s] FineWeb-Edu: 4500000607token [59:41, 1256363.31token/s] Simple English Wikipedia: 1000002889token [59:41, 279192.58token/s] 2026-09-13 19:58:45.874744: Done generating tokens 2026-09-13 19:58:45.876043: FineWeb: 4,499,997,964 / 4,500,000,000 (1.000, 1 iterators) 2026-09-13 19:58:45.876048: FineWeb-Edu: 4,500,000,607 / 4,500,000,000 (1.000, 1 iterators) 2026-09-13 19:58:45.876052: Simple English Wikipedia: 1,000,002,889 / 1,000,000,000 (1.000, 6 iterators) 2026-09-13 19:58:45.876054: Total: 10,000,001,460 2026-09-13 19:58:45.876056: Catting... 2026-09-13 20:00:18.811748: Catted into a tensor of shape torch.Size([10000001460]) 2026-09-13 20:00:18.813169: Saving... 2026-09-13 20:00:22.773873: Saved 2026-09-13 20:00:22.773936: Uploading to gpjt/fw-fwedu-simplewiki-gpt2-tokens Processing Files (1 / 1) : 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 20.0GB / 20.0GB, 143MB/s New Data Upload : 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 19.8GB / 19.8GB, 142MB/s ...plewiki/train.safetensors: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 20.0GB / 20.0GB 2026-09-13 20:01:59.270021: Done One thing that is worth noting in that output is the "6 iterators" for the Simple English Wikipedia. If a source dataset runs out of items while we're building up the results in this script, we start iterating over it again (with a different seed for the shuffle so that the ordering is different). The "6 iterators" means that it needed to do that 6 times -- the original creation of the iterator at the start of the script, and five more. So that means that the Simple English Wikipedia is repeated (oversampled) somewhere between five and six times in the dataset. That's not a bad thing! From what I've read, it's actually quite standard to oversample highly educational content in LLM training datasets. And anyway, the dataset the script generated was 10B tokens, of which we're only using 3.2B for the training run in this post, so it would only appear somewhere between one and two times. The repetition would likely only really cut in if and when we did an overtrained model on the dataset. Anyway, I ran my check against the uploaded dataset -- the first few items were clearly from FineWeb, FineWeb-Edu, and the Simple English Wikipedia. It was time to kick off a training run: giles@poppy:~/Dev/jax-gpt2-from-scratch (main)$ XLA_PYTHON_CLIENT_MEM_FRACTION=0.95 uv run train.py full-llm-full-train-with-mha-output-bias-fw-fwedu-simplewiki datasets/ 2026-09-13 20:24:48.037024 Downloading dataset Downloading (incomplete total...): 0.00B [00:00, ?B/s] Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. | 0/2 [00:00<?, ?it/s] WARNING:huggingface_hub.utils._http:Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. Fetching 2 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [02:51<00:00, 85.85s/it] Download complete: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20.0G/20.0G [02:51<00:00, 435MB/s] 2026-09-13 20:27:39.934884 Loading dataset into RAM Download complete: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20.0G/20.0G [02:51<00:00, 116MB/s] 2026-09-13 20:31:20.492877 Creating model 2026-09-13 20:31:24.054143 Creating optimizer 2026-09-13 20:31:25.100832 Start train 0%| | 0/33165 [00:00<?, ?it/s] 2026-09-13 20:32:29.650379 Saving checkpoint 0%|▎ | 107/33165 [08:38<39:05:39, 4.26s/it, loss=7.631, tps=20,293] Again, this was interrupted by the power outage that hit the 50:50 training run, but I was able to restart from a checkpoint. After another 22 hours, it crashed with an error that I've seen before: jax.errors.JaxRuntimeError: INTERNAL: CUDA error: Failed to end stream capture: CUDA_ERROR_STREAM_CAPTURE_INVALIDATED: operation failed due to a previous error during capture [executable_name='jit_train_step'] I put it aside as a one-off oddity when I hit it last time, but this time I dug in a bit more. I noted that it had not ever happened on perry, but seemed to be an issue on poppy, and that poppy had an older version of CUDA and the Nvidia drivers -- might that be the cause? I decided to upgrade those before kicking off the next run, but for now just restarted the run from the most recent checkpoint. (Note for anyone who is hitting the same error: it has not occurred since the upgrade, so that's worth trying.) This time it completed OK: Training complete in 59,564.515 seconds 2026-09-15 15:56:52.909888 Tokens seen: 1,367,212,032 2026-09-15 15:56:52.909894 Throughput: 22,953 tokens/second 2026-09-15 15:56:52.909912 Final train loss: 3.332 2026-09-15 15:56:52.909959 Done Again, these numbers just show what happened after the most recent restart. I copied it over to perry, converted it into a format that was compatible with my PyTorch code, and ran the smoke test: Every effort moves you by the air, for it will make you a better athlete, so your body becomes bigger and stronger Coherent enough -- time for the loss eval: giles@perry:~/Dev/ddp-base-model-from-scratch (main)$ uv run test_loss.py datasets/ ~/Dev/jax-gpt2-from-scratch/runs/full-llm-full-train-with-mha-output-bias-fw-fwedu-simplewiki/model.json ~/Dev/jax-gpt2-from-scratch/runs/full-llm-full-train-with-mha-output-bias-fw-fwedu-simplewiki/checkpoints/latest/pytorch-model.safetensors Fetching 4 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 1007.64it/s] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3200/3200 [03:57<00:00, 13.48it/s] Loss against our test dataset: 3.542460 Again, in line with my predictions -- worse than the JAX FineWeb-only models, and indeed than the very best PyTorch one, 1xrtx3090-stacked-interventions, and also worse than the 50:50 split, but better than the FineWeb-Edu one. I uploaded it to Hugging Face, and it was time to move on to what was meant to be the final model for this set of experiments. The OpenWebText run Again, this was a simple enough config to set up: { "seed": 42, "tokens_desired": 10000000000, "upload_dataset_name": "gpjt/openwebtext-gpt2-tokens", "sources": [ { "name": "OpenWebText", "hf_id": "Skylion007/openwebtext", "hf_name": "plain_text", "hf_split": "train", "item_field": "text", "weight": 50 } ] } ...and the build and upload process worked well (and took much less time -- for some reason, sampling randomly from a single dataset is faster than sampling from two or three): giles@perry:~/Dev/prepare-llm-training-dataset (main)$ uv run prepare-dataset.py runs/openwebtext/ Resolving data files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 80/80 [00:00<00:00, 32723.26it/s] Resolving data files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 80/80 [00:00<00:00, 97940.55it/s] Loading dataset shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 80/80 [00:00<00:00, 1200.13it/s] 2026-09-15 13:16:47.622617: Generating dataset; per-source counts 2026-09-15 13:16:47.622645: OpenWebText: 10,000,000,000 Resolving data files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 80/80 [00:00<00:00, 45602.65it/s] Resolving data files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 80/80 [00:00<00:00, 67650.06it/s] Loading dataset shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 80/80 [00:00<00:00, 307.11it/s] OpenWebText: 10000000024token [31:46, 5246208.64token/s] 2026-09-15 13:48:33.761350: Done generating tokens 2026-09-15 13:48:33.762021: OpenWebText: 10,000,000,024 / 10,000,000,000 (1.000, 2 iterators) 2026-09-15 13:48:33.762026: Total: 10,000,000,024 2026-09-15 13:48:33.762028: Catting... 2026-09-15 13:49:33.115508: Catted into a tensor of shape torch.Size([10000000024]) 2026-09-15 13:49:33.115923: Saving... 2026-09-15 13:49:36.365978: Saved 2026-09-15 13:49:36.366027: Uploading to gpjt/openwebtext-gpt2-tokens Processing Files (0 / 1) : 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████▉| 20.0GB / 20.0GB, 147MB/s New Data Upload : 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 19.9GB / 19.9GB, 147MB/s ...webtext/train.safetensors: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████▉| 20.0GB / 20.0GB 2026-09-15 13:51:16.202890: Done Note that it needed to oversample -- that "2 iterators". OpenWebText is about 40 GiB uncompressed, and so that's about 10B GPT-2 tokens -- presumably just a little bit less. Again, given that I was planning to use just the first 3.2B tokens of the dataset, I didn't feel that it would matter. I ran the check script on the newly-uploaded Hugging Face dataset and all looked well, so that was all set for the training run. I upgraded poppy first with a sudo pacman -Syu to see if that helped with the weird error that I got in the previous run (which, as I said, it looks like it did), then kicked it off: giles@poppy:~/Dev/jax-gpt2-from-scratch (main)$ XLA_PYTHON_CLIENT_MEM_FRACTION=0.95 uv run train.py full-llm-full-train-with-mha-output-bias-openwebtext datasets/ 2026-09-15 16:42:32.606185 Downloading dataset Fetching 2 files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 941.38it/s] Download complete: : 0.00B [00:00, ?B/s] | 0/2 [00:00<?, ?it/s] 2026-09-15 16:42:32.879987 Loading dataset into RAM Download complete: : 0.00B [00:00, ?B/s] 2026-09-15 16:45:40.438791 Creating model 2026-09-15 16:45:43.840269 Creating optimizer 2026-09-15 16:45:44.848351 Start train 0%| | 0/33165 [00:00<?, ?it/s] 2026-09-15 16:46:50.632075 Saving checkpoint 1%|█ | 332/33165 [24:33<38:45:54, 4.25s/it, loss=6.623, tps=22,154] About 31 hours in, it crashed again, but this time it was my own dumb fault: poppy has a relatively small disk and I ran out of space. I fixed that and kicked it off again from the most recent checkpoint, and this time it completed: Training complete in 33,927.995 seconds 2026-09-17 11:25:10.835989 Tokens seen: 779,747,328 2026-09-17 11:25:10.835994 Throughput: 22,982 tokens/second 2026-09-17 11:25:10.836012 Final train loss: 3.165 2026-09-17 11:25:10.836018 Done I converted it to PyTorch for the smoke test: Every effort moves you through each phase, so it's not a complete picture. I'm sure your story was ...which looked solid, so it was time for the test loss eval: giles@perry:~/Dev/ddp-base-model-from-scratch (main)$ uv run test_loss.py datasets/ ~/Dev/jax-gpt2-from-scratch/runs/full-llm-full-train-with-mha-output-bias-openwebtext/model.json ~/Dev/jax-gpt2-from-scratch/runs/full-llm-full-train-with-mha-output-bias-openwebtext/checkpoints/latest/pytorch-model.safetensors Fetching 4 files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 674.76it/s] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3200/3200 [03:59<00:00, 13.37it/s] Loss against our test dataset: 4.045255 Our worst score yet in this experiment! Worse than any of my models so far, apart from the two FineWeb-Edu ones I did without optimised hyperparameters. Now, the first draft of this post went straight to the results from here, but the story wasn't quite over yet... Test set contamination GPT-6 Astra is relentless. Before I publish any of these posts, I run them past an editorial board of LLMs to look for issues. GPT-6 Astra not only checked the text, it also visited the code I'd linked to to check that out too, and spotted something problematic. It's obvious in retrospect, but my code to build the new datasets had a high risk of including the contents of the -- in theory held-back -- test set. The way that the test set was generated was that I downloaded the 10B sample of FineWeb back in December, splitting it into 99% training data and 1% "validation". That validation split was about 100M tokens, and I was only using the first 19M or so for actual validation runs during training, so I (somewhat arbitrarily) designated about 19M other tokens starting at position 50M in there as my test set. Now, my new dataset-generation code was just sampling randomly from the complete 10B sample of FineWeb. So there was nothing stopping it from pulling in data that was in that old validation split! That meant that it was quite likely that my new "curated" and "50:50" datasets contained at least some of the test set that was meant to have been held back from the models during training. On reflection, the problem was potentially even worse. FineWeb-Edu is a subset of FineWeb; my existing FineWeb-Edu dataset came from the 10B sample of the Hugging Face original, and so it also could potentially contain documents that I'd put into the test set. The first thing to do was to establish the size of the problem. I wrote a script to take in a "forbidden" dataset and split; this was assumed to be formatted as one big tensor of GPT-2 tokens, which is what all of my datasets are. It would then split it by end-of-text tokens, and generate a hash and a token count for each resulting "document". Optionally, you could restrict it to only considering a subset -- the n tokens starting at position p -- and it would then generate hashes/lengths for the documents inside that slice, or that overlapped it at the start or the end. I ran that to generate a list of hashes for the entire validation set -- the validation split of gpjt/fineweb-gpt2-tokens -- and then used a second script to check my various training sets (and the validation set itself) to see how much of a contamination problem there was. I got these results: Dataset Split Contamination with validation set gpjt/fineweb-gpt2-tokens validation 102163003 out of 102163003 tokens (100.00%) gpjt/fineweb-gpt2-tokens train 636166 out of 102163003 tokens (0.62%) gpjt/fineweb-edu-gpt2-tokens train 672189 out of 102163003 tokens (0.66%) gpjt/fw-fwedu-5050-gpt2-tokens train 49224580 out of 102163003 tokens (48.18%) gpjt/fw-fwedu-simplewiki-gpt2-tokens train 44233824 out of 102163003 tokens (43.30%) gpjt/openwebtext-gpt2-tokens train 212 out of 102163003 tokens (0.00%) So: The validation set was 100% "contaminated" with itself, which was a useful sanity check. The training set of gpjt/fineweb-gpt2-tokens had what I felt was a small level of contamination. It was interesting that there was any at all -- I think that must mean that there are some repeated documents in the original dataset, and some of them wound up with copies in both my training and validation splits. The gpjt/fineweb-edu-gpt2-tokens dataset also had what felt like a reassuringly low level of contamination. Both gpjt/fw-fwedu-5050-gpt2-tokens and gpjt/fw-fwedu-simplewiki-gpt2-tokens, however, looked problematic. In both cases, the training datasets had more than 40% of the validation/test set in them. gpjt/openwebtext-gpt2-tokens was, as you'd expect, almost completely uncontaminated. It looks like maybe one document happened to have been picked up by both the OpenWebText and the FineWeb crawls and then included in the bit of FineWeb I was using for validation. However, these numbers -- while scary, at least for the 50:50 and the curated datasets -- were not quite the ones to use. They showed how much of the full validation set showed up in the full training set; what I actually cared about was how much of the test set -- those 19M tokens starting at position 50M in the validation split -- was in the actual subset of the training datasets that I actually trained on -- the first ~3.2B of them. I re-ran the script to generate hashes for just the test set, and then re-ran the contamination-checking script, telling it just to look at the appropriate subset of the training tokens, and got this: Dataset (first 3.2B tokens only) Split Contamination with test set gpjt/fineweb-gpt2-tokens train 26557 out of 19632681 tokens (0.14%) gpjt/fineweb-edu-gpt2-tokens train 32079 out of 19632681 tokens (0.16%) gpjt/fw-fwedu-5050-gpt2-tokens train 2986889 out of 19632681 tokens (15.21%) gpjt/fw-fwedu-simplewiki-gpt2-tokens train 2682430 out of 19632681 tokens (13.66%) gpjt/openwebtext-gpt2-tokens train None It was clear that there was a problem -- certainly with gpjt/fw-fwedu-5050-gpt2-tokens and gpjt/fw-fwedu-simplewiki-gpt2-tokens. They'd seen what felt like a significant amount of the test set while training, so their results on the test loss eval were dubious at best. I decided to train those two models afresh, and see what the result was in terms of loss. If the difference was huge, I'd look into the risks of the (much smaller) contamination of gpjt/fineweb-gpt2-tokens and gpjt/fineweb-edu-gpt2-tokens. But if it was pretty small, I'd not worry about that too much. I extended the script that prepared datasets so that the config file could specify a forbidden_dataset. Any documents in the source datasets that matched forbidden ones would be excluded from the output. I then updated the config for gpjt/fw-fwedu-5050-gpt2-tokens and gpjt/fw-fwedu-simplewiki-gpt2-tokens so that the whole validation split of gpjt/fineweb-gpt2-tokens was forbidden, and re-generated them. You can see the updated datasets here and here. Running the contamination-checker script against them showed that they were clear. I then re-did the full training runs for those models; the uncontaminated version of the 50:50 split model is here, and the curated one is here. And the good news: both of them actually did very slightly better at the test loss eval than their equivalents that had been trained on the contaminated data: Model Contaminated Test loss JAX, FineWeb/FineWeb-Edu 50:50 No 3.449257 JAX, FineWeb/FineWeb-Edu 50:50 Yes 3.462454 JAX, curated No 3.534068 JAX, curated Yes 3.542460 There are a number of possibilities that come to mind; perhaps learning from the test set just doesn't happen with tiny 163M models like this, or perhaps while the contaminated models were learning, the benefit they got from that was outweighed by the data that they got instead of the test set data being in some way better for training purposes, at least in terms of the loss eval. But anyway, I felt that if the effect of seeing more than 10% of the test set data during training was so tiny, then the effect of seeing less than 0.2% -- which is what the FineWeb-Edu model in this set of training runs had, as did all of my other FineWeb-only models from previous experiments -- would be even smaller and I'd disregard it. That was excellent news! I didn't need to start all of my experiments from scratch. For the rest of this post, I will include the numbers and results for the contaminated models as well as the uncontaminated ones -- they're interesting for several reasons -- but for future posts I'll skip the contaminated ones. So -- finally! -- let's start digging into the final results. Results Firstly, I think it's worth taking a look at all of the test loss results in context. Here they are in a table, with the new models in bold: Test loss OpenAI weights: medium 3.231442 JAX, overtrained one long epoch 3.324953 JAX, overtrained two normal epochs 3.326482 JAX, with MHA bias, no dropout 3.418784 JAX, no MHA bias, no dropout 3.420089 JAX, FineWeb/FineWeb-Edu 50:50 (uncontaminated) 3.449257 JAX, FineWeb/FineWeb-Edu 50:50 (contaminated) 3.462454 JAX, no MHA bias, with dropout 3.476802 OpenAI weights: small 3.499677 JAX, curated (uncontaminated) 3.534068 1xrtx3090-stacked-interventions 3.538161 JAX, curated (contaminated) 3.542460 8xa100m40-stacked-interventions-1 3.577761 JAX, FineWeb-Edu 3.632900 Cloud FineWeb, 8x A100 40 GiB 3.673623 1xrtx3090-baseline 3.683835 8xa100m40-baseline 3.691526 Cloud FineWeb, 8x H100 80 GiB 3.724507 Cloud FineWeb, 8x A100 80 GiB 3.729900 Cloud FineWeb, 8x B200 160 GiB 3.771478 Local FineWeb train 3.943522 JAX, openwebtext 4.045255 Local FineWeb-Edu extended train 4.134991 Local FineWeb-Edu train 4.166892 I think there's something very clear here: with the new models, the more FineWeb that was in the training mix, the better the model did on this eval. I think I might have been subconsciously expecting that in the predictions I did before running these experiments, but in retrospect it's so incredibly obvious that I feel silly for not mentioning it explicitly! But that tells us something interesting. From the description in the paper, whatever OpenAI did the GPT-2 training run on, it was not like FineWeb. It was probably more similar to OpenWebText -- and yet, that model was the one that performed the worst on this test eval, so if it is more like OpenWebText, there must be some other factor involved. But moving on for now: how about the IFT test -- the one that kicked off all of this work in the first place? I generated a set of IFT responses for all of the new models, and then ran them (plus responses for all of the other models on that table above) past GPT 5.5, and found that one of my new models was getting quite close to the original GPT-2 small weights! So I did four more runs, so that I could get an average. Here are the results -- the "IFT score" is the average across all five runs of the judge, and the "IFT rank" is based on that. The "IFT epochs" was from the original result-generation script. Test loss IFT epochs IFT score IFT rank OpenAI weights: medium 3.231442 2 42.36 1 JAX, overtrained one long epoch 3.324953 3 18.67 7 JAX, overtrained two normal epochs 3.326482 4 18.71 6 JAX, with MHA bias, no dropout 3.418784 4 17.90 8 JAX, no MHA bias, no dropout 3.420089 5 20.50 4 JAX, FineWeb/FineWeb-Edu 50:50 (uncontaminated) 3.449257 4 17.69 9 JAX, FineWeb/FineWeb-Edu 50:50 (contaminated) 3.462454 4 19.30 5 JAX, no MHA bias, with dropout 3.476802 5 13.02 21 OpenAI weights: small 3.499677 2 25.19 2 JAX, curated (uncontaminated) 3.534068 4 16.63 10 1xrtx3090-stacked-interventions 3.538161 4 13.51 19 JAX, curated (contaminated) 3.542460 4 13.58 18 8xa100m40-stacked-interventions-1 3.577761 4 10.19 24 JAX, FineWeb-Edu 3.632900 4 24.56 3 Cloud FineWeb, 8x A100 40 GiB 3.673623 3 16.59 11 1xrtx3090-baseline 3.683835 4 15.15 12 8xa100m40-baseline 3.691526 3 13.64 16 Cloud FineWeb, 8x H100 80 GiB 3.724507 4 13.59 17 Cloud FineWeb, 8x A100 80 GiB 3.729900 3 10.79 23 Cloud FineWeb, 8x B200 160 GiB 3.771478 4 13.70 15 Local FineWeb train 3.943522 5 11.87 22 JAX, openwebtext 4.045255 4 13.28 20 Local FineWeb-Edu extended train 4.134991 5 14.29 14 Local FineWeb-Edu train 4.166892 5 14.69 13 If you want to see the full numbers, they're below. The number that initially surprised me, and made me decide to do multiple LLM-judge runs was the one for the "JAX, FineWeb-Edu" model. In my first run it came in at 24.35 vs the OpenAI small weights' 24.93 -- so close that I wondered if it might even beat them on a re-run. However, in the further four runs its score was consistently lower than the OpenAI model's, and the gap extended a bit in some. So, was FineWeb-Edu the clear winner here? Perhaps. If you look at the contaminated/uncontaminated pairs, something interesting pops out. For the 50:50 mix, the model trained with the contaminated dataset got 19.30, and the one trained on the uncontaminated one got 17.69 -- a difference of 1.61. For the "curated" dataset, the situation was even more interesting: uncontaminated got 16.63, while contaminated got 13.58, a delta of 3.05 points. Remember, the contamination issue is about whether or not the model saw the held-back test set during training. It was an issue for the test loss that is based on that test set, but is entirely orthogonal to the IFT test. From the IFT perspective, both contaminated and uncontaminated models in each case saw training data that was -- in theory, at least -- essentially the same in terms of quality. Indeed, the uncontaminated run saw almost the same data in the same order as the contaminated one, except that some items were omitted, and then extra ones were added to the end. The purpose of this set of experiments was to see how data quality affected the results on the IFT test set. But in the case of the curated model, something that should be unrelated to data quality changed the results by 3.05 points! If something as simple as changing which data of the same quality the model is trained with can affect the IFT score so drastically, it makes it a bit harder to be certain as to whether or not data quality really had the effect we were looking for. On the other hand, the FineWeb-Edu model came in at 24.56, which is 4.06 points better than the 20.50 that the closest other model got -- more than the 3.05 points we see in difference between the two curated dataset models. And it's worth noting that the model with 20.50 is "JAX, no MHA bias, no dropout", which has a subtly different architecture -- no bias on the output projection of the multi-head attention blocks. A better comparison might be "JAX, with MHA bias, no dropout", which got a score of 17.90, for a whacking great difference of 6.66 points. I think that without doing a very large number of training runs on different datasets with different mixes, each one created with a different seed, it would be hard to work out exactly what is in the noise here and what is not. However, that would cost a lot in terms of time. I think that the best thing here is to chalk this up as a fairly decent indication that FineWeb-Edu improves matters for the IFT eval, but far from a certainty. But it's certainly worth noting that whatever the noise is, it has a range of at least 3.05 points -- and the FineWeb-Edu model is just 0.63 points short of GPT-2 small! So there could well be something there. Of course, we don't know whether that model got (by chance) the best possible balance of FineWeb-Edu tokens, and could never win -- or whether it got a bad balance and would actually beat GPT-2 with a better one. So that's certainly worth keeping in mind. As an aside, the result for the curated dataset really surprised me. I had expected that it would be the best one, simply because it almost certainly contained more facts. I took a look at its answers to the questions -- one possibility that came to mind might be that it would get better responses to questions like "What is the chemical symbol for chlorine" or "Who wrote Pride and Prejudice" than the others, but would fail on less knowledge-based tasks. But it was terrible at fact-based questions too: Name the author of 'Pride and Prejudice'. What is the periodic symbol for chlorine? As I understand it, many real-world training runs do include (often oversampled) amounts of highly educational training data like this model's dataset did. But perhaps the models that I'm training are just too small to be able to make use of the data they gained that way -- maybe doing things this way and expecting good results is like asking six-year-old children to memorise stuff before they've learned enough to be able to make use of it 1. It's worth noting that the GPT-2 small model also failed on those factual questions. Well, anyway: I think we have some useful results here, so let's work out what that means for next steps. Conclusion The results we got in these experiments point in two interesting directions. The perfect connection between the amount of FineWeb in the training set and the result on the (FineWeb-based) test loss eval, while perfectly obvious in retrospect, really does highlight how mysterious it is that the OpenAI small weights do so well on that test. The fact that FineWeb-Edu did well on the IFT test tells us that there does seem to be value in using richer training data -- though the less-spectacular results of the 50:50 mix and the curated one weaken that a bit, as does the indicator of what the noise due to data selection from equivalently high-quality datasets might be. The OpenWebText result I think I'll ignore, given that -- while in theory it should be similar to what OpenAI trained on -- there are no guarantees, and it might differ in non-obvious ways for non-obvious reasons. I think that the right direction to take this going forward is to separate these two angles. I should chase a higher IFT score, and then once I have nailed that down, I should see what (if anything) might allow me to get the resulting model to improve its test score. But I will need to make sure that whatever dataset I use, I use various "mixes" of it -- versions created with different random seeds. In my earlier experiments with overtraining, I did find that it didn't seem to improve the IFT results -- but it did improve the test loss. So perhaps identifying the right combination of other factors to boost the IFT score, then overtraining the result, might help? Of course, my overtraining tests were with FineWeb, so the connection might not hold up as well if the starting model (as seems likely) was trained on a different dataset. Also, while working through the results here, I've come to the conclusion that the set of models I'm using is a bit confusing -- there are now different hyperparameter settings, small architectural differences (the MHA bias thing), dropout settings during the pre-training, and now datasets. I think that's OK for now; I should see this part of this series as more ideation than actually running the proper experiments. But at the end, when I have some solid hypotheses with a reasonable amount of backup, I should start from scratch: a baseline model, then staged interventions to build up to what (hopefully) will be a model as good as GPT-2 small. Anyway, I'll wrap this one up here. I think that the next lever to pull is (perhaps surprisingly) going to be weight tying. I had previously kind of disregarded that as a possibility, but while I was working on this post, something popped into my mind. The OpenAI models were originally trained with weight tying. My codebase does actually support doing it -- but because I got the OpenAI weights I'm using from the code in "Build a Large Language Model (from Scratch)", when I'm running the IFT test, the weights are not actually tied! We load up a model that has separate but identical embedding and output head matrices, and then we fine-tune that. So those two matrices can vary independently during fine-tuning -- to put it another way, while GPT-2 small was pre-trained with 124M parameters, the IFT test is being done on a 163M-parameter version. Does that give them some non-obvious advantage? And would adding weight-tying to my own models help, either with or without the output heads being independent at fine-tuning time? Stay tuned :-) Appendix: all IFT judge runs Here are the numbers for all of the IFT judge runs, included for completeness. You can see that the LLM judge ranks models very consistently between runs, but there is variation -- that is, on some runs it's in what I think of as a "better mood" than others, and if that's the case, it will give better scores -- but it will give them almost consistently between models, so all of the models do better. Note that (unlike the table above) this one is sorted by the average IFT score rather than the test loss. Model Run 1 Run 2 Run 3 Run 4 Run 5 Average OpenAI weights: medium 42.24 42.16 42.95 41.83 42.61 42.36 OpenAI weights: small 24.93 24.96 25.39 25.01 25.66 25.19 JAX, FineWeb-Edu 24.35 24.55 24.3 24.68 24.9 24.56 JAX, no MHA bias, no dropout 20.5 19.9 20.76 21.25 20.07 20.50 JAX, FineWeb/FineWeb-Edu 50:50 (contaminated) 19.16 18.86 19.61 19.17 19.7 19.30 JAX, overtrained two normal epochs 18.47 18.29 19.17 18.69 18.91 18.71 JAX, overtrained one long epoch 18.04 18.71 19.62 18.41 18.57 18.67 JAX, with MHA bias, no dropout 17.49 17.35 18.33 17.73 18.62 17.90 JAX, FineWeb/FineWeb-Edu 50:50 (uncontaminated) 17.37 17.73 17.53 18.01 17.83 17.69 JAX, curated (uncontaminated) 16.77 16.03 17.3 16.08 16.96 16.63 Cloud FineWeb, 8x A100 40 GiB 16.44 16.23 17.14 16.62 16.54 16.59 1xrtx3090-baseline 14.85 15.07 15.19 15.14 15.51 15.15 Local FineWeb-Edu train 14.37 14.23 15.08 14.79 15 14.69 Local FineWeb-Edu extended train 14.4 14.07 13.82 14.56 14.61 14.29 Cloud FineWeb, 8x B200 160 GiB 13.37 13.05 13.85 13.67 14.57 13.70 8xa100m40-baseline 13.64 13.36 13.9 13.32 13.97 13.64 Cloud FineWeb, 8x H100 80 GiB 13.45 13.32 13.6 13.51 14.07 13.59 JAX, curated (contaminated) 13.09 13.48 13.95 13.23 14.15 13.58 1xrtx3090-stacked-interventions 13.37 13.11 14.04 13.84 13.17 13.51 JAX, openwebtext 12.88 12.7 13.74 13.53 13.53 13.28 JAX, no MHA bias, with dropout 13.19 12.86 12.98 12.85 13.24 13.02 Local FineWeb train 11.75 11.75 12.21 11.46 12.19 11.87 Cloud FineWeb, 8x A100 80 GiB 10.68 10.2 11.03 10.55 11.49 10.79 8xa100m40-stacked-interventions-1 9.44 9.79 10.84 10.2 10.66 10.19 A small boy asleep on his right side, the right arm stuck out, the right hand hanging limp over the edge of the bed. Through a round grating in the side of a box a voice speaks softly. "The Nile is the longest river in Africa and the second in length of all the rivers of the globe. Although falling short of the length of the Mississippi-Missouri, the Nile is at the head of all rivers as regards the length of its basin, which extends through 35 degrees of latitude …" At breakfast the next morning, "Tommy," some one says, "do you know which is the longest river in Africa?" A shaking of the head. "But don't you remember something that begins: The Nile is the …" "The - Nile - is - the - longest - river - in - Africa - and - the - second - in - length - of - all - the - rivers - of - the - globe …" The words come rushing out. "Although - falling - short - of …" "Well now, which is the longest river in Africa?" The eyes are blank. "I don't know." "But the Nile, Tommy." "The - Nile - is - the - longest - river - in - Africa - and - second …" "Then which river is the longest, Tommy?" Tommy burst into tears. "I don't know," he howls. Brave New World, Aldous Huxley ↩
The unstated disagreement that underpins safety debates
It’s disheartening how much power gets generated and then promptly lost as it travels through grid networks. This leaking of electricity happens when it vanishes as heat as well as when it is pilfered by thieves and nonpaying customers. More than half of the countries that track these metrics lost at least 10 percent of their electricity in 2023, according to the World Bank. Losses topped 20 percent for 24 of those nations. Two countries lost more than half of what they generated. With numbers this high, cutting down on losses makes sense. Electricity demand is rising beyond what many grid operators can supply; reducing waste would help meet some of that demand without having to build new power plants. Plus, when the power comes from fossil fuels, any loss means emitting even more greenhouse gases into the atmosphere. And electric losses hit the bottom lines of power providers, which ultimately pass those costs on to everyone else. The trouble is, reducing electricity losses is a hard and expensive process that takes a long time. Typically, the less maintained the grid infrastructure, the more electricity that’s lost. And the more fragile the region’s law enforcement and government, the more prevalent the power theft. Natural disasters and war make things worse. Delhi’s Power Grid Comeback Fixing a power grid requires a systemic approach across many sectors. There’s no one technology that will solve the problem. At the outset, the obstacles to success may feel insurmountable. Equipment across entire grid networks must be updated. Multiple arms of government must agree to reforms and coordinate to ensure power providers are set up to succeed. Regulations must be written or revised, investments made, cultures changed. The city of Delhi did all those things. Over the past 25 years, it cut its electricity losses from about 50 to 5 percent. How the city pulled off that impressive feat is the focus of “The Epic Comeback of Delhi’s Power Grid” by Mini Shaji Thomas, an electrical engineer at the university Jamia Millia Islamia who has lived in Delhi since the 1990s. She gives us a view from the inside—as a resident and a power systems expert. Delhi is a shining example, but some other regions have significantly lowered electricity losses over the last quarter century too. The country of Georgia went from losses of over 16 percent in 2002 to about 8 percent in 2023. In Singapore, losses dropped from 6.6 percent to a nearly nonexistent 0.2 percent over the same time period. Global Electricity Theft Crisis But there are many parts of the world where electricity losses remain a problem or have gotten worse. In Jamaica, where power theft is rampant, losses have hovered between 21 and 28 percent for years. Argentina’s losses nearly doubled between 2015 and 2023, going from an all-time low of about 12 percent to an all-time high of nearly 24 percent. The main problem: Transmission and distribution companies lacked the capital to maintain and upgrade their networks, which left equipment operating under stress. A delay in the installation of smart meters has allowed thieves to more easily siphon power and tamper with meters. Thomas says she hopes her account of Delhi’s grid comeback will serve as a blueprint for others. It’s possible to replicate the sweeping changes Delhi made, she says. But it “requires a concerted effort from all stakeholders, customers, the utility, the government, and their employees.”