Full Width [alt+shift+f] Shortcuts [alt+shift+k]
Sign Up [alt+shift+s] Log In [alt+shift+l]
39
Humanity only has one engineering project, building better engineers than humans. After that, the thing we built can do the engineering. Clips have been making the rounds on Twitter from my second Lex about the “bishop guy” in a chess engine, or a “cone guy” in a self driving car. These engineering ideas look ever more ridiculous. Since the beginning of comma, I wanted to make a machine that could drive cars like a human. Obviously there’s no reference to traffic cones inside human DNA, they learn about them from data. So there shouldn’t be any reference to traffic cones in your codebase. Rich Sutton stated this most iconically in 2019. One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning. But then where does it stop? Why draw the line at...
11 months ago

Improve your reading experience

Logged in users get linked directly to articles resulting in a better reading experience. Please login for free, it takes less than 1 minute.

More from the singularity is nearer

AMD YOLO

AMD is sending us the two MI300X boxes we asked for. They are in the mail. It took a bit, but AMD passed my cultural test. I now believe they aren’t going to shoot themselves in the foot on software, and if that’s true, there’s absolutely no reason they should be worth 1/16th of NVIDIA. CUDA isn’t really the moat people think it is, it was just an early ecosystem. tiny corp has a fully sovereign AMD stack, and soon we’ll port it to the MI300X. You won’t even have to use tinygrad proper, tinygrad has a torch frontend now. Either NVIDIA is super overvalued or AMD is undervalued. If the petaflop gets commoditized (tiny corp’s mission), the current situation doesn’t make any sense. The hardware is similar, AMD even got the double throughput Tensor Cores on RDNA4 (NVIDIA artificially halves this on their cards, soon they won’t be able to). I’m betting on AMD being undervalued, and that the demand for AI has barely started. With good software, the MI300X should outperform the H100. In for a quarter million. Long term. It can always dip short term, but check back in 5 years.

5 days ago 6 votes
The Demoralization is just Beginning

This is a map of primary trading partners, US vs China, and how it has evolved over the last 20 years. Think about it, and realize this probably reflects your experience. I know there was a similar panic about Japan in the 80s, but Japan by population has always been 3x smaller than the US, whereas China is 3x larger. In addition, we had and have military bases in Japan. This is not the same situation. The US, since I have been born, has been coasting. The main product made by the US is the dollar, and it used those manufactured dollars to outsource everything. Most jobs in the US are now basically fake. It’s basically an economy in which five people stick a pipe in the ground, but that pipe is the fed and the oil was the good will built up over 1870-1970. In 2008, with the bailouts, it was made clear that the US has no interest in reform. The next decade, in perhaps a spitting in your face move, the fed made the interest rate 0. Known as ZIRP, this had never been done before. This led to insane perversions. When I got into business, I didn’t understand that business in America was mostly a total scam. Sure, you might look at a single business, and be like, oh, that sounds reasonable, but then you zoom out and look at the entire system, and it doesn’t really make sense. It’s scams feeding other scams. Wanna each start a business, pass dollars back and forth over and over again, and drive both our revenues super high? Sure, we don’t produce anything, but we have companies with high revenues and we can raise money based on those revenues. We’ll both be rich! Let’s do it with a bunch of extra steps so people don’t catch on though. They’ll only see it reflected in the lack of movement of real macro metrics. You see, the US is a “developed” country, which means real growth is over? You do understand that guns and boats are made of steel, right? Oh, airplanes aren’t, they are made of aluminum. Oh…right, yea, it’s not just steel it is absolutely everything. The future is chips you say? All the good chips are made in the Republic of China you say? This 2021 article lays it out clearly, and it also explains why nothing I saw in Silicon Valley made any sense. I’m not going to go into the personal stories, but I just had an underlying assumption that the goal was growth and value production. It isn’t. It’s self licking ice cream cone scams, and any growth or value is incidental to that. It isn’t until you understand this that people’s behavior starts to make sense. America really is at a fork in the road. In one world, they abandon all hopes of being an empire, becoming a regional power with highly protectionist economics. This happened before, and it’s called Europe. I know it’s hard to believe now, but Europe used to be the seat of power for the whole world. The sun never set on the British empire. Now they put you in jail for memes. Protectionist America is a boring place and not somewhere I want to be. It kicks the can further down the road of poverty, basically embraces socialism, is stagnant, is stale, is a museum…etc, again there’s a contemporary example of this. When I said on Lex they were gonna nationalize NVIDIA, look at the AI Diffusion Framework, and notice how Trump hasn’t repealed it. It allows export of GPUs to only 18 countries. Nationalization with American characteristics. It tells the other 177 countries that they should plan on purchasing their AI infrastructure from China. The other path, which is the exciting path, is the attempt to maintain an empire. An empire has to compete on its merits. There’s two simple steps to restore American greatness: 1) Brain drain the world. Work visas for every person who can produce more than they consume. I’m talking doubling the US population, bringing in all the factory workers, farmers, miners, engineers, literally anyone who produces value. Can we raise the average IQ of America to be higher than China? 2) Back the dollar by gold (not socially constructed crypto), and bring major crackdowns to finance to tie it to real world value. Trading is not a job. Passive income is not a thing. Instead, go produce something real and exchange it for gold. The first will bring the value of “American” labor in line with its global market value. It is a particularly unique advantage of the US over China, the US has a potentially much larger pool of talent. Non ironically, diversity is our strength. Unfortunately, there’s a lot of resistance to American labor finding its market value. The second will prevent a lot of the scams. The reason the banking industry is so big is that it is close to the source of the made up dollars. If currency is gold backed, you could imagine something similar happening to the mining industry instead. However, the mining industry is real! It uses steel and aluminum to build physical things. And imagine when we start to mine space. That’s a way better reward function than scamming politicians out of fake dollars. Unfortunately, I doubt either will happen. They very much both can, but people haven’t been demoralized enough yet.

a week ago 11 votes
Money is the Map

A lot of smooth brains on Hacker News about the last post. I’m sorry if you spent your whole life worshipping money, but hey, the Bible warned you about false idols, don’t shoot the messenger. “It’s easier to imagine the end of the world than the end of capitalism” – Mark Fisher It’s actually very easy to imagine the end of capitalism. Imagine capitalism as a game of sharks, where eventually the biggest shark ends up gobbling up all the fish, and that one shark is the last player left standing with all the money. When one person (or company) has all the money, do you see how the money would be worthless? I’ll spell this out clearly. Money is a map, it is not a territory. Please understand what I mean by this before continuing to read. You can erase the mountains from the map, but you still have to climb over them in real life, and even worse, now you don’t have a map! “Everything around you that you call ‘life’ was made up by people who were no smarter than you” – Steve Jobs So, if money is the map, what territory is it attempting to capture? Presumably something having to do with value, but increasingly, as we are buying and selling baskets of derivatives of memecoins, nothing. A map that doesn’t accurately capture a territory is not a Schelling point. It’s not a useful map. And maps are only as good as their usefulness. Useless maps die out. Do you agree or disagree that money is supposed to be a map of value? If you disagree, that’s an ought and I can’t use logic to convince you otherwise, I can just call you a moron who refuses to burn paper $100 bills for warmth on a deserted island. Many capitalists I meet are as stupid as communists, trying to give a moral justification for their system. This is my money, I deserve it. I should be able to passively deploy my capital into the markets and live off the returns. “Moral victories are for minor league coaches.” – JAY-Z A economic system is only good in so much as it effectively deploys capital for real growth. If real economic growth is only 3 percent, any time you are earning beyond that, somebody else is losing. And yet somehow, today, you can put your money in money market accounts and earn a “risk-free” 5 percent…hmm something doesn’t make sense. Who is losing? You will eventually be unable to squeeze the productive people any further. The worst was an e-mail I got with someone who supposedly agreed with me. “Value creation (for all stakeholders) is at the core of the organization/ business model I am putting together…Anyway I wanted to let you know others out there who share your vision.” – anon email Fuck your stakeholders. Fuck your business model. You don’t understand me at all. Stop worrying so much about the distribution of the pie. Start thinking about how to make the pie bigger. With exponential (what 3 percent year over year is) growth, the latter outstrips the former by so much. The right distribution is simply: From each according to his ability, to each according to his ability to effectively deploy capital to achieve real economic growth. Communism is dumb cause it goes to the poor (who routinely demonstrate that they poorly deploy capital). Capitalism is dumb cause it goes to the rent-seekers (who frequently deploy capital to increase their moat). Acceleration is the way.

2 weeks ago 14 votes
Nobody Profits

Intellectual property is a really dumb idea. “But piracy is theft. Clean and simple. It’s smash and grab. It ain’t no different than smashing a window at Tiffany’s and grabbing merchandise.” - Joe Biden, 46th president of the USA Except it isn’t and Joe Biden is a senile moron. Because when you smash the windows and grab the stuff, Tiffany’s no longer has the stuff. With piracy, everyone has the stuff. It’s a lot more like taking a picture, which Tiffany’s probably encourages. Win-win cooperation. Wealth is being increasingly concentrated. What’s shocking to me is how much everyone still cares about money. Even the die-hard complain about capitalism type deeply cares, because the opposite of love isn’t hate, it’s indifference. I hate scammers, but I’m pretty indifferent to money. The best outcome of AI is if it delivers huge amounts of value to society but no profit to anyone. The old days of the Internet were this goldmine. The Internet delivered huge value but no profit, and that’s why it was good. Suddenly we had all these new powers. Then people figured out how to monetize it. It was a race to extract every tiny bit of value, and now we have today’s Internet. Can this play out differently with AI? Let’s build technology and open source software that market breaks everything. Let’s demoralize the scammers so hard that they don’t even try. Every loser and grifter will be gone from technology because there’s nothing to be gained there. They can play golf all day or something. If I ever figure out how to channel power like Elon, I will do this. Spin up open source projects in every sector to eliminate all the capturable value. This is what I’m trying to do with comma.ai and tinygrad. I dream of a day when company valuations halve when I create a GitHub repo. Someday.

3 weeks ago 14 votes
Death of the Visceral

Pulled up at a stop light Imagine flying an x-wing down a corridor, having to turn the plane sideways to fit, a missile on your tail and closing, hitting the turbo, feeling the g force, coming up on the end of the corridor, pulling back hard on the stick the second the corridor opens, turning 90 degrees and watching the missile continue straight. Tingles. Adrenaline. Release. Or if you don’t want sci-fi, imagine winter circa 1645 in America. Several of your group almost dead from lack of food, tracking a deer, spotting it, shooting it with your bow, hitting but the deer is trying to run, fast twitch muscles charging and leaping, plunging a knife into its heart and knowing at that moment everyone is going to be okay. Heart rate calming laying on the warm deer. The modern world doesn’t have any real experiences like this any more. Survival has become a technocratic plod, making the right boring and careful decisions. There’s only fake experiences like the above, video games, sports, and drugs. And things like reckless driving, which are just kind of stupid. As we march toward ASI, this will only get worse. What the unabomber describes as Type 2 experiences, ones where you can achieve results with serious effort, will vanish. All that will be left are things you can have for no effort (like food) and things you can never have (like world peace). Even when humanity goes to Mars, we will be going as cargo. I was told recently I’m not engaged in my life, and it’s pretty true. Until I see a solution to this problem, even a sketch of a solution, what’s the point? Why sprint if you aren’t sure where you are going? I’m trying my best with comma and tiny corp, how do you make technology itself more accessible, not a fucking packaged product like when the default world talks about making technology more accessible. That’s just hiding complexity. But it’s so hard. Companies don’t work like how I thought they did, they just…exist. Which I guess in retrospect is obvious, there’s no adults in the room. Knowing the future doesn’t help you change it. I am continually shocked at how little people understand about anything, they don’t even understand that they don’t understand. Am I the same way? I try extremely hard to constantly test myself, if my predictions are wrong it’s clear I don’t understand. If I can’t build it I don’t understand. I’ll frequently read comments saying I don’t understand, but when I engage with these people they can’t explain what my world model gets wrong. A different meta world model? Or are they just idiots? To anyone who wants to supersede rationality, you better understand how to steelman every rationality argument. If I want to succeed, I believe I have to change who I am, and I’m not sure if that’s possible. I believe I’ve been making efforts in that direction, but I haven’t seen results yet. Working on AI is both the only thing that matters and also so demoralizing because of the above. I believe you have to give individuals control over the technology. And not by setting permissions in AWS that can be revoked, I mean in a nature sense. The ghost gunner is the real second amendment. This ideology holds me back so much in business, to the point I struggle to be competitive. But if you abandon that ideology, what’s the point to doing it at all? I have to win with a hand tied behind my back. We only make products for spiritual tops, not the majority of the world which is spiritual bottoms. Now, perhaps I have an ace in the hole. With the rise of AI, the spiritual bottoms will soon have no cash, because AI is the ultimate spiritual bottom. It would take a highly skilled terrorist to build spiritual top AI and even I’m not that crazy. So we’ll only have bottom AI, and it will outcompete all the human bottoms. Advertising will vanish once the hypnodrones have been released. Those humans will likely wirehead themselves out of the picture. This is the world I’m building for. Have you ever unconstrained your mind and thought about where the world is going? This won’t be like the steam engine replacing the horse, because all horses were bottoms. Ever seen a horse riding a human? Humanity bifurcates. Humans will retain control for the foreseeable future, the only question is, how many humans? If it’s 10, I’m out. If it’s 10k, 50/50 I’m in. If it’s 10M, I’m definitely in. My goal is to make this number as large as possible, it’s my best chance of survival. Give control of the technology to as many people as possible in a deep nature sense, not a permissions sense. In my opinion, this is what Elon gets wrong. Of course, he’s likely to be one of the 10, so maybe that’s why he doesn’t care. But what if he isn’t? Tesla and SpaceX are huge silos begging to be co-opted. I don’t think building silos like this is a good idea, compare the fate of the Telegram founder to the Signal founder. Build technology and structures that are inseparable from the narrative you want, as opposed to ones you think you can wield for good. On a long enough timeline, it will always end up in your enemy’s hands. Imagine if the only thing they could do with it furthers your goals.

a month ago 34 votes

More in programming

Supa Pecha Kucha

slug: supapechakucha

18 hours ago 3 votes
The Power of Principles in Web Development Decision-Making (article)

Discover how The Epic Programming Principles can transform your web development decision-making, boost your career, and help you build better software.

9 hours ago 2 votes
Closing the borders alone won't fix the problems

Denmark has been reaping lots of delayed accolades from its relatively strict immigration policy lately. The Swedes and the Germans in particular are now eager to take inspiration from The Danish Model, given their predicaments. The very same countries that until recently condemned the lack of open-arms/open-border policies they would champion as Moral Superpowers.  But even in Denmark, thirty years after the public opposition to mass immigration started getting real political representation, the consequences of culturally-incompatible descendants from MENAPT continue to stress the high-trust societal model. Here are just three major cases that's been covered in the Danish media in 2025 alone: Danish public schools are increasingly struggling with violence and threats against students and teachers, primarily from descendants of MENAPT immigrants. In schools with 30% or more immigrants, violence is twice as prevalent. This is causing a flight to private schools from parents who can afford it (including some Syrians!). Some teachers are quitting the profession as a result, saying "the Quran run the class room". Danish women are increasingly feeling unsafe in the nightlife. The mayor of the country's third largest city, Odense, says he knows why: "It's groups of young men with an immigrant background that's causing it. We might as well be honest about that." But unfortunately, the only suggestion he had to deal with the problem was that "when [the women] meet these groups... they should take a big detour around them". A soccer club from the infamous ghetto area of Vollsmose got national attention because every other team in their league refused to play them. Due to the team's long history of violent assaults and death threats against opposing teams and referees. Bizarrely leading to the situation were the team got to the top of its division because they'd "win" every forfeited match. Problems of this sort have existed in Denmark for well over thirty years. So in a way, none of this should be surprising. But it actually is. Because it shows that long-term assimilation just isn't happening at a scale to tackle these problems. In fact, data shows the opposite: Descendants of MENAPT immigrants are more likely to be violent and troublesome than their parents. That's an explosive point because it blows up the thesis that time will solve these problems. Showing instead that it actually just makes it worse. And then what? This is particularly pertinent in the analysis of Sweden. After the "far right" party of the Swedish Democrats got into government, the new immigrant arrivals have plummeted. But unfortunately, the net share of immigrants is still increasing, in part because of family reunifications, and thus the problems continue. Meaning even if European countries "close the borders", they're still condemned to deal with the damning effects of maladjusted MENAPT immigrant descendants for decades to come. If the intervention stops there. There are no easy answers here. Obviously, if you're in a hole, you should stop digging. And Sweden has done just that. But just because you aren't compounding the problem doesn't mean you've found a way out. Denmark proves to be both a positive example of minimizing the digging while also a cautionary tale that the hole is still there.

19 hours ago 2 votes
We all lose when art is anonymised

One rabbit hole I can never resist going down is finding the original creator of a piece of art. This sounds simple, but it’s often quite difficult. The Internet is a maze of social media accounts that only exist to repost other people’s art, usually with minimal or non-existent attribution. A popular image spawns a thousand copies, each a little further from the original. Signatures get cropped, creators’ names vanish, and we’re left with meaningless phrases like “no copyright intended”, as if that magically absolves someone of artistic theft. Why do I do this? I’ve always been a bit obsessive, a bit completionist. I’ve worked in cultural heritage for eight years, which has made me more aware of copyright and more curious about provenance. And it’s satisfying to know I’ve found the original source, that I can’t dig any further. This takes time. It’s digital detective work, using tools like Google Lens and TinEye, and it’s not always easy or possible. Sometimes the original pops straight to the top, but other times it takes a lot of digging to find the source of an image. So many of us have become accustomed to art as an endless, anonymous stream of “content”. A beautiful image appears in our feed, we give it a quick heart, and scroll on, with no thought for the human who sweated blood and tears to create it. That original artist feels distant, disconected. Whatever benefit they might get from the “exposure” of your work going viral, they don’t get any if their name has been removed first. I came across two examples recently that remind me it’s not just artists who miss out – it’s everyone who enjoys art. I saw a photo of some traffic lights on Tumblr. I love their misty, nighttime aesthetic, the way the bright colours of the lights cut through the fog, the totality of the surrounding darkness. But there was no name – somebody had just uploaded the image to their Tumblr page, it was reblogged a bunch of times, and then it appeared on my dashboard. Who took it? I used Google Lens to find the original photographer: Lucas Zimmerman. Then I discovered it was part of a series. And there was a sequel. I found interviews. Context. Related work. I found all this cool stuff, but only because I knew Lucas’s name. Traffic Lights, by Lucas Zimmerman. Published on Behance.net under a CC BY‑NC 4.0 license, and reposted here in accordance with that license. The second example was a silent video of somebody making tiny chess pieces, just captioned “wow”. It was clearly an edit of another video, with fast-paced cuts to make it accommodate a short attention span – and again with no attribution. This was a little harder to find – I had to search several frames in Google Lens before I found a summary on a Russian website, which had a link to a YouTube video by metalworker and woodworker Левша (Levsha). This video is four times longer than the cut-up version I found, in higher resolution, and with commentary from the original creator. I don’t speak Russian, but YouTube has auto-translated subtitles. Now I know how this amazing set was made, and I have a much better understanding of the materials and techniques involved. (This includes the delightful name Wenge wood, which I’d never heard before.) https://youtube.com/watch?v=QoKdDK3y-mQ A piece of art is more than just a single image or video. It’s a process, a human story. When art is detached from its context and creator, we lose something fundamental. Creators lose the chance to benefit from their work, and we lose the opportunity to engage with it in a deeper way. We can’t learn how it was made, find their other work, or discover how to make similar art for ourselves. The Internet has done many wonderful things for art, but it’s also a machine for endless copyright infringement. It’s not just about generative AI and content scraping – those are serious issues, but this problem existed long before any of us had heard of ChatGPT. It’s a thousand tiny paper cuts. How many of us have used an image from the Internet because it showed up in a search, without a second thought for its creator? When Google Images says “images may be subject to copyright”, how many of us have really thought about what that means? Next time you want to use an image from the web, look to see if it’s shared under a license that allows reuse, and make sure you include the appropriate attribution – and if not, look for a different image. Finding the original creator is hard, sometimes impossible. The Internet is full of shadows: copies of things that went offline years ago. But when I succeed, it feels worth the effort – both for the original artist and myself. When I read a book or watch a TV show, the credits guide me to the artists, and I can appreciate both them and the rest of their work. I wish the Internet was more like that. I wish the platforms we rely on put more emphasis on credit and attribution, and the people behind art. The next time an image catches your eye, take a moment. Who made this? What does it mean? What’s their story? [If the formatting of this post looks odd in your feed reader, visit the original article]

yesterday 1 votes
Apple does AI as Microsoft did mobile

When the iPhone first appeared in 2007, Microsoft was sitting pretty with their mobile strategy. They'd been early to the market with Windows CE, they were fast-following the iPod with their Zune. They also had the dominant operating system, the dominant office package, and control of the enterprise. The future on mobile must have looked so bright! But of course now, we know it wasn't. Steve Ballmer infamously dismissed the iPhone with a chuckle, as he believed all of Microsoft's past glory would guarantee them mobile victory. He wasn't worried at all. He clearly should have been! After reliving that Ballmer moment, it's uncanny to watch this CNBC interview from one year ago with Johny Srouji and John Ternus from Apple on their AI strategy. Ternus even repeats the chuckle!! Exuding the same delusional confidence that lost Ballmer's Microsoft any serious part in the mobile game.  But somehow, Apple's problems with AI seem even more dire. Because there's apparently no one steering the ship. Apple has been promising customers a bag of vaporware since last fall, and they're nowhere close to being able to deliver on the shiny concept demos. The ones that were going to make Apple Intelligence worthy of its name, and not just terrible image generation that is years behind the state of the art. Nobody at Apple seems able or courageous enough to face the music: Apple Intelligence sucks. Siri sucks. None of the vaporware is anywhere close to happening. Yet as late as last week, you have Cook promoting the new MacBook Air with "Apple Intelligence". Yikes. This is partly down to the org chart. John Giannandrea is Apple's VP of ML/AI, and he reports directly to Tim Cook. He's been in the seat since 2018. But Cook evidently does not have the product savvy to be able to tell bullshit from benefit, so he keeps giving Giannandrea more rope. Now the fella has hung Apple's reputation on vaporware, promised all iPhone 16 customers something magical that just won't happen, and even spec-bumped all their devices with more RAM for nothing but diminished margins. Ouch. This is what regression to the mean looks like. This is what fiefdom management looks like. This is what having a company run by a logistics guy looks like. Apple needs a leadership reboot, stat. That asterisk is a stain.

2 days ago 3 votes