More from the singularity is nearer
You know about Critical Race Theory, right? It says that if there’s an imbalance in, say, income between races, it must be due to discrimination. This is what wokism seems to be, and it’s moronic and false. The right wing has invented something equally stupid. Introducing Critical Trade Theory, stolen from this tweet. If there’s an imbalance in trade between countries, it must be due to unfair practices. (not due to the obvious, like one country is 10x richer than the other) There’s really only one way the trade deficits will go away, and that’s if trade goes to zero (or maybe if all these countries become richer than America). Same thing with the race deficits, no amount of “leg up” bullshit will change them. Why are all the politicians in America anti-growth anti-reality idiots who want to drive us into the poor house? The way this tariff shit is being done is another stupid form of anti-merit benefits to chosen groups of people, with a whole lot of grift to go along with it. Makes me just not want to play.
Intel is sitting on a huge amount of card inventory they can’t move, largely because of bad software. Most of this is a summary of the public #intel-hardware channel in the tinygrad discord. Intel currently is sitting on: 15,000 Gaudi 2 cards (with baseboards) 5,100 Intel Data Center GPU Max 1450s (without baseboards) If you were Intel, what would you do with them? First, starting with the Gaudi cards. The open source repo needed to control them was archived on Feb 4, 2025. There’s a closed source version of this that’s maybe still maintained, but eww closed source and do you think it’s really maintained? The architecture is kind of tragic, and that’s likely why they didn’t open source it. Unlike every other accelerator I have seen, the MMEs, which is where all the FLOPS are, are not controllable by the TPCs. While the TPCs have an LLVM port, the MME is not documented. After some poking around, I found the spec: It’s highly fixed function, looks very similar to the Apple ANE. But that’s not even the real problem with it. The problem is that it is controlled by queues, not by the TPCs. Unpacking habanalabs-dkms-1.19.2-32.all.deb you can find the queues. There is some way to push a command stream to the device so you don’t actually have to deal with the host itself for the queues. But that doesn’t prevent you having to decompose the network you are trying to run into something you can put on this fixed function block. Programmability is on a spectrum, ranging from CPUs being the easiest, to GPUs, to things like the Qualcomm DSP / Google TPU (where at least you drive the MME from the program), to this and the Apple ANE being the hardest. While it’s impressive that they actually got on MLPerf Training v4.0 training GPT3, I suspect it’s all hand coded, and if you even can deviate off the trodden path you’ll get almost no perf. Accelerators like this are okay for low power inference where you can adjust the model architecture for the target, Apple does a great job of this. But this will never be acceptable for a training chip. Then there’s the Data Center GPU Max 1450. Intel actually sent us a few of these. You quickly run into a problem…how do you plug them in? They need OAM sockets, 48V power, and a cooling solution that can sink 600W. As far as I can tell, they were only ever deployed in two systems, the Aurora Supercomputer and the Dell XE9640. It’s hard to know, but I really doubt many of these Dell systems were sold. Intel then sent us this carrier board. In some ways it’s helpful, but in other ways it’s not at all. It still doesn’t solve cooling or power, and you need to buy 16x MCIO cables (cheap in quantity, but expensive and hard to find off the shelf). Also, I never got a straight answer, but I really doubt Intel has many of these boards. And that board doesn’t look cheap to manufacturer more of. The connectors alone, which you need two of per GPU, cost $26 each. That’s $104 for just the OAM connectors. tiny corp was in discussions to buy these GPUs. How much would you pay for one of these on a PCIe card? The specs look great. 839 TFLOPS, 128 GB of ram, 3.3 TB/s of bandwidth. However…read this article. Even in simple synthetic benchmarks, the chip doesn’t get anywhere near its max performance, and it looks to be for fundamental reasons like memory latency. We estimate we could sell PCIe versions of these GPUs for $1,000; I don’t think most people know how hard it is to move non NVIDIA hardware. Before you say you’d pay more, ask yourself, do you really want to deal with the software? An adapter card has four pieces. A PCB for the card, a 12->48V voltage converter, a heatsink, and a fan. My quote from the guy who makes an OAM adapter board was $310 for 10+ PCBs and $75 for the voltage converter. A heatsink that can handle 600W (heat pipes + vapor chamber) is going to cost $100, then maybe $20 more for the fan. That’s $505, and you still need to assemble and test them, oh and now there’s tariffs. Maybe you can get this down to $400 in ~1000 quantity. So $200 for the GPU, $400 for the adapter, $100 for shipping/fulfillment/returns (more if you use Amazon), and 30% profit if you sell at $1k. tiny would net $1M on this, which has to cover NRE and you have risk of unsold inventory. We offered Intel $200 per GPU (a $680k wire) and they said no. They wanted $600. I suspect that unless a supercomputer person who already uses these GPUs wants to buy more, they will ride it to zero. tl;dr: there’s 5100 of these GPUs with no simple way to plug them in. It’s unclear if they worth the cost of the slot they go in. I bet they end up shredded, or maybe dumped on eBay for $50 each in a year like the Xeon Phi cards. If you buy one, good luck plugging it in! The reason Meta and friends buy some AMD is as a hedge against NVIDIA. Even if it’s not usable, AMD has progressed on a solid steady roadmap, with a clear continuation from the 2018 MI50 (which you can now buy for 99% off), to the MI325X which is a super exciting chip (AMD is king of chiplets). They are even showing signs of finally investing in software, which makes me bullish. If NVIDIA stumbles for a generation, this is AMD’s game. The ROCm “copy each NVIDIA repo” strategy actually works if your competition stumbles. They can win GPUs with slow and steady improvement + competition stumbling, that’s how AMD won server CPUs. With these Intel chips, I’m not sure who they would appeal to. Ponte Vecchio is cancelled. There’s no point in investing in the platform if there’s not going to be a next generation, and therefore nobody can justify the cost of developing software, therefore there won’t be software, therefore they aren’t worth plugging in. Where does this leave Intel’s AI roadmap? The successor to Ponte Vecchio was Rialto Bridge, but that was cancelled. The successor to that was Falcon Shores, but that was also cancelled. Intel claims the next GPU will be “Jaguar Shores”, but fool me once… To quote JazzLord1234 from reddit “No point even bothering to listen to their roadmaps anymore. They have squandered all their credibility.” Gaudi 3 is a flop due to “unbaked software”, but as much as I usually do blame software, nothing has changed from Gaudi 2 and it’s just a really hard chip to program for. So there’s no future there either. I can’t say that “Jaguar Shores” square instills confidence. It didn’t inspire confidence for “Joseph B.” on LinkedIn either. From my interactions with Intel people, it seems there’s no individuals with power there, it’s all committee like leadership. The problem with this is there’s nobody who can say yes, just many people who can say no. Hence all the cancellations and the nonsense strategy. AMD’s dysfunction is different. from the beginning they had leadership that can do things (Lisa Su replied to my first e-mail), they just didn’t see the value in investing in software until recently. They sort of had a point if they were only targeting hyperscalars. but it seems like SemiAnalysis got through to them that hyperscalars aren’t going to deal with bad software either. It remains to be seem if they can shift culture to actually deliver good software, but there’s movement in that direction, and if they succeed AMD is so undervalued. Their hardware is good. With Intel, until that committee style leadership is gone, there’s 0 chance for success. Committee leadership is fine if you are trying to maintain, but Intel’s AI situation is even more hopeless than AMDs, and you’d need something major to turn it around. At least with AMD, you can try installing ROCm and be frustrated when there are bugs. Every time I have tried Intel’s software I can’t even recall getting the import to work, and the card wasn’t powerful enough that I cared. Intel needs actual leadership to turn this around, or there’s 0 future in Intel AI.
If you give some monkeys a slice of cucumber each, they are all pretty happy. Then you give one monkey a grape, and nobody is happy with their cucumber any more. They might even throw the slices back at the experimenter. He got a god damned grape this is bullshit I don’t want a cucumber anymore! Nobody was in absolute terms worse off, but that doesn’t prevent the monkeys from being upset. And this isn’t unique to monkeys, I see this same behavior on display when I hear about billionaires. It’s not about what I have, they got a grape. The tweet is here. What do you do about this? Of course, you can fire this women, but what percent of people in American society feel the same way? How much of this can you tolerate and still have a functioning society? What’s particularly absurd about the critique in the video is that it hasn’t been thought through very far. If that house and its friends stopped “ordering shit”, the company would stop making money and she wouldn’t have that job. There’s nothing preventing her from quitting today and getting the same outcome for herself. But of course, that isn’t what it’s about, because then somebody else would be delivering the packages. You see, that house got a grape. So how do we get through this? I’ll propose something, but it’s sort of horrible. Bring people to power based on this feeling. Let everyone indulge fully in their resentment. Kill the bourgeois. They got grapes, kill them all! Watch the situation not improve. Realize that this must be because there’s still counterrevolutionaries in the mix, still a few grapefuckers. Some billionaire is trying to hide his billions! Let the purge continue! And still, things are not improving. People are starving. The economy isn’t even tracked anymore. Things are bad. Millions are dead. The demoralization is complete. Starvation and real poverty are more powerful emotions than resentment. It was bad when people were getting grapes, but now there aren’t even cucumbers anymore. In the face of true poverty for all, the resentment fades. Society begins to heal. People are grateful to have food, they are grateful for what they have. Expectations are back in line with market value. You have another way to fix this? Cause this is what seems to happen in history, and it takes a generation. The demoralization is just beginning.
AMD is sending us the two MI300X boxes we asked for. They are in the mail. It took a bit, but AMD passed my cultural test. I now believe they aren’t going to shoot themselves in the foot on software, and if that’s true, there’s absolutely no reason they should be worth 1/16th of NVIDIA. CUDA isn’t really the moat people think it is, it was just an early ecosystem. tiny corp has a fully sovereign AMD stack, and soon we’ll port it to the MI300X. You won’t even have to use tinygrad proper, tinygrad has a torch frontend now. Either NVIDIA is super overvalued or AMD is undervalued. If the petaflop gets commoditized (tiny corp’s mission), the current situation doesn’t make any sense. The hardware is similar, AMD even got the double throughput Tensor Cores on RDNA4 (NVIDIA artificially halves this on their cards, soon they won’t be able to). I’m betting on AMD being undervalued, and that the demand for AI has barely started. With good software, the MI300X should outperform the H100. In for a quarter million. Long term. It can always dip short term, but check back in 5 years.
More in programming
When interviewing with a Japanese company, you’ll naturally want to know: “Is this a good place to work?” And while Glassdoor is the standard in English-speaking countries for employees leaving online reviews, the site is only rarely used in Japan, and then primarily by non-Japanese workers. Many countries have a culture that endorses directly reviewing employers in an open, public environment—Japan does not. However, there are still sites where you can find important information on your potential employer. What to watch out for In particular, you want to avoid signing on with a company that engages in exploitative practices—or as they’re known in Japan, a “black company” (ブラック企業, burakku kigyou). The Ministry of Health, Labor, and Welfare has a FAQ describing what defines these companies: Imposing extremely long working hours with high quotas. Recognition of workers’ rights is low throughout the company; unpaid overtime and/or workplace bullying (パワハラ, pawahara) are common. The company assigns shifts to workers without consent. The company discriminates among workers in the above circumstances. In a 2023 survey, those who had worked for such toxic companies listed high turnover rates as the most common sign that something was wrong, followed by long working hours and unpaid overtime. As you examine online review sites and other sources, look for clues such as: Turnover rate: how long do employees typically stay? Internal promotion: can you see employees rising in the ranks? Upper management: are there any non-Japanese employees in management positions? Recent company announcements: do they often make sudden pivots in their business policies? If you discover, for example, that the company can’t retain employees, shows no history of internal promotions, and has just issued a return-to-office order out of the blue, it’s safe to assume you don’t want to work there. OpenWork OpenWork, also known as Vorkers, hosts over 19 million company reviews. The reviews are represented in a radar chart for easy visual reference, and are also broken down into different categories, such as work-life balance, the ease of working for women, and reasons for considering quitting. In addition, applicants can post questions for employees to answer. If you don’t speak Japanese, the site is still readable with Google Translate. You’ll need to make a free account to see all of the information, but much of it is accessible even without an account. Other Japanese sites JobTalk and Engage Hyouban are other Japanese-language review sites. JobTalk contains 4.4 million reviews of around 230,000 different companies, and Engage Hyouban boasts 30 million reviews for 220,000 companies. Neither of these sites offer as much information on tech companies in Japan as OpenWork does. If you’re applying to a large company such as Rakuten, you may find some additional reviews there, but many of TokyoDev’s clients are smaller companies that aren’t listed at all. Google Maps Reviews An unusual but occasionally helpful place to find company reviews is on Google Maps. If you search for a business’s main corporate office location—usually in Tokyo—you will sometimes find reviews written by current or former employees. Whether these reviews are high-quality or trustworthy is another matter. Rakuten, for example, has reviews with a range of opinions. Cybozu, by contrast, mostly has reviews from those who would like to work for the company but currently don’t. Still, the reviews of its corporate office are consistently positive, so you can at least get an impression of the physical environment. LinkedIn “If you’re worried that a company might be a poor place to work, try contacting current or past employees via LinkedIn,” suggested Paul McMahon, founder of TokyoDev. “This probably works best if you’re late in the hiring process.” You can send a connect request saying, ‘I’ve received an offer from company X, and want to confirm what it’s really like to work there as an engineer. Mind if I ask you a couple of questions?’ Whether or not they respond, you can still glean good information from the profiles of past and current employees. Check to see if developers tend to leave the company quickly, for example, or how long the average employee goes before being promoted. You should keep in mind though that LinkedIn is not popular in Japan, for several good reasons. If you are applying to a primarily Japanese company, many of your future coworkers won’t be active there, which means you still may not be getting a complete picture. TokyoDev In 2020, TokyoDev began interviewing developers in order to provide a more complete, boots-on-the-ground picture of daily life at specific companies. Our Developer Stories feature interviews with developers at top Japanese tech companies, who share details about both their specific jobs and the general work environment. The goal is to give applicants a good sense of how a company operates on a day-to-day basis, from the perspective of those on the inside. So far, TokyoDev has interviewed developers from Mercari, PayPay, Givery, HENNGE, KOMOJU, and more. In addition, TokyoDev’s job board is a selective one, listing only companies that we feel good about sending applicants to. In the rare event that employees later reach out with poor reviews of a business, if those reports can be confirmed, then TokyoDev will end its relationship with that company. Conclusion In short, the answer to the question “Is there a Japanese equivalent to Glassdoor?” is, “Not really.” However, by combining some of the alternatives—OpenWork, LinkedIn, TokyoDev, and perhaps even Google Maps—you can gather enough information to decide whether you want to work with a particular Japanese company. You could also ask fellow developers in our Discord. Curious about working in Japan in general? See our articles on the subject, as well as moving to Japan, living in Japan, starting a business in Japan, and more.
Our battle with Apple over their gangster attempt to extort 30% of our HEY revenues was one of the defining moments of my career. It was the kind of test that calls you to account for what you believe and asks what you're willing to risk to see it through. Well, we risked everything, but also secured a four-year truce, and now near-total victory is at hand: HEY is finally for sale on the iPhone in the US! Credit for this amazing turn of events goes to Epic Games founders Tim Sweeney and Mark Rein, who did what no small developer like us could ever dream of doing: they spent over $100 million to sue Apple in court. And while the first round yielded very little progress, Apple's (possibly criminal) contempt of court is what ultimately delivered the resolution. Thanks to their fight for Fortnite, app developers everywhere are now allowed to link out of apps to their own web-based payment system in the US store (but, sadly, nowhere else yet). This is all we ever wanted from Apple: to have a way to distribute our iPhone apps and keep the customer relationship by billing directly. The 30% toll gets all the attention, and it is ludicrously egregious, but to us, it's just as much about retaining that direct customer relationship, so we can help folks with refunds, so they don't tie their billing for a multi-platform email system to a single manufacturer. Apple always claims to put the needs of the users first, and that whatever hardship developers have to carry is justified by their customer-focused obsession. But in this case, it's clear that the obsession was with collecting the easiest billions Apple has ever made, by taking an obscene cut of all software and subscription sales on the platform. This obsession with squeezing every last dollar from developers has produced countless customer-hostile experiences on the iPhone. Like how you couldn't buy a book in the Kindle app before this (now you can!). Or sign up for a Netflix subscription (now you can!). Before, users would hunt in vain for an explanation inside these apps, and thanks to Apple's gag orders, developers were not even allowed to explain the confusing situation. It's been the same deal with HEY. While we successfully fought off Apple's attempt to extort us into using their in-app payment system (IAP), we've been stuck with an awkward user experience ever since. One that prevented new customers from signing up for a real email address in the application, and instead sent them down this bizarre burner-account setup. All so the app would "do something", in order to please an argument that App Store chief Phil Schiller made up on the fly in an interview. That's what we can now get rid of. No more weird burner accounts. Now you can sign up directly for a real email address in HEY, and if you like what we have to offer (and I think you will!), you'll be able to pay the $99/year for a subscription via a web-based flow that it's now kosher to link to from the app itself. What a journey, and what a needless torching of the developer relationship from Apple's side. We've always been happy to pay Apple for hosting our application on the App Store, as all developers have always needed to do via the $99/year developer fee. But being forced to hand over 30% of the business, as well as the direct customer relationship, was always an unacceptable overreach. Now that's been arrested by Judge Yvonne Gonzalez Rogers from the United States District Court of Northern California, who has delivered app developers the only real relief that we've seen in this whole sordid monopoly affair that's been boiling since 2020. It's a beautiful thing. It also offers Apple an opportunity to bury the hatchet with developers. They can choose to accept the court's decision in full and worldwide. Allow developers everywhere the right to link to their own billing flow, so they can retain their own customer relationship, and so business models that can't carry a 30% toll can flourish. Besides, Apple's own offering will likely still have plenty of pull. I'm sure many small developers would continue to consider IAP to avoid having to worry about international taxes or even direct customer service. Nobody is taking that away from Apple or those developers. All Judge Rogers is demanding is that Apple compete fairly with alternative arrangements. In case Apple doesn't accept the court's decision — and there's sadly some evidence to that — I hope the European antitrust regulators watch the simple yet powerful mechanism that Judge Rogers has imposed on Apple. While I'd love side loading as much as the next sovereign techie who wants to own the hardware I buy, I think we can get the lion's share of independence by simply being allowed to link out of the apps, just like has been so ordered by this District Court. I do hope, though, that Apple does accept the court's decision. Both because it would be a stain on their reputation to get convicted of criminal contempt of court, but also because I really want Apple to return to being a shining city on the hill. To show that you can win in the market merely by making better products. Something Apple never used to be afraid of doing. That they don't need these gangster extortion techniques to make the numbers that Cook has promised Wall Street. Despite moving on to Linux and Android, I have a real soft spot for Apple's taste, aesthetics, and engineering prowess. They've lost their way and moral compass over the last half decade or so, but that's only one leadership pivot away from being found again. That won't win back all the trust and good faith that was squandered right away, but they'll at least be on the long road to recovery. Who knows, maybe developers would even be inclined to assist Apple next time they need help launching a new device in need of third-party software to succeed.
In his post about “Vibe Drive Development”, Robin Rendle warns against what I’ll call the pseudoscientific approach to product building prevalent across the software industry: when folks at tech companies talk about data they’re not talking about a well-researched study from a lab but actually wildly inconsistent and untrustworthy data scraped from an analytics dashboard. This approach has all the theater of science — “we measured and made decisions on the data, the numbers don’t lie” etc. — but is missing the rigor of science. Like, for example, corroboration. Independent corroboration is a vital practice of science that we in tech conveniently gloss over in our (self-proclaimed) objective data-driven decision making. In science you can observe something, measure it, analyze the results, and draw conclusions, but nobody accepts it as fact until there can be multiple instances of independent corroboration. Meanwhile in product, corroboration is often merely a group of people nodding along in support of a Powerpoint with some numbers supporting a foregone conclusion — “We should do X, that’s what the numbers say!” (What’s worse is when we have the hubris to think our experiments, anecdotal evidence, and conclusions should extend to others outside of our own teams, despite zero independent corroboration — looking at you Medium articles.) Don’t get me wrong, experimentation and measurement are great. But let’s not pretend there is (or should be) a science to everything we do. We don’t hold a candle to the rigor of science. Software is as much art as science. Embrace the vibe. Email · Mastodon · Bluesky
Explore Remix's new React Server Components (RSC) preview in react-router! Learn usage, different approaches, and trade-offs.
Have you thought about doing the opposite of whatever you're doing or considering? It's a really helpful way to test your assumptions and your values. What does the opposite look like, how would it work? It's so easy to get stuck in a groove of what works, what you believe to be right. But helpful assumptions have a half-life, just like facts. And it's ever so easy to miss the shift when circumstances change, if you're not regularly stress-testing your core beliefs. That doesn't mean you're just a flag in the wind, blowing whichever way. But it does mean having enough intellectual humility and creative flexibility to consider that what you believe to be true about your business, about your team, about your technology might not be so. We did this a while back with full-time managers. We'd been working for nearly two decades without any, but exactly because it'd been so long, we were drawn to try the opposite, just to see what we might have missed. So we did. Hired a few full-time managers to help us test that assumption for a few years. In the end, we decided that our managers-of-one culture worked better, but it wasn't a given at the outset. To try the opposite, you really have to believe that you might have been wrong. Because you're wrong about something. I guarantee it. We all are.