More from macwright.com
I was just enjoying Simon Willison’s predictions and, heck, why not. 1: The web becomes adversarial to AI The history of search engines is sort of an arms race between websites and search engines. Back in the early 2000s, juicing your ranking on search engines was pretty easy - you could put a bunch of junk in your meta description tags or put some text with lots of keywords on each page and make that text really tiny and transparent so users didn’t notice it but Google did. I doubt that Perplexity’s userbase is that big but Perplexity users are probably a lot wealthier on average than Google’s, and there’s some edge to be achieved by getting Perplexity to rank your content highly or recommend your website. I’ve already noticed some search results including links to content farms. There are handful of startups that do this already, but the prediction is: the average marketing exec at a consumer brand will put some of their budget to work on fooling AI. That means serving different content to AI scrapers, maybe using some twist on Glaze and other forms of adversarial image processing to make their marketing images more tantalizing to the bots. Websites will be increasingly aware that they’re being consumed by AI, and they will have a vested interest in messing with the way AI ‘perceives’ them. As Simon notes in his predictions, AIs are gullible: and that’s before there are widespread efforts to fool them. There’s probably some way to detect an AI scraper, give it a special payload, and trick it into recommending your brand of razors whenever anyone asks, and once someone figures it out this will be the marketing trend of the decade. 2: Copyright nihilism breeds a return to physical-only media The latest lawsuit about Meta’s use of pirated books, allegedly with Mark Zuckerberg’s explicit permission, if true, will be another reason to lose faith in the American legal system’s intellectual property system entirely. We’ve only seen it used to punish individuals and protect corporations, regardless of the facts and damages, and there’s no reason to believe it will do anything different (POSIWID). The result, besides an uptick in nihilism, could be a rejuvenation of physical-only releases. New albums only released on vinyl. Books only available in paperback format. More private screenings of hip movies. When all digital records are part of the ‘training dataset,’ a niche, hipster subset will be drawn to things that aren’t as easily captured and reproduced. This is parallel, to the state of closed-source models from Anthropic or OpenAI. They’re never distributed or run locally. They exist as bytes on some hard drive and in some massive GPU’s memory in some datacenter, and there aren’t Bittorrents pirating them because they’re kept away from people, not because of the power of copyright law. What can be accessed can be copied, so secrecy and inaccessibility is valuable. 3: American tech companies will pull out of Europe because they want to do acquisitions The incoming political administration will probably bring an end to Lina Khan’s era of the FTC, and era in which the FTC did stuff. We will go back to a ‘hands off’ policy in which big companies will acquire each other pretty often without much government interference. But, even in Khan’s era, the real nail in the coffin for one of the biggest acquisitions - Adobe’s attempt to buy Figma – was regulators from the EU and UK. Those regulators will probably keep doing stuff, so I think it’s likely that the next time some company wants to acquire a close competitor, they just close up shop in the EU, maybe with a long-term plan to return. 4: The tech industry’s ‘DEI backlash’ will run up against reality The reality is that the gap between women and men in terms of college degrees is really big: “Today, 47% of U.S. women ages 25 to 34 have a bachelor’s degree, compared with 37% of men.” And that a great deal of the tech industry’s workforce is made of up highly-skilled people who are on H-1B visas. The synthesis will be that tech workers will be more diverse, in some respects, but by stripping away the bare-bones protections around their presence, companies will keep them in a more vulnerable and exploitable position. But hard right-wingers will have plenty to complain about because these companies will continue to look less white and male, because the labor pool is not that. 5: Local-first will have a breakthrough moment I think that Zero Sync has a good chance at cracking this really hard problem. So does electric and maybe jazz, too. The gap between the dream of local-first apps and the reality has been wide, but I think projects are starting to come to grips with a few hard truths: Full decentralization is not worth it. You need to design for syncing a subset of the data, not the entire dataset. You need an approach to schema evolution and permission checking These systems are getting there. We could see a big, Figma-level application built on Zero this year that will set the standard for future web application architecture. 6: Local, small AI models will be a big deal Embedding models are cool as heck. New text-to-speech and speech-to-text models are dramatically better than what came before. Image segmentation is getting a lot better. There’s a lot of stuff that is coming out of this boom that will be able to scale down to a small model that runs on a phone, browser, or at least on our own web servers without having to call out to OpenAI or Anthropic APIs. It’ll make sense for costs, performance, and security. Candle is a really interesting effort in this area. Mini predictions Substack will re-bundle news. People are tired of subscribing to individual newsletters. Substack will introduce some ~$20/month plan that gives you access to all of the newsletters that participate in this new pricing model. TypeScript gets a zeitwork equivalent and lots of people use it. Same as how prettier brought full code formatting from TypeScript, autoloading is the kind of thing that once you have it, it’s magic. What if you could just write <SomeComponent /> in your React app and didn’t have to import it? I think this would be extremely addictive and catch on fast. Node.js will fend off its competitors. Even though Val Town is built around Deno’s magic, I’ve been very impressed that Node.js is keeping up. They’ve introduced permissions, just like Deno, and native TypeScript support, just like the upstarts. Bun and Deno will keep gaining adherents, but Node.js has a long future ahead of it. Another US city starts seriously considering congestion pricing. For all the chatter and terrible discourse around the plan, it is obviously a good idea and it will work, as it has in every other case, and inspire other cities to do the same. Stripe will IPO. They’re still killing it, but they’re killing it in an established, repeatable way that public markets will like, and will let up the pressure on the many, many people who own their stock.
Happy end-of-2024! It’s been a pretty good year overall. I’m thankful. There’s no way that I’ll be able to remember and carve out the time around New Years to write this, so here’s some end-of-year roundup, ahead of schedule! Running This was my biggest year for running on record: 687 miles as of today. I think the biggest difference with this year was just that nothing stood in the way of my being pretty consistent and putting in the miles: the weather has been mild, I haven’t had any major injuries, and long runs have felt pretty good. I was happy to hit a half-marathon PR (1:36:21), but my performance in 5Ks was far short of the goal of sub-20 – partly because Brooklyn’s wonderful 5K series was run at the peak of summer, with multiple races at over 85°F. I learned the value of good lightweight running gear: Bakline’s singlets and Goodr sunglasses were super helpful in getting me through the summer. Work Val Town raised a seed round and hired a bunch of excellent people. We moved into a new office of our own, which has a great vibe. It’s been good: we’re doing a lot of ground-up work wrangling cgroups and low-level worker scheduling, and a lot of UX-in work, just trying to make it a pleasant tool. Frankly, with every product I’ve worked on, I’ve never had a feeling that it was good enough, and accordingly, for me, Val Town feels like it has a long way to go. It’s probably a good tendency to be sort of unsatisfied and motivated to constantly improve. New York It’s still such a wonderful place to live. Late this year, I’ve been rediscovering my obsession with cycling, and realizing how much I whiffed the opportunity to ride more when I lived in San Francisco. I guess that’s the first time I felt genuinely nostalgic for the West coast. I miss DC a bit too: it’s one of the few cities where my friends have been able to stay in the city proper while raising children, and I miss the accessible, underdog punk scene. But Brooklyn is just a remarkable place to live. My walk score is 100. The degree to which people here are in the city because they want to be, not because they have to, shapes so much of what makes it great. Other ‘metrics’ Relative to my old level obsession about self-quantification, my ‘metrics’ are pretty moderate now. Everything’s just backward-looking: I’m not paying much attention to the numbers as I go, it’s just fun to look at them year-over-year trends. That said, this was a lackluster year for reading: just 18 books so far. I think I just read an above-average number of books that I didn’t enjoy very much. Next year I’m going to return to authors who I already love, and stay away from genres that – the data shows – I don’t like. Whereas this was a banner year for watching movies: not great! Next year, I want to flip these results. Of everything I saw, Kinds of Kindness will probably stick with me the most. Placemark It seems like a decade ago that I released Placemark as open source software, as developing it as a closed-source SaaS application for a few years. But I did that in January. There have been a few great open source contributions since then, but it’s pretty quiet. Which is okay, somewhat expected: there is no hidden crowd of people with extra time on their hands and unending enthusiasm for ‘geospatial software’ waiting to contribute to that kind of project. Placemark is also, even with my obsessive focus on simplicity, a pretty complicated codebase. The learning curve is probably pretty significant. Maps are a challenging problem area: that’s what attracts a lot of people to them, but people who use maps persistently have the feeling that it couldn’t be that complicated, which means that few users convert into contributors. There are a few prominent efforts chasing similar goals as Placemark: Atlas.co is aiming to be an all-in-one editing/analysis platform, Felt a cloud-native GIS platform, and then there are plenty of indiehackers-style projects. I hope these projects take off! Figma plugins I also kept maintaining the Figma plugins I developed under the Placemark name. Potentially a lot of people are using them, but I don’t really know. The problem with filling in water shapes in the plugins is still unsolved: it’s pretty hard and I haven’t had the time or motivation to fix it. The most energy into those plugins this year, unfortunately, was when someone noticed that the dataset I was using - Natural Earth – marked Crimea as part of Russia. Which obviously: I don’t draw the countries in datasets, but it’s a reasonable thing to point out (but to assume that the author is malicious was a real downer, again, like, I don’t draw the countries). This decision from Natural Earth’s maintainer is heavily discussed and they aren’t planning on changing it, so I switched to world-atlas, which doesn’t have that problem. Which was fine, but a reminder of the days when I worked on maps full-time and this kind of unexpected “you’re the baddie” realization came up much more often. Sometimes it was silly: people who complain about label priority in the sense of “why, at zoom level 3, does one country’s name show up and not anothers?” was just silly. The answer, ahem, was that there isn’t enough space for the two labels and one country had a higher population or a geometry that gave their label more distance from the other country’s centroid. But a lot of the territorial disputes are part of people’s long cultural, political, military history and the source of intergenerational strife. Of course that’s serious stuff. Making a tool that shows a globe with labels on it will probably always trigger some sort of moment like that, and it’s a reason to not work on it that much because you’re bound to unintentionally step on something contentious. Other projects I released Obsidian Freeform, and have been using it a bit myself. Obsidian has really stuck for me. My vault is well over 2,000 notes, and I’ve created a daily note for almost every day for the last year. Freeform was a fun project and I have other ideas that are Obsidian plugin-shaped, though I’ve become a little bit let down by the plugin API - the fact that Obsidian-flavored-Markdown is nonstandard and the parser/AST is not accessible to plugins is a pretty big drawback for the kinds of things I want to build. Elsewhere recently I’ve been writing a bit: Recently I’ve written about dependency bloat and a developer analytics tool we built at Val Town, and started writing some supplementary documentation for Observable Plot about parts of its API that I think are unintuitive. On the micro blog, I wrote about not using GitHub Copilot and how brands should make a comeback. This blog got a gentle redesign in May, to show multiple categories of posts on the home page, and then in August I did a mass update to switch all YouTube embeds to lite-youtube-embed to make pages load faster. I’m still running Jekyll, like I have been for the last decade, and it works great. Oh, and I’ve basically stopped using Twitter and am only on Mastodon and Bluesky. Bluesky more than Mastodon recently because it seems like it’s doing a better job at attracting a more diverse community. I’m looking forward to 2025, to cycling a lot more and a new phase of startup-building. See you in the new year.
I still use Bandcamp almost exclusively to buy music, and keep a big library of MP3s. The downside is that this marks me as a weirdo, but otherwise it’s great and has been working well for me. Since I last wrote about it, Bandcamp was acquired by Epic games (?) and then acquired from them by Songtradr, and its employees are trying to get recognized as a union. Times are changing and Bandcamp is no longer a lovely indie company, but it’s still a heck of a lot better than Spotify. People (who?) are sharing their ‘Spotify wrapped’ auto-generated compilations and I wanted the same, for my Bandcamp purchases, so I built it on Val Town. You can create your own! Or edit the code of the tool that generates them. Because of API limitations – really, the absence of an API – it requires you to copy & paste content from your purchases page, but isn’t copy-and-paste really a kind of API? Anyway: Vampire Empire / Born For Loving You by Big Thief Patterns by Pool Boys Acadia by Yasmin Williams Cascade by Floating Points of course i still love you by Darwin Deez (pre-order) 4 | 2 | 3 by MIZU Son by Rosie Lowe & Duval Timothy Imaginal Disk by Magdalena Bay Dirty Projectors by Dirty Projectors Green Disco by Justine Electra Daedalus by Daedelus You Look A Lot Like Me (2016) by Mal Blum Big City Boys by Cailin Pitt Promises by Floating Points, Pharoah Sanders & The London Symphony Orchestra Windswept by Photay Jessie Mae Hemphill by Jessie Mae Hemphill Rituals by Ishmael Ensemble 1992 - 2001 by Acetone Final Summer by Cloud Nothings Bright Future by Adrianne Lenker La Forêt (2024) by Xiu Xiu Frog Poems by Mister Goblin Living is Easy by Agriculture Again by Oneohtrix Point Never Put The Shine On by CocoRosie The Light Is On You Return by Ben Levin Mercurial World by Magdalena Bay Burn It Down by Lovebirds Room 25 by Noname Wall Of Eyes by The Smile Forest Scenes by MIZU Looking back on the year, I like how I can remember a few of these albums from my first exposure to them in odd places - I heard Jessie Mae Hemphill playing in a Chipotle, and Rosie Lowe playing in my hair salon. It was apparently a big year for instrumental, electronic, minimalist music. The only ‘rock’ album that hooked me was Wall of Eyes, and the only pop album that made an impact was Imaginal Disk - the fuzzy outro of Image is something I keep re-listening to. MIZU has been on heavy rotation, too – the only of these artists that I learned about by seeing them live - she opened for Tim Hecker and I think made a lot of fans there with a really theatric and heavy performance. Buy some music! Listen to it repeatedly, and put it in your MP3 player!
I haven’t been posting much to the ‘main blog’ recently, but I have been keeping the micro blog updates humming. If you want more content in your RSS reader, you can subscribe to those posts, which are shorter, more scattered, and even less copyedited. It feels bad to have multiple “Recently” headings in the blog listing, so I’ll give them short subtitles from now on. Anyway, what’s up? October was all right. At Val Town, we spent a lot of time interviewing job candidates and improving the AI assistant, Townie. I also got some time to tackle long-awaited technical debt cleanups: I conquered the ‘big scary function’ that did the actual ‘running’ of val code. Cycling Outside of work, a lot of my October-related excitement was related to being outdoors. It’s been a great year for running – I just passed 600 miles so far and will probably hit 650 barring any injuries or life complications. But cycling is on the mind. We just rode the Old Croton Aqueduct trail from Ossining back to Brooklyn. It’s a fairly rough trail: plenty of rocks and terrain. Rideable on my ~32mm tires, but it’d be a lot easier with a mountain bike. We rode past some osage orange trees with their funky-looking and inedible fruit the size of large grapefruits. The trail passed right next to the Lyndhurst Estate, which was owned by a series of rich and powerful people, including Jay Gould, who is especially hated. Upstate, a lot of the attractions are like this, other big historic houses. The trail was mostly really beautiful, though the parts closer to Yonkers have a lot of trash. It’s much more popular with hikers than with cyclists. Even though bikes are explicitly permitted, locals seemed a little surprised by our presence, even though we were ringing bells, going slow, and making lots of space. It’s kind of funny to compare the general spatial awareness of people upstate to those in the city: we encountered a lot of people upstate who were standing in the center of the trail, completely zoned out and surprised by the presence of another human, and then on the way back were on city streets with four people within a few feet of us on foot, bikes, cars, scooters, all mostly aware and ready to silently negotiate how to move together through a shared space. I remarked that I think that when some people move out of the city because of the ‘inconvenience’, the inconvenience is people, and once you leave, you lose a certain ability to live around other people - from then on, you expect to have a suburban yard-sized perimeter around your personal space. Micro I wrote a lot on the microblog this month: about the Arc browser’s recent news that it’ll be abandoned, Reddit adopting Web Components, domain squatting, Python datascience tech, and Knip, a tool for finding dead code in TypeScript systems. Content I watched a bunch of films, which are on my Letterboxd, and the only new album on my rotation is Yasmin Williams’s Acadia: Acadia by Yasmin Williams This YouTube channel is showing all of the steps involved in doing a multi-day bikepacking trip through India. It’s a lot of fun: And that’s it for this month! I’ll write a full-fledged blog post one of these days.
More in programming
I occasionally receive emails asking me to look at the writer's new language/library/tool. Sometimes it's in an area I know well, like formal methods. Other times, I'm a complete stranger to the field. Regardless, I'm generally happy to check it out. When starting out, this is the biggest question I'm looking to answer: What does this technology make easy that's normally hard? What justifies me learning and migrating to a new thing as opposed to fighting through my problems with the tools I already know? The new thing has to have some sort of value proposition, which could be something like "better performance" or "more secure". The most universal value and the most direct to show is "takes less time and mental effort to do something". I can't accurately judge two benchmarks, but I can see two demos or code samples and compare which one feels easier to me. Examples Functional programming What drew me originally to functional programming was higher order functions. # Without HOFs out = [] for x in input { if test(x) { out.append(x) } } # With HOFs filter(test, input) We can also compare the easiness of various tasks between examples within the same paradigm. If I know FP via Clojure, what could be appealing about Haskell or F#? For one, null safety is a lot easier when I've got option types. Array Programming Array programming languages like APL or J make certain classes of computation easier. For example, finding all of the indices where two arrays differ. Here it is in Python: x = [1, 4, 2, 3, 4, 1, 0, 0, 0, 4] y = [2, 3, 1, 1, 2, 3, 2, 0, 2, 4] >>> [i for i, (a, b) in enumerate(zip(x, y)) if a == b] [7, 9] And here it is in J: x =: 1 4 2 3 4 1 0 0 0 4 y =: 2 3 1 1 2 3 2 0 2 4 I. x = y 7 9 Not every tool is meant for every programmer, because you might not have any of the problems a tool makes easier. What comes up more often for you: filtering a list or finding all the indices where two lists differ? Statistically speaking, functional programming is more useful to you than array programming. But I have this problem enough to justify learning array programming. LLMs I think a lot of the appeal of LLMs is they make a lot of specialist tasks easy for nonspecialists. One thing I recently did was convert some rst list tables to csv tables. Normally I'd have to do write some tricky parsing and serialization code to automatically convert between the two. With LLMs, it's just Convert the following rst list-table into a csv-table: [table] "Easy" can trump "correct" as a value. The LLM might get some translations wrong, but it's so convenient I'd rather manually review all the translations for errors than write specialized script that is correct 100% of the time. Let's not take this too far A college friend once claimed that he cracked the secret of human behavior: humans do whatever makes them happiest. "What about the martyr who dies for their beliefs?" "Well, in their last second of life they get REALLY happy." We can do the same here, fitting every value proposition into the frame of "easy". CUDA makes it easier to do matrix multiplication. Rust makes it easier to write low-level code without memory bugs. TLA+ makes it easier to find errors in your design. Monads make it easier to sequence computations in a lazy environment. Making everything about "easy" obscures other reason for adopting new things. That whole "simple vs easy" thing Sometimes people think that "simple" is better than "easy", because "simple" is objective and "easy" is subjective. This comes from the famous talk Simple Made Easy. I'm not sure I agree that simple is better or more objective: the speaker claims that polymorphism and typeclasses are "simpler" than conditionals, and I doubt everybody would agree with that. The problem is that "simple" is used to mean both "not complicated" and "not complex". And everybody agrees that "complicated" and "complex" are different, even if they can't agree what the difference is. This idea should probably expanded be expanded into its own newsletter. It's also a lot harder to pitch a technology on being "simpler". Simplicity by itself doesn't make a tool better equipped to solve problems. Simplicity can unlock other benefits, like compositionality or tractability, that provide the actual value. And often that value is in the form of "makes some tasks easier".
Ask an engineering leader about their incident response protocol and they’ll tell you about their severity scale. “The first thing we do is we assign a severity to the incident,” they’ll say, “so the right people will get notified.” And this is sensible. In order to figure out whom to get involved, decision makers need … Continue reading Incident SEV scales are a waste of time
Thou shalt not suffer a flaky test to live, because it’s annoying, counterproductive, and dangerous: one day it might fail for real, and you won’t notice. Here’s what to do.
The ware for January 2025 is shown below. Thanks to brimdavis for contributing this ware! …back in the day when you would get wares that had “blue wires” in them… One thing I wonder about this ware is…where are the ROMs? Perhaps I’ll find out soon! Happy year of the snake!
Explore how JSDOM's browser simulation works, and learn front-end testing approaches using Vitest Browser Mode for direct browser testing and native APIs