Full Width 
Shortcuts 
Sign Up 
Log In 
Articles 
Reading List More from tonsky.me
Каким должен быть редактор кода в 2024 году? Почему Vim морально устарел, а IDEA, кажется, сдает позиции? Популярность Zed, минимализм SublimeText, гибкость Emacs и многое другое в новом выпуске.
Squint is a light-weight dialect of ClojureScript with a compiler and standard library. “The idea is that when you squint, it still looks like CLJS”.
There’s a wonderful article by Sebastian Bensusan: “We need visual programming. No, not like that.” (the dot is part of the title ¯\_(ツ)_/¯). In it, Sebastian argues that we shouldn’t try to replace all code with visual programming but instead only add graphics where it makes sense: Most visual programming environments fail to get any usage. Why? They try to replace code syntax and business logic but developers never try to visualize that. Instead, developers visualize state transitions, memory layouts, or network requests. In my opinion, those working on visual programming would be more likely to succeed if they started with aspects of software that developers already visualize. I love diagrams myself! Whenever I encounter a complicated task and try to solve it in code, it always gets messy. But after drawing a diagram, my understanding improves, and the code gets cleaner. Win-win! Here’s one I made for button states in Humble UI: I bet you thought buttons are easy? Me too, at first. But after certain threshold your head just can’t fit all the states and transitions. Or for an image upload component: Again: it would’ve been easy if not for error handling. But with a principled approach, you can get through any of that. Sebastian gives many more examples of useful visualizations in his article, too. But now, how does all this relate to code? I think there’re four levels. Level 0: Diagrams live separately You draw them in a separate tool, then use that to help you write code. Maybe put them on a wiki for other people to see. The point is: the diagram lives completely separate from the code. Downsides: hard to discover, can get out of date. This is what I did in the two examples above, and I guess what most of us can do given modern tools. But hey—it’s still not that bad! Level 1: Diagrams live next to code One simple trick would solve the problem of discovery: what if we could put images into our text files? Currently, the best you can do is this: +-----+ --> | N_4 |------ <--- +-----+ +-----+ | |-----| R_3 | | 15 | | 5 +-----+ |50 | | | +-----+ ---> | +-----+ | 70 | N_2 |------ | | N_3 | | +-----+ | | +-----+ | | 15 | | | 30 | | 10 | +-----+ <--- | | @ | ----| S |--------| | @ | <@@@ +-----+ | V | | | | | 10 | | | +-----+ | V | | R_2 | +-----+ | +-----+ | E | | | | +-----+ | | | 40 | | | V | 10 | | | | +-----+ | V | -----| R_1 |-----| | +-----+ | | ---> +-----+ | |------------------| D |--------- 10 +-----+ But it gets messy real quick. What if we could do this instead? Upsides: easy to implement (once everybody agrees on how to do that), universal (probably many other use cases). Downsides: still can get out of date. “Comments are not code”—the same applies here. Oh, and if you are coding in a terminal, this party is not for you. Sorry. We are thinking about the future here. Level 2: Diagrams are generated from code This is what Sebastian was hinting at. Code and diagrams co-exist, one is generated from the other. Generating diagrams from code is definitely something IDEs can do: Upsides: Always up to date. Non-invasive: can be integrated into IDE without affecting how code is stored. Downsides: It can help you understand, but can it help you think? Probably not very visually appealing, as these things tend to be. It’s hard to automatically lay out a good diagram. Level 3: Diagrams are code This is what the endgame should be IMO. Some things are better represented as text. Some are best understood visually. We should mix and match what works best on a case-by-case basis. Don’t try to visualize simple code. Don’t try to write code where a diagram is better. One of the attempts was Luna. They tried dual representation: everything is code and diagram at the same time, and you can switch between the two: From luna-lang.org But this way, you are not only getting benefits of both ways, you are also constrained by both text and visual media at the same time. You can’t do stuff that’s hard to visualize (loops, recursions, abstractions) AND you can’t do stuff that’s hard to code. No, I think textual coding should stay textual where it works, BUT we should also be able to jump into a diagram tool, draw a state machine there and execute it the same way we execute text code. And when I mean draw, I mean draw. With direct manipulation, all that jazz. And without converting it back to text. So what I’m saying is: diagrams should not replace or “augment” text. They should be just another tool that lives next to the text. But a tool on its own. Think of it as a game engine like Godot or Unity. In them, you can write normal text code, but you can also create and edit scenes. These scenes are stored in their own files, have specialized editors that know how to edit them, and have no code representation. Because why? The visual way in this particular case is better. So the challenge here is not about integrating diagrams, but to think about which types of diagrams can be useful, can work better than code, and be directly executed. Non-goal: Diagrams replace code Important note: we are not talking about doing code graphically. This is just a less convenient way of doing things that text already does. We are also not talking about no-code platforms: sometimes code is just better. But until this bright future arrives, put a diagram or two on the wiki. Your teammates will thank you for that.
So I was at the Local-First Conf the other day, listening to Martin Kleppmann, and this slide caught my attention: Specifically, this part: But first, some context. What is local-first? For the long version, go to Ink & Switch, who coined the term. Or listen for Peter van Hardenberg explaining it on LocalFirst.fm. Here’s my short version: It’s software. That prefers keeping your data local. But it still goes to the internet occasionally to sync with other users, fetch data, back up, etc. If it doesn’t go to the internet at all, it’s just local software. If it doesn’t work offline with data it already has, then it’s just normal cloud software. You all know the type — sorry, Dave, I can’t play the song I just downloaded because your internet disappeared for one second... But somewhere in the middle — local-first. We love it because it’s good for the end user, you and me, not for the corporations that produce it. What’s the problem with local-first? The goal of local-first software is to get control back into the hands of the user, right? You own the data (literally, it’s on your device), yada-yada-yada. That part works great. However, local-first software still has this online component. For example, personal local-first software still needs to sync between your own devices. And syncing doesn’t work without a server... So here we have a problem: somebody writes local-first software. Everybody who bought it can use it until the heat death of the universe. They own it. But if the company goes out of business, syncing will stop working. And companies go out of business all the time. What do we do? Cue Dropbox The solution is to use something widely available that will probably outlive our company. We need something popular, accessible to everyone, has multiple implementations, and can serve as a sync server. And what’s the most common end-user application of cloud sync? Dropbox! Well, not necessarily Dropbox, but any cloud-based file-syncing solution. iCloud Drive, OneDrive, Google Drive, Syncthing, etc. It’s perfect — many people already have it. There are multiple implementations, so if Microsoft or Apple go out of business, people can always switch to alternatives. File syncing is a commodity. But file syncing is a “dumb” protocol. You can’t “hook” into sync events, or update notifications, or conflict resolution. There isn’t much API; you just save files and they get synced. In case of conflict, best case, you get two files. Worst — you get only one :) This simplicity has an upside and a downside. The upside is: if you can work with that, it would work everywhere. That’s the interoperability part from Martin’s talk. The downside is: you can’t do much with it, and it probably won’t be optimal. But will it be enough? Version 1: Super-naive Let’s just save our state in a file and let Dropbox sync it (in my case, I’m using Syncthing, but it’s the same idea. From now on, I’ll use “Dropbox” as a common noun). Simple: But what happens if you change the state on two machines? Well, you get a conflict file: Normally, it would’ve been a problem. But it’s not if you are using CRDT! CRDT is a collection of data types that all share a very nice property: they can always be merged. It’s not always the perfect merge, and not everything can be made into a CRDT, but IF you can put your data into a CRDT, you can be sure: all merges will go without conflicts. With CRDT, we can solve conflicts by opening both files, merging states, and saving back to state.xml. Simple! Even in this form, Dropbox as a common sync layer works! There are some downsides, though: conflicting file names are different between providers, some providers might not handle conflicts at all, it needs state-based CRDT. Version 2: A file per client The only way to avoid conflicts is to always edit locally. So let’s give each client its own file! Now we just watch when files from other clients get changed and merge them with our own. And because each file is only edited on one machine, Dropbox will not report any conflicts. Any conflicts inside the data will be resolved by us via CRDT magic. Version 3: Operations-based What if your CRDT is operation-based? Meaning, it’s easier to send operations around, not the whole state? You can always write operations into a separate append-only file. Again, each client only writes to its own, so no conflicts on the Dropbox level: Now, the operations log can grow quite long, and we can’t count on Dropbox to reliably and efficiently sync only parts of the file that were updated. In that case, we split operations into chunks. Less work for Dropbox to sync and less for us to catch up: You can, of course, save the position in the file to only apply operations you haven’t seen. Basic stuff. Theoretically, you should be able to do operational transformations this way, too. Demo A very simple proof-of-concept demo is at github.com/tonsky/crdt-filesync. Here’s a video of it in action: Under the hood, it uses Automerge for merging text edits. So it’s a proper CRDT, not just two files merging text diffs. Conclusion If you set out to build a local-first application that users have complete control and ownership over, you need something to solve data sync. Dropbox and other file-sync services, while very basic, offer enough to implement it in a simple but working way. Sure, it won’t be as real-time as a custom solution, but it’s still better for casual syncs. Think Apple Photos: only your own photos, not real-time, but you know they will be everywhere by the end of the day. And that’s good enough! Imagine if Obsidian Sync was just “put your files in the folder” and it would give you conflict-free sync? For free? Forever? Just bring your own cloud? I’d say it sounds pretty good.
More in programming
Last year I wrote about inlining just the fast path of Lemire’s algorithm for nearly-divisionless unbiased bounded random numbers. The idea was to reduce code bloat by eliminating lots of copies of the random number generator in the rarely-executed slow paths. However a simple split prevented the compiler from being able to optimize cases like pcg32_rand(1 << n), so a lot of the blog post was toying around with ways to mitigate this problem. On Monday while procrastinating a different blog post, I realised that it’s possible to do better: there’s a more general optimization which gives us the 1 << n special case for free. nearly divisionless Lemire’s algorithm has about 4 neat tricks: use multiplication instead of division to reduce the output of a random number generator modulo some limit eliminate the bias in (1) by (counterintuitively) looking at the lower digits fun modular arithmetic to calculate the reject threshold for (2) arrange the reject tests to avoid the slow division in (3) in most cases The nearly-divisionless logic in (4) leads to two copies of the random number generator, in the fast path and the slow path. Generally speaking, compilers don’t try do deduplicate code that was written by the programmer, so they can’t simplify the nearly-divisionless algorithm very much when the limit is constant. constantly divisionless Two points occurred to me: when the limit is constant, the reject threshold (3) can be calculated at compile time when the division is free, there’s no need to avoid it using (4) These observations suggested that when the limit is constant, the function for random numbers less than a limit should be written: static inline uint32_t pcg32_rand_const(pcg32_t *rng, uint32_t limit) { uint32_t reject = -limit % limit; uint64_t sample; do sample = (uint64_t)pcg32_random(rng) * (uint64_t)limit); while ((uint32_t)(sample) < reject); return ((uint32_t)(sample >> 32)); } This has only one call to pcg32_random(), saving space as I wanted, and the compiler is able to eliminate the loop automatically when the limit is a power of two. The loop is smaller than a call to an out-of-line slow path function, so it’s better all round than the code I wrote last year. algorithm selection As before it’s possible to automatically choose the constantly-divisionless or nearly-divisionless algorithms depending on whether the limit is a compile-time constant or run-time variable, using arcane C tricks or GNU C __builtin_constant_p(). I have been idly wondering how to do something similar in other languages. Rust isn’t very keen on automatic specialization, but it has a reasonable alternative. The thing to avoid is passing a runtime variable to the constantly-divisionless algorithm, because then it becomes never-divisionless. Rust has a much richer notion of compile-time constants than C, so it’s possible to write a method like the follwing, which can’t be misused: pub fn upto<const LIMIT: u32>(&mut self) -> u32 { let reject = LIMIT.wrapping_neg().wrapping_rem(LIMIT); loop { let (lo, hi) = self.get_u32().embiggening_mul(LIMIT); if lo < reject { continue; } else { return hi; } } } assert!(rng.upto::<42>() < 42); (embiggening_mul is my stable replacement for the unstable widening_mul API.) This is a nugatory optimization, but there are more interesting cases where it makes sense to choose a different implementation for constant or variable arguments – that it, the constant case isn’t simply a constant-folded or partially-evaluated version of the variable case. Regular expressions might be lex-style or pcre-style, for example. It’s a curious question of language design whether it should be possible to write a library that provides a uniform API that automatically chooses constant or variable implementations, or whether the user of the library must make the choice explicit. Maybe I should learn some Zig to see how its comptime works.
I developed seasonal allergies relatively late in life. From my late twenties onward, I spent many miserable days in the throes of sneezing, headache, and runny eyes. I tried everything the doctors recommended for relief. About a million different types of medicine, several bouts of allergy vaccinations, and endless testing. But never once did an allergy doctor ask the basic question: What kind of air are you breathing? Turns out that's everything when you're allergic to pollen, grass, and dust mites! The air. That's what's carrying all this particulate matter, so if your idea of proper ventilation is merely to open a window, you're inviting in your nasal assailants. No wonder my symptoms kept escalating. For me, the answer was simply to stop breathing air full of everything I'm allergic to while working, sleeping, and generally just being inside. And the way to do that was to clean the air of all those allergens with air purifiers running HEPA-grade filters. That's it. That was the answer! After learning this, I outfitted everywhere we live with these machines of purifying wonder: One in the home office, one in the living area, one in the bedroom. All monitored for efficiency using Awair air sensors. Aiming to have the PM2.5 measure read a fat zero whenever possible. In America, I've used the Alen BreatheSmart series. They're great. And in Europe, I've used the Philips ones. Also good. It's been over a decade like this now. It's exceptionally rare that I have one of those bad allergy days now. It can still happen, of course — if I spend an entire day outside, breathing in allergens in vast quantities. But as with almost everything, the dose makes the poison. The difference between breathing in some allergens, some of the time, is entirely different from breathing all of it, all of the time. I think about this often when I see a doctor for something. Here was this entire profession of allergy specialists, and I saw at least a handful of them while I was trying to find a medical solution. None of them even thought about dealing with the environment. The cause of the allergy. Their entire field of view was restricted to dealing with mitigation rather than prevention. Not every problem, medical or otherwise, has a simple solution. But many problems do, and you have to be careful not to be so smart that you can't see it.
A few months ago I wrote about what it means to stay gold — to hold on to the best parts of ourselves, our communities, and the American Dream itself. But staying gold isn’t passive. It takes work. It takes action. It takes hard conversations that ask
