More from Hixie's Natural Log
Over the past few months I've been writing notes for how I would write an LLM-based development tool. Specifically, giving these tools the power to run programs (especially compilers and tests) without having to either check every command for unexpected side-effects or trusting an LLM to do this checking for you, so as to avoid <a href="https://embracethered.com/blog/posts/2025/the-normalization-of-deviance-in-ai/">a disaster. This week, I dumped the <a href="https://github.com/Hixie/llmdevsilo/blob/main/docs/DESIGN.md">entire notes document into Anthropic's <a href="https://www.anthropic.com/news/claude-fable-5-mythos-5">Fable with a 1M context window and "Ultracode" mode, and gave it the following prompt: Implement this design in ~/dev/llmdevsilo/ Continue uninterrupted until it is complete. Two hours and about 2 million tokens later (approx $10 cost based on the plan I'm using), it had implemented the entire thing. There were cosmetic issues, to be sure. The native UI overflowed on small window sizes, the web client needed a tweak to work around browsers not liking the self-signed cert on the local websocket, that kind of thing. But overall it had implemented the entire thing. One shot, first attempt. (I'm not counting the two false starts caused by bugs in the Claude desktop app; those attempts were canceled within the first few minutes and didn't influence the prompt of the actual attempt.) Fable is... well, <a href="https://simonwillison.net/2026/Jun/11/fable-is-relentlessly-proactive/">relentlessly proactive is pretty accurate. So, I can now present <a href="https://github.com/Hixie/llmdevsilo/tree/main">llmdevsilo, or just "Silo" as Claude decided to call it. The details are in the design doc (and the additional documentation Claude generated, some as a result of additional prompting later), but at a high level: You launch a desktop app, and tell it to start a session ("harness"), pointing it at a directory and an LLM model. Currently it supports the Anthropic REST API, the Open AI REST and WebSocket APIs, and local models. (I've no idea how the local model stuff works; the design notes just said to support it and something was implemented for it, but I haven't tested it.) A harness process is instantiated. The directory you specified gets locked behind a disk image and mounted inside a sandbox. The only sandbox I've tested so far is macOS sandbox-exec, which seems to be the state of the art on Mac (despite being deprecated). On Linux it supports gVisor, but I haven't tested that yet. You can now talk to the LLM, and the LLM can run programs in the sandbox (it is given a set of tools similar to what the Claude interfaces implement, like Read/Write, Bash, etc). It cannot read your local files, only what was in the directory you gave it and a select set of binaries (like those in /usr/bin). The sandbox is also provided with a scratch directory. By default, any code running in the sandbox (and thus the LLM) can't access the network. If configured, the harness can provide the sandbox with network access via an HTTP proxy. The proxy supports TLS, there's a whole mechanism whereby a freshly minted temporary root CA is injected into the sandbox and the proxy generates certificates on the fly. Under gVisor, DNS is also proxied; only allow-listed names can be resolved. What all this means is you can use an LLM with untrusted third party dependencies, give it access to the web, give it a real dev environment with compilers, and be safe in the knowledge that none of your secrets will be <a href="https://medium.com/@nem841425bal/openais-codex-leaks-secrets-28f0bd749cc3">exfiltrated, your <a href="https://www.reddit.com/r/vibecoding/comments/1r96647/gpt_53_codex_wiped_my_entire_f_drive_with_a/">hard disk won't be wiped, and you never need to see a permissions prompt again, so no risk of prompt fatigue. This isn't the inherently imperfect "model watches the model" security, it's enforced by real sandbox boundaries. A core part of the design that I like is that the UI is separate from the harness. You can run the harness on one computer and connect to it from another, or from your phone. There is a Flutter-based app UI that works as a native app on desktop and mobile (so far only tested on macOS) and also works on web. There is a Rust-based terminal client. In principle there could be any number of other apps too. The UIs have a secure mechanism to connect (by design; as with everything else, I haven't studied the implementation). To add another client, you get a pairing code from an existing client and connect; they then mint asymmetric key pairs for reconnecting securely later. There are some limitations. The biggest is that I haven't audited any of this code yet. It could be full of holes. The entire codebase is itself written by an LLM. The security-sensitive parts are mainly in Rust and I am not fluent in Rust, nor in the security technologies it attempts to leverage and implement. Another big one is that if you're not able to use local models, you will need to use one of the APIs, which are a lot more expensive than the plans. For example, eyeballing the price tables, I think the original one-shot to create this would have cost about $100-$150 dollars in about two hours. Open AI and Claude only seem to allow you to use their plans with their own software, custom software is billed by the token. The final limitation worth being explicit about is that while everything outside the sandbox is supposed to be safe from exfiltration, if you allow any network egress then the contents of the sandbox are not. This matters if what you're developing isn't going to be open source, for example. I am happy to accept patches, but I may be slow to adopt them. Feel free to fork and run with this if you are interested. I have not finished tweaking it, I decided to post this in its current state because I ran out of tokens, and it seemed interesting enough to publish as is. The <a href="https://github.com/Hixie/llmdevsilo/commit/a252a24407a64aab2c7a08624ea13165787c5ea2">first commit in the repo is what the original prompt generated; subsequent commits are from additional discussions with the agent.
Over the past few months, I've been putting together some notes for future creators of UI frameworks, based on my experiences of the past 25-30 years. The work was originally commissioned by a customer for their use, but it seems like it might be of wider interest so I've decided to publish it. The primary copy is a Google doc, available at https://software.hixie.ch/ui-frameworks, on which people can leave comments. There is also a canonical PDF copy of this first edition of the document if that is a more convenient format for anyone. As always, I welcome feedback. The most helpful way to leave comments is on the Google doc itself.
My involvement in web standards started with the CSS working group. One of the things that we struggled with as a working group was that we would specify how the technology should work, but the browser vendors' implementations weren't exactly what we intended, and web authors would then write web pages that worked with those browsers, even though that meant the web pages themselves were also not doing things like the specifications said they should. The folks I worked with at the W3C (especially the academics and people working for organizations that did not themselves implement browsers) would frequently bemoan this state of affairs, expressing surprise at how they, the people in charge of the standards, were not being respected by the people implementing the standards. One of the key insights I had very early on in my work, before working on HTML5, which really influenced the WHATWG and its work, is the realization that the power dynamics at work were not at all the power dynamics that the folks at the W3C described. The reality of the situation was that the power lay entirely in the hands of the users. The users chose browsers. A browser vendor that ignored what the users wanted would lose market share. Market share is everything in this space. Browser vendors want users because they can convert users into dollars (in various ways, but they typically boil down to someone showing them ads and paying the browser vendors for the privilege). In turn, the browser vendors had more power than the specifications. What they implement is, by definition, what the technology is. The specification can say in absolute clarity that the keyword "marigold" should look yellow, but if a browser vendor makes it look red, then no web author is going to use it to mean yellow, and many will use it to mean red. There is a feedback loop here: if one browser implements "marigold" to mean red, and some important web site (or many unimportant web sites) rely on it, and say something like "best viewed in ThisOrThat browser!" because that's the one they use and in that browser it looks red and red is what looks best, then the other browser vendors are incentivised to make sure that the web page looks good in their browser too. Regardless of what the specification says, therefore, they are going to make "marigold" look red and not yellow. When I realized this, I also realized a corollary: if you have two competing specifications that both claim to define the same technology, but one matches what the browsers already do while the other one does not, the browser vendors are going to find it more useful to follow the one that matches what they do. This is because they can trust that implementing that specification will get them more market share. It means they won't have to stop and think at every step, "will following this specification cause me to lose users?". It is easier for them to use a specification that takes into account their needs in this way. We actually tried to explain this to the W3C membership. There was a big meeting in 2004 at Adobe in San Jose, the "W3C Workshop on Web Applications and Compound Documents". We tried to convey the above (I didn't quite understand it in the stark "power dynamic" terms yet, or at least, I didn't really express it in those terms, but if you read our position paper you can see this insight starting to crystalize). At this meeting, we made a pitch for the W3C to continue to maintain HTML and to care about what the browser vendors wanted. Representatives from Microsoft and Sun (in many ways arch enemies at the time) supported us. I seem to recall Apple being more quiet about it at the meeting but also essentially supporting the principles. The W3C membership resoundly rejected this whole concept. One of the W3C staff even explicitly said something along the lines of "if you want to do this you should do it elsewhere". That's what led to the WHATWG being founded a few weeks later. The WHATWG was founded on this core principle — the specifications need to actually specify reality. When the browsers disagree with the spec, the spec is by definition incorrect and needs to change, regardless of how much technically superior the design in the spec is. Naturally, when you provide browser vendors with something that valuable, they will follow. You end up with a weird inverted power dynamic. The spec writer (when they follow this principle) has all the power, but only within the space that the browser vendors are themselves willing to play; and the browser vendors have all the power, but only within the space that the users are willing to put up with. It's very easy to appear to be in control when you tell people to do the thing they were going to do anyway (or at least, one of the things they were willing to do if they were to think about it). There is a (probably apocryphal) quote supposedly by Alexandre Auguste Ledru-Rollin that is often cited in mockery of bad leadership, but that perfectly matches the power dynamic here: "There go my people; I must find out where they are going so I can lead them". (Thanks to Leonard Damhorst for prompting me to write this post.)
I often get asked how many people contribute to Flutter. It's a hard question to answer because "contribute" is a very vague concept. There's tens of thousands of packages on pub.dev, all of which are written by contributors to the community. There's over 100,000 of issues filed in our issue database, filed by more than 35,000 people over the years (the exact number is hard to pin down because people sometimes delete their GitHub accounts; about 700 issues have been filed by people who have since deleted their account). Many more people still have used the "thumbs-up" reaction to indicate that an issue matters to them, with almost 165,000 thumbs up from about 45,000 people. All of these people are valuable contributors to Flutter. Usually, when pressed, people try to clarify by asking about "the core team". Again though it's hard to say exactly what that means, but let's assume they mean "people with commit access". That is, people we trust enough to have added to the GitHub repo as collaborators. This includes people who work on Flutter for companies like Google, Canonical, or Nevercode, and it includes people like me who are self-employed and/or contribute to Flutter on a volunteer basis. Currently that's about 280 people. So is that the answer? Well, no, not really. Some people have commit access but aren't active (maybe they got access because of their employer, but were then reassigned to work on another project, and the bureaucracy hasn't caught up with them yet — we only audit the membership occasionally because it's rather tedious to do). Some people have been very active recently but don't have commit access (e.g. because they were just laid off and a bot automatically removed their access; they might even resume working on Flutter in the future, as a volunteer or funded by another company). So what's the answer? I recently drilled down through our data to see if I could answer this. I will caveat the following numbers by saying that this changes all the time. We added a new team member just today (hi Nate!) who is not counted as a team member in the following numbers because we collected the data a few weeks ago (it takes literally days to scrape all the data from GitHub, and then hours to explore the resulting very large and very slow spreadsheet). Also, some of my definitions are a bit arbitrary, and slightly tweaking the limits would probably change the numbers noticeably. First, I collected a list of everyone who has ever created an issue, commented on an issue, put an emoji reaction on the first comment of an issue, or submitted a PR, excluding bots and people who deleted their GitHub account. (Actually Piinks did the actual data collection. Thanks!) I limited this to a subset of the GitHub repos of the flutter org that is relatively inclusive but does not count everything (we have a lot of historical repositories and so forth). This finds about 94,357 people. (So there you go. The Flutter team is about a hundred thousand people!) To avoid padding the numbers with people who left the project long ago, and to avoid counting "drive-by" contributors who came, did a bunch of work, and then left, I then limited the data set to people who contributed over a period of more than 180 days, and who last contributed sometime in 2024. Because of the definition of "contributed" described above, that means that someone who added a thumbs-up to an issue in December 2020 and then filed an issue in January 2024, and did nothing else, is included, but someone who submitted two PRs in March 2024 is not. Like I said, this is a bit arbitrary. Anyway, that leaves 3,839 people, of which 182 currently have commit access, 27 once had commit access but don't currently (these are mainly people who either got laid off recently and had their commit access revoked by an automated process, or people who were once team members, left, lost access from inactivity long ago, and then later came to comment on issues or file new issues — it's surprisingly common for people who once worked on Flutter full time to stick around even when their employment changes), and about 3,627 people who have never had commit access. Of those who have never had commit access, 2,407 have filed at least one issue or submitted at least one PR (accounting for a total of 12,383 issues and 2,613 PRs). Of those, 341 have filed 5 to 9 issues (2,242 issues total), and 296 have filed 10 or more issues in their lifetime (7,021 total issues). Similarly, of the "never had commit access" cohort, 73 people have sent 5 to 9 pull requests in their lifetime (458 total PRs) and 47 have sent 10 or more (1,321 PRs total). (For context, 4,663 people have ever submitted a pull request, and 429 have ever submitted more than 10 PRs.) Of the people who currently have commit access, 98 people have submitted more than one PR every 3 weeks on average since they first got involved (accounting for 49,173 PRs), 75 people have closed at least one issue every 3 weeks (accounting for 48,490 total issue closures), of which 10 are not in the first group (mostly that's our triage team), and 150 people have commented at least once every 3 weeks. A follow-up question a lot of people ask is "do they all work for Google?". This is a surprisingly hard question to answer. There are a lot of weird edge cases. For example, one person worked on Flutter for a company that Google hired to work on Flutter, but then quit that company, asked for their commit privileges to be removed, but continued to be active in the community. Several people who have quit Google (such as myself), or been laid off by Google, have continued to be active in one sense or another (I think I submit more code to Flutter now than I did in my last year at Google). It's also hard to answer because a lot more people at Google contribute to Flutter than just those on Google's Flutter team, and a lot of people on Google's Flutter team contribute in ways that don't show up on GitHub (e.g. product management, marketing, developer relations, internal tooling). Of the 98 people who have commit access, have been active for more than 180 days, have contributed at least once this year, and have submitted more than one PR every 3 weeks on average for the entire time they've been contributing, I estimate (based on what I know of people's employment and so forth) that about 85% are Googlers or somehow get their funding from Google, and about 15% are currently independent of Google. (This is by no means the entirety of the Google team contributing to Flutter; as I mentioned earlier, many folks at Google working on Flutter don't appear in these statistics.) I'm not sure what conclusion to draw from this; it's both more people than I expected to see funded by Google, which is great, and fewer people that aren't funded by Google, which is less great. On the other hand, it's still a significant number of non-Google-funded people. Is it enough? I think that really depends on what your goals are. I think if your goal is for Flutter to be an order of magnitude better than other UI frameworks, then frankly no, it's not enough. There is a ton of work to be done to get there. We know what it would take, but we don't have the people to do it today. On the other hand if your goal is to be a great framework, on par with others, then it's probably adequate. It would certainly be difficult to continue to be great with fewer people today. Of course, that may change as we complete big efforts, or as we take on new ones, or as the landscape changes, it's all hard to predict. That said, I would love to see more direct contributions from non-Google sources, if for no other reason but to end this silly "will Google cancel Flutter" line of questioning that has followed the project since its inception. It's a dumb question. Flutter's an open source UI framework. It will never die. It will become old and something else will shine brighter one day, just as happens with literally every other UI framework ever. That's just how our industry works. There's no reason to believe that'll happen any time soon though, and certainly no reason for it to happen earlier for Flutter than any other modern UI framework.
More in programming
Helping Scout focus and remember
tl;dr: the Explorer Pattern is one individual or team building a functional implementation that minimizes cross-team and cross-functional impacts as much as humanly possible, and then reconciling the details with other teams after that prototype is completed, accepting that this approach can require very substantial rework to that initial development. Why write this? Imprint previously called this the “Tech-led” or “Engineering-led” development model, but it’s pretty unclear to folks what that means. This writeup provides a more defined term, the “Explorer Pattern”, and explains in more detail how this works. Why do we need to adjust the way we work? Before 2026: Code is expensive, and we design process to minimize implementation and rework In and after 2026: Writing software is extremely cheap, but it’s a wider phenomenon: generating output is effectively-free across many dimensions. Code is nearly free — if we have appropriate constraints. Designs are nearly free — if they follow patterns. Analysis is nearly free — if we’ve instrumented things properly Reworking software is extremely cheap. Historically industry has optimized around avoiding engineering rework because engineering was the highest cost / narrowest constraint. In many companies this was interpreted as writing code being the constraint, which was never true, but many common practices were designed as if that were true (e.g. bringing engineering into problem-solving late “to protect their time”) Working on the system is much more effective than historically if we get automated integration and feedback loops right, e.g. not improving a one-off design but improving the design system. Any place where improving the higher-leverage process is slower than solving the one-off is a bug in our process Explorer Pattern: primary-and-replicas metaphor The easiest metaphor to think about the Explorer Pattern is database replication. Databases like MySQL and PostgreSQL have a primary database, where writes are performed. Every write is verified at that point, and once it’s verified, you know that the writes will flow downstream to various secondary replicas. Those downstream replicas can be delayed but they can’t be wrong because writes are verified upstream. (I’ve resisted the desire to talk about the CAP theorem.) A different model for database design is that used by Dynamo or Cassandra (or MongoDB to a certain extent), where you have a replica set, and have to get an acknowledgement from the majority of the replica set for a write to be strongly consistent. This is a very reliable setup, because it means you can tolerate losing a node, but it also introduces inconsistencies, and on a fixed amount of hardware they tend to be much, much slower than running a primary-secondary setup (or rely on bad performance metrics that don’t fsync to disk to create the illusion of throughput). The Explorer Pattern is the former primary-secondary model, where the vast majority of the work is done by the working individual or team. This makes all work internally consistent because it’s passed through the same small group. It doesn’t, however, mean that the work is perfect. For example, you can always insert data that meets the table’s requirements but is still inherently wrong (e.g. 200 is a valid integer, but it doesn’t mean that my credit limit was supposed to be 200, that depends on the surrounding application logic). Other teams become aware of the changes downstream, not synchronously upfront, and it’s instead the working group’s responsibility to both rationalize approach and create visibility into their work for other teams to engage in a non-blocking fashion. Example: Support for Financial Aggregators A concrete example of this practice at Imprint was rolling out financial aggregator support, e.g. allowing folks to connect to their Imprint powered accounts via Plaid. This has always been something folks want, but it’s particularly important in the realm of agentic commerce where more folks’ agents are analyzing their financial state for them. For this project, one of our engineering leaders (Ron), built the first pass of the feature at a low-burn while understanding the shape of the problem in a private alpha. He chatted through some of the details periodically to keep other teams aware, but it was a bit of a side project. However, once he found the full shape of the problem and had implemented a full solution, we cut the replica-lag across teams down to near-zero, identified and ran through the small number of remaining issues, and shipped it. Like many of our projects this year, it was never officially on a roadmap, and shipped a quality feature that our members have wanted for some time. How to follow the Explorer pattern The rough rules for following the Explorer pattern: Know your goal and sponsor: every initiative ultimately has to have a specific outcome it’s pursuing and a sponsor who’s bankrolling that pursuit. This doesn’t always need to be explicit, especially for smaller projects, but once a project goes beyond ~3 weeks of effort, if you can’t very explicitly identify who the stakeholder is sponsoring your project and they would not enthusiastically agree, then you are likely closer to “lost” than “exploring” Small working group is small: at most 5 people, and preferably smaller. Everyone in the group has to keep the entire project in their head for as long as possible to detect problems and opportunities Small working group is flexible: folks in small working groups have to “find a way to yes” rather than “find a way to no”. Exploring is about uncovering the fundamental shape of a problem rather than pretending we know the fundamental shape Publish progress in a public channel: explorers have to report home with the progress they are making so others are aware. This is both a Linear Project and a Slack channel. Make the narrowest viable changes to what already exists: when we introduce unnecessary changes, we invalidate others’ mental models about how the world works. When we haven’t made those changes with care, we’re not only invalidating others’ mental model, we are creating the expectation that we’ve done it very intentionally for a good reason, which has a chilling effect on others improving them later, which is very expensive, whereas the Explorer pattern depends on being as inexpensive as possible Exercise good judgment to identify reasonable hypotheses: discard bad ideas immediately, and then filter the rest by trying to implement them as quickly as possible. Once you see the shape of the implementation, you can figure out which of the remainders are interesting Hold assumptions constant to avoid getting stuck: the only way to navigate complex ambiguous problems is to hold things constant, even when you know they are imperfect. For example, to do headcount planning we have to know our revenue, to know our revenue we have to forecast it, to forecast it we have to know the day that partners will go live next year. We don’t truly know any of those things, so just anchoring on a plausible guess to unblock us is much more valuable than trying to get them accurate. A little prework is useful, but a lot is just artificial precision that’s a distraction from progress Hold space for being totally wrong: when we move fast, with low information, we always find the way out, but often it’s not the way out we anticipated when we got started. Being open to throwing it all away, and starting a second, third, fourth time Ship all the way to production behind a feature flag: features and products move fast when we get the pieces working every step from frontend to backend to data pulls. This is the first step of exploration. Just make sure it’s behind a feature flag that’s only enabled for internal folks!
Comparing two different ways of work and working environments.
I own an old, cheap TV. It has no Internet connection, no fancy smart features that make you want to install a Pi-hole, and no ask-me-again-later updates. However, it is connected to my own desktop computer through an HDMI cable, so I can watch movies and TV series from the couch. Setting it up used […] The post Making a dumb TV just smart enough appeared first on Quentin Santos.
Five days and 540km on a bike across Kyushu, without a laptop. Notes on the peace of riding all day, convenience store dinners, and Japan's emptying countryside.