Full Width 
Shortcuts 
Sign Up 
Log In 
Articles 
Reading List More from Notes on software development
Transactions are not an intrinsic part of a storage system. Any storage system can be made transactional: Redis, S3, the filesystem, etc. Delta Lake and Orleans demonstrated techniques to make S3 (or cloud storage in general) transactional. Epoxy demonstrated techniques to make Redis (and any other system) transactional. And of course there's always good old Two-Phase Commit. If you don't want to read those papers, I wrote about a simplified implementation of Delta Lake and also wrote about a simplified MVCC implementation over a generic key-value storage layer. It is both the beauty and the burden of transactions that they are not intrinsic to a storage system. Postgres and MySQL and SQLite have transactions. But you don't need to use them. It isn't possible to require you to use transactions. Many developers, myself a few years ago included, do not know why you should use them. (Hint: read Designing Data Intensive Applications.) And you can take it even further by ignoring the transaction layer of an existing transactional database and implement your own transaction layer as Convex has done (the Epoxy paper above also does this). It isn't entirely clear that you have a lot to lose by implementing your own transaction layer since the indexes you'd want on the version field of a value would only be as expensive or slow as any other secondary index in a transactional database. Though why you'd do this isn't entirely clear (I will like to read about this from Convex some time). It's useful to see transaction protocols as another tool in your system design tool chest when you care about consistency, atomicity, and isolation. Especially as you build systems that span data systems. Maybe, as Ben Hindman hinted at the last NYC Systems, even proprietary APIs will eventually provide something like two-phase commit so physical systems outside our control can become transactional too. Transactions are a protocol short new post pic.twitter.com/nTj5LZUpUr — Phil Eaton (@eatonphil) April 20, 2025
This is an external post of mine. Click here if you are not redirected.
Last month I completed my first year at EnterpriseDB. I'm on the team that built and maintains pglogical and who, over the years, contributed a good chunk of the logical replication functionality that exists in community Postgres. Most of my work, our work, is in C and Rust with tests in Perl and Python. Our focus these days is a descendant of pglogical called Postgres Distributed which supports replicating DDL, tunable consistency across the cluster, etc. This post is about how I got here. Black boxes I was a web developer from 2014-2021†. I wrote JavaScript and HTML and CSS and whatever server-side language: Python or Go or PHP. I was a hands-on engineering manager from 2017-2021. I was pretty clueless about databases and indeed database knowledge was not a serious part of any interview I did. Throughout that time (2014-2021) I wanted to move my career forward as quickly as possible so I spent much of my free time doing educational projects and writing about them on this blog (or previous incarnations of it). I learned how to write primitive HTTP servers, how to write little parsers and interpreters and compilers. It was a virtuous cycle because the internet (Hacker News anyway) liked reading these posts and I wanted to learn how the black boxes worked. But I shied away from data structures and algorithms (DSA) because they seemed complicated and useless to the work that I did. That is, until 2020 when an inbox page I built started loading more and more slowly as the inbox grew. My coworker pointed me at Use The Index, Luke and the DSA scales fell from my eyes. I wanted to understand this new black box so I built a little in-memory SQL database with support for indexes. I'm a college dropout so even while I was interested in compilers and interpreters earlier in my career I never dreamed I could get a job working on them. Only geniuses and PhDs did that work and I was neither. The idea of working on a database felt the same. However, I could work on little database side projects like I had done before on other topics, so I did. Or a series of explorations of Raft implementations, others' and my own. Startups From 2021-2023 I tried to start a company and when that didn't pan out I joined TigerBeetle as a cofounder to work on marketing and community. It was during this time I started the Software Internals Discord and /r/databasedevelopment which have since kind of exploded in popularity among professionals and academics in database and distributed systems. TigerBeetle was my first job at a database company, and while I contributed bits of code I was not a developer there. It was a way into the space. And indeed it was an incredible learning experience both on the cofounder side and on the database side. I wrote articles with King and Joran that helped teach and affirm for myself the basics of databases and consensus-based distributed systems. Holding out When I left TigerBeetle in 2023 I was still not sure if I could get a job as an actual database developer. My network had exploded since 2021 (when I started my own company that didn't pan out) so I had no trouble getting referrals at database companies. But my background kept leading hiring managers to suggest putting me on cloud teams doing orchestration in Go around a database rather than working on the database itself. I was unhappy with this type-casting so I held out while unemployed and continued to write posts and host virtual hackweeks messing with Postgres and MySQL. I started the first incarnation of the Software Internals Book Club during this time, reading Designing Data Intensive Applications with 5-10 other developers in Bryant Park. During this time I also started the NYC Systems Coffee Club. Postgres After about four months of searching I ended up with three good offers, all to do C and Rust development on Postgres (extensions) as an individual contributor. Working on extensions might sound like the definition of not-sexy, but Postgres APIs are so loosely abstracted it's really as if you're working on Postgres itself. You can mess with almost anything in Postgres so you have to be very aware of what you're doing. And when you can't mess with something in Postgres because an API doesn't yet exist, companies have the tendency to just fork Postgres so they can. (This tendency isn't specific to Postgres, almost every open-source database company seems to have a long-running internal fork or two of the database.) EnterpriseDB Two of the three offers were from early-stage startups and after more than 3 years being part of the earliest stages of startups I was happy for a break. But the third offer was from one of the biggest contributors to Postgres, a 20-year old company called EnterpriseDB. (You can probably come up with different rankings of companies using different metrics so I'm only saying EnterpriseDB is one of the biggest contributors.) It seemed like the best place to be to learn a lot and contribute something meaningful. My coworkers are a mix of Postgres veterans (people who contributed the WAL to Postgres, who contributed MVCC to Postgres, who contributed logical decoding and logical replication, who contributed parallel queries; the list goes on and on) but also my developer-coworkers are people who started at EnterpriseDB on technical support, or who were previously Postgres administrators. It's quite a mix. Relatively few geniuses or PhDs, despite what I used to think, but they certainly work hard and have hard-earned experience. Anyway, I've now been working at EnterpriseDB for over a year so I wanted to share this retrospective. I also wanted to cover what it's like coming from engineering management and founding companies to going back to being an individual contributor. (Spoiler: incredibly enjoyable.) But it has been hard enough to make myself write this much so I'm calling it a day. :) I wrote a post about the winding path I took from web developer to database developer over 10 years. pic.twitter.com/tf8bUDRzjV — Phil Eaton (@eatonphil) February 15, 2025 † From 2011-2014 I also did contract web development but this was part-time while I was in school.
This is an external post of mine. Click here if you are not redirected.
I was so intimidated to go at first, but it is in fact easy and fun to start playing beginner volleyball in New York. The people are so friendly and welcoming that it has been easy to keep playing consistently every week since I started for the first time this August. It's been a great workout and a great way to make friends! The two platforms I've used to find volleyball games are Goodrec and New York Urban. While these platforms may also offer classes and leagues, I mostly use them to play "pickup" games. Pickup games are where you show up and join (or get assigned to) a team to play for an hour or two. Easy to go on your own or with friends. I'm not an expert! My only hope with this post is that maybe it makes trying out volleyball in New York feel a little less intimidating for you! Goodrec With Goodrec you have to use their mobile app. Beginner tier is called "social" on Goodrec. So browse available games until you find one at the level you want to play. You enroll in (buy a place in) sessions individually. Sessions are between 90-120 minutes long. They ask you not to arrive more than 10 minutes early at the gym. When you arrive you tell the gym managers (usually in a desk up front somewhere) you're there for Goodrec and the tier (in case the gym has multiple level games going on at the same time). Then you wait until the Goodrec "host" arrives and they will organize everyone into teams. Goodrec hosts are players who volunteer to organize the games. They'll explain the rules of the game (makes Goodrec very good for beginners) and otherwise help you out. Always say thank you to your host! New York Urban With New York Urban, pickup sessions are called "open play". There is no mobile app, you just use the website to purchase a spot in a session. The sessions are longer and cheaper than Goodrec. But there is no host; players self-organize. The options are more limited too. You play at one of four high schools on either a Friday night or on Sunday. And session slots tend to sell out much more quickly than with Goodrec. Big City Volleyball You can also check out Big City Volleyball but I haven't used it yet. Volo I haven't ever done Volo but I think I've heard it described as "beer league". That even some of the beginner tier sessions with Goodrec and New York Urban are more competitive. But also, Volo is built around leagues so you have to get the timing right. Goodrec's and New York Urban's pickup games make it easy to get started playing any time of year. Making friends It was super awkward to go at first! I went by myself. I didn't know what I was doing. I couldn't remember, and didn't know, many rules. I didn't have court shoes or knee pads. But the Goodrec host system is particularly great for bringing beginners in and making them feel welcome. You have a great time even if you're terrible. The first game I went to, I tried to hang out afterward to meet people. But people either came with their SO or with their friends or by themselves so they all just left immediately or hung out in their group. So you can't just go once and expect to make friends immediately. But if you keep going at the same place and time regularly week over week, you'll see familiar faces. Maybe half the people I play with each week are regulars. If you're friendly you'll start making friends with these people and eventually start going out to bars with them after the games. Improving Even if you find yourself embarrassingly bad at first, just keep going! I'm 29, 6'1, 190lbs and from observation the past 5 months, age, height, and weight have a very indirect relation to playing ability. Most of the people who play are self-taught, especially at the lower tiers I've played at. But some people played for the school team in high school or college. These people are fun to play with and you can learn a lot from them. Most people who are self-taught seem to watch YouTube videos like Coach Donny, helpful for learning how to serve, set, block, etc. Or they take "clinics" (classes) with Goodrec or other platforms. (I have no idea about these, I've never done them before.) At first I played 2 hours a week and I was completely exhausted after the session. Over time it got easier so I started playing 2-3 sessions a week (6-9-ish hours). With practice and consistency (after about 3-4 months), I started playing Intermediate tier with Goodrec and New York Urban. And I don't think I'll play Beginner/Social at all anymore. I still primarily play for fun and for the workout and to meet people. But it's also fun to get better! I played with one person much better than myself in an Intermediate session one time and he mentioned he will probably stop playing Intermediate and only play High Intermediate. He mentioned you get better when you keep pushing yourself to play with better and better players. Good advice!
More in technology
Yesterday, a group of MIT researchers released an article titled “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights”. The piece describes a set of experiments which convincingly show OpenAI’s latest Astra model is able to understand and successfully alter raw weight data. Wowzers! It’s a rough analogy, but an LLM’s weights are sort of like its brain. A set of weights is a large pile of numbers generated during training, and an LLM’s output is derived by running input through the weights using some matrix multiplication and a bit of randomization. While the steps to generate weights are well understood, the weights themselves are extremely opaque. There’s ongoing work to try and explain how weights work, but researchers currently have a limited understanding of what these numbers do. Based on this, I was quite surprised to learn that LLMs are able to act on raw weight data — even if it’s still just in limited cases. It’s unclear if OpenAI’s model was trained on weight data. Even if it was, I still find these results surprising. The article shows the model accurately predicting what a never-before-seen set of weights was built to do, fixing another model by updating its weights, and creating new sets of weights that performed simple tasks. I would not have guessed any of that would be possible a couple of days ago. This whole thing is slightly unnerving. The whole “weights as a black box” thing has always felt a bit sketchy to me, even when I was working with much simpler machine learning models. Now these experiments hint at a future where LLMs could be self-improving in ways that aren’t human-verifiable. Machines building machines, and we can only hope that they’re not doing anything weird. I don’t know if this strategy is scalable, but I’d guess that it’s something more people will be exploring very soon. More and more often these days, I wish we were living in less interesting times.
Harry threw me off course in episode 48 of Traction Heroes. At first, I thought was reading from a book about mind mapping. Soon, it became clear it was about something else: how to define a clear vision for a team or organization. Visions are notoriously hard to articulate. Drawing a picture — especially a picture of the whole forest, plus the trees — can help. The passage Harry read from Cameron Herold’s Vivid Vision explores this idea: So to visualize something a little more tangible, think about a tree trunk in the earth, inner core with its branches stretching out into the sky. Each of those branches contains twigs extend further out. The mind map works in a similar way, a core that branches out into smaller components. Now, apply this to the world of business. As the leader of an organization, you’re firmly planted at the center, and the various areas that contribute to the business branch out: marketing, sales, engineering, and so forth. With that in place, build off that by describing those branches in about three or four bullet points each. In other words, describe what your marketing looks like and sales and so on until you’ve captured every idea. Remember too that among the branches of your business, you want to consider every aspect of the organization: the vibe, informal codes of conduct, the fiscal environment, the scenery, all of that. Deliberately leave out numbers and data. Get a feel for the place. When you step out of your time machine three years from now and walk around the office, what’s happening? What can you say that is typical of the business three years from now? The mind map is an initial foray into the vivid vision and will act as an outline for the rough draft that you, the leader, will then write. I’ve sketched diagrams before when outlining articles, workshops, and presentations, never in service of aligning around a company’s vision. Harry’s done it several times and found it easier than trying to write it out. Seems useful — check out our conversation for more: Traction Heroes episode 48: Visions
Sharing a few data points because some people did not expect these drives to last that long.
I'm 100% for banning social media for children. I would even extend it to banning it all together in its current form. But I am also against the methods for enforcement. The problem with banning social media for one group and not another is that you have to differentiate the different groups somehow. And right now, in an age where information and personal data means everything, there are several entities that are salivating to get this data. I've written before about all the problems that come from uploading sensitive data online. It doesn't even matter if they delete the data afterward, as long as the data goes through the wire in the first place, there is a chance for a leak. What appears to be the same simple request "please verify your identity", becomes fundamentally different when mediated by technology. The question isn't whether these digital systems will be compromised, but when. And unlike that movie theater clerk who can't perfectly recall my birthdate minutes after seeing it, computers have perfect memory. They store, copy, backup, and transmit our most sensitive information through networks we don't control, to companies we've never heard of, under policies we'll never read. When kids are not allowed to use social media, it's the adult who pays the price. It's the adult who has to prove that they are not children. They have to risk uploading their government ID to a third party, or biometric data that may be saved and backed up forever. I reject this method. I also reject the method where I have to provide my age to the operating system. No system should compel me to provide personal data in order to use it. Especially if this feature is intended to provide the information to a third party. And what happens if I don't provide this information to my freebsd machine? Or worse, what if I provide false information? Am I going to get punished? How does that protect children in any way? Breaking news: Grown up arrested when OS reported forged identity in his laptop. One question I asked myself was what happens when a minor circumvents the ban. What are the consequences? At least in Australia, a country that has implemented a social media ban for those under 16, there are no consequences. The law places legal responsibility on platforms, not children or parents, and it currently applies to Facebook, Instagram, Kick, Reddit, Snapchat, Threads, TikTok, Twitch, X and YouTube. This almost sounds reasonable. Why should you care, if it becomes facebook responsibility when you don't even own a facebook account? But there is always a caveat. When facebook becomes a tool your school uses to communicate with parents. When Google classroom is the service students use to do their homework. When you are required to login on a system to receive government service. You may not care about a particular social media, but they raise the bar for every other entrant in the game. Every website frequented by a large number of people inherently become social media, and demand data you didn't need to provide before. Technology is not a magical thing that can just solve every problem. It is especially weak when it comes to social problems. However, society can use technology to solve their problems. The responsibility of children falls onto their parents. And there are great tools for parents that their kids access on their devices.
This weekend I ran the marathon of The Hague. I gave everything, and the result was 3 hours and 40 minutes. I am satisfied, because I know there was really nothing more in it. Tired legs, maximum heart rate, and still no collapse. It is beautiful that you can look for the limit like that. But the most important part of this marathon did not happen on the road. It happened in the weeks before, at home, looking out of the window. Two marathons in four weeks Four weeks earlier I ran Köln. Two days before the start my body was fighting a virus. I slept extra, stopped working and switched the computers off. The next afternoon I wrote: "the body is winning, energy is flowing through the veins again". Köln became 3:48:27. The Hague went better in every way. The extra experience made it possible to operate longer at the maximum: eating, drinking and the mental side were better than ever. I understand very well now what makes the performance. And it is not only the training. The Hague, 1 November 2026, just before the start Work as a long run At the end of October I wrote something less cheerful in my notes. Not only sport, but work too had taken on the character of a long run. Steady, long, without an end in sight. More than ever before, the balance tipped to the negative: no sparkling fun, warmth or pleasure. "It really is a marathon. I am physically, and mentally, ready for rest." That is the trap of a pace you can hold: you can hold it so long that you forget you are running. The answer was not to stop. It was to stay pragmatic, limit the loss of energy, and focus on the things we want to influence. And nothing more. Room to breathe On 15 October I did something different. The phone on "do not disturb" and out of reach. Questions about the jam supply and the invoices came in, all very important, for later. Reachable for emergencies, but there was no emergency. I wrote: "The best in me comes up when there is room. This is not running away or hiding. It is a necessary pause to breathe." Three days later, on a Sunday, I tried a whole day with zero screen time. A small experiment to feel where we still depend on it. Physical things stood out more: clutter, dust. Conversations with the children were easier, because I had so much more bandwidth to ask good questions. And I simply looked out of the window. I thought about my body, about the fast food of the day before, and about how little I really need. A green, orange or red light would be enough. The Sunday without screen time: wood, a plant by the window, daylight on the table That light exists now. The home screen of Contact, the software I work with every day, is one dot. Green means: all is well, nothing is waiting for me. It took a Sunday without screens to see that one dot was enough. The home screen of Contact: one green dot, nothing is waiting Why a window works Looking out of the window is not doing nothing. In 1984 Roger Ulrich published a small study in Science. Patients recovering from the same operation, in the same hospital, were given a room with a view of trees or a view of a brick wall. The ones looking at trees went home sooner, 7.96 days against 8.70, and needed fewer strong painkillers. A view, nothing more. There is a name for designing with that in mind: biophilic design. Daylight, plants, wood, a view and a place to retreat to. I only learned the word properly this autumn, while building levendkantoor.nl for Martin Cuypers, a client in Venray, not far from home. Their page on what biophilic design is puts it in three layers: nature in the space, nature as a reference, nature as an experience. Reading it, I recognised my own Sunday. Wood under my hands, a plant by the window, daylight on the table. Nothing to buy. Just noticing. As is, not add-on The morning after The Hague I wrote down the conclusion. The gym is no longer "nice to have". We are making it an integral part again, on days that keep the impact on working hours small. The goal: a strong physical base, not as an add-on, but as is. Next year I want to break 3:30, and that can only happen if we make everything stronger, in all dimensions: strength training, sprints, and longer runs at low intensity, around 130 beats per minute. An hour on the treadmill, then a round on the machines Last week I ran an hour on a treadmill, followed by a round on the machines. I loved it. An hour on such a thing asks for a different mindset, and that is exactly the point. Like the window and the green dot, it is not something extra on top of the work. It is what my work stands on, the foundation for thinking. I wrote about running before, in Marathon and in Run Nicely. This time the lesson is not in the legs. Conclusion Two marathons in four weeks taught me more about work than about running. The pace you can hold is not enough. You also need room: a morning without a phone, a Sunday without screens, a view of something alive. The best in me comes up when there is room, and that room is not a reward for after the finish. It is part of the training. There is one more race on the calendar this year, purely to finish, at a slower and conscious pace. First recover. And look out of the window a bit more, ha! The best part of a finish: celebrating with friends!
