More from Sedimental
In my last post, I mentioned founding a startup. It's called FinFam, and we're building collaborative financial planning. The GitHub of money, if you will. Enough with the telling, time for the show! Here's a 3-minute demo: We launched beta this week and I couldn't be more excited. Let me tell you why. Where's this coming from? Growing up as the son of starving grad students, when it comes to money, I've been known to default to what we affectionately call "poor man brain." Cautious to a fault. But my student parents turned into scientists, so I'm eminently convinceable. I just need to see the math, or better yet, a spreadsheet. The problem? Making those spreadsheets. And more importantly, trusting them. A few years back, a friend was house hunting in the Bay Area. They came to me to talk through the famous rent-vs-buy problem. They didn't want an advisor to manage money or sell them products. They came to me because I worked in fintech and thus "knew money", even though I've only been involved in two home purchases, and wouldn't consider myself an expert. That experience crystallized something I'd been noticing everywhere: People trust their friends and family to have their best interests in mind... But can't trust them to have the best information. We can trust experts to have knowledge... but can't always trust their incentive alignment. In an era of ever-advancing information (and misinformation) glut, how do we get to a place of confidence in our hard-won justified true belief? Trust is social After 15+ years building fintech at PayPal and Stripe, I saw money movement simplified and commoditized. But the difficulty of transacting moved upstream. We made the how of buying easier, while advances in technology made the what, when, and why so much harder. From BNPL to crypto to vibecession, our economic realities aren't getting simpler. To compensate, 79% of young adults get financial guidance from social media (Forbes). Not because TikTok or YouTube has better models than Morgan Stanley, but because they trust the people sharing their stories. Millions of people are already collaborating on financial decisions. Privately on WhatsApp, obscurely on Discord, and full-blown publicly on Reddit: /r/personalfinance - 21 million members /r/financialindependence - 2.3 million /r/financialplanning - 1 million And dozens more subreddits and Internet forums (shoutout Bogleheads and Refinery29's Money Diaries). They're sharing detailed financial profiles with strangers on the internet, seeking advice from generous folks in full view. There's a fast-emerging story about AI here, and I've got whole posts dedicated to that coming soon. For now, suffice to say, we need the tools to catch up to the times. Enter FinFam FinFam[^name] lets families and friends collaborate on financial decisions with each other, using expert information without any commitments to said experts. We want to holistically solve the problem of financial decisionmaking using interaction models proven by GitHub + StackOverflow + app stores. How? First, creators publish interactive models, to FinFam's View marketplace. Then, you, a user who has a financial question or decision to make: Pick up a relevant expert view. If one doesn't exist, ask in FinFam's Q&A board. Plug in your own numbers and save it to a private workspace for your inner circle Discuss the results, with optional AI-assisted guidance as needed Move forward with confidence. We scale the expert knowledge while embracing the fundamentals of human social trust. Users get better decisions and peace of mind. Open-source meets fintech My years of work in open-source and wiki ecosystems showed me the power of collaborative, transparent tools. FinFam brings that same philosophy to personal finance. To further scale the knowledge, we make it possible for anyone to create a View. The View "source" format is XLSX, and can be edited with Google Sheets, Excel, or LibreOffice. Any published View can also be open-sourced. Just like with code, financial models are now reviewable, forkable, and improvable by the community. Curious users can check the community's math. Numbers and discussions happen with people you trust. Everything is private by default, shareable by design. What's next We launched beta this week and there are now daily spots available as we add capacity. You can sign up for early access here. I've been using it with friends and family for months now, and it has replaced Google Sheets for the financial decisions we face. There's so much more I want to share about the vision, the technology, and the journey so far. But this feels like we're off to a good start. Want to follow along? Subscribe to FinFam here and Sedimental here. If you're wondering about the name, just log in, go to your default space, create a thread, and ask Finn. ↩
Reliably tap into your creativity with the 4 Cs: Consume, critique, curate, create. This is one of my oldest ideas, finally published on the GitHub ReadME Project blog, along with a profile, in June 2022. For more like this, follow me on Twitter or Mastodon. (You can also read it in 中文 here. Thanks Dominic Huang!) We all have creative potential. Whether it gets you up in the morning or keeps you up at night, you've felt its gnaw. Turning that potential into productivity can prove challenging in an internet-connected environment that offers a constant stream of consumables. How do we pick a direction? In this guide, you'll see how to distill the elements of creativity into four deliberate stages, and how to put the process to use: Contents Consume Critique Curate Create Debugging the process Putting it into practice These 4 Cs comprise a straightforward, adaptable approach that works well in both group and solo settings. You've already started on step 1. Read on to find out what to do next. Consume Step 1: Turn passive consumption into active research. From the moment you open your eyes in the morning, you're accosted with calls to consume. Articles, videos, podcasts, the newest Wordle variant. When consumption is the default mode of our modern computing environment, how is a builder supposed to build? To create, we must first recognize its inverse: consumption. Consumption is a useful stage, but can be dangerous if it’s terminal. An infinite loop in this stage kills any chance of creation. Little is created in a vacuum. Creation still starts with consumption, albeit consumption disarmed with an intention. Turn pure consumption into active research, punctuated with critique. Critique Step 2: Capture your reactions in critiques, and research no faster than you can react. As with so many software problems of our day, the answer is simple: React. No, not the JavaScript framework, but the human act of reaction. Intentional creation starts with giving yourself pause on new inputs. Seek a reaction from yourself. A semi-structured reaction, or critique, is a time-honored practice in creative fields, like architecture. Infinite scroll may prove challenging to overcome, but before turning your consideration to the next item in your feed, activate your critical senses. Draw some conclusions. Even unvetted, they're yours. If you're finding it hard to summon a critique, this is a clear sign you're consuming faster than you can reflect. If you're not reflecting, you're not learning. You may need to go deeper on individual items, or just take a break. Your critiques have never been easier to capture, whether typed in markdown, dictated to automatic transcription, or written down in a notepad on your desk. Try opening your editor or critique tool before opening any new resources. Feel free to open one now. If you're not reflecting, you're not learning. Curate Step 3: Curate critiques into collections that act as reservoirs of creative reference. Critiques are only proto-creative output. Writing anything helps prime the creative pump, but criticism is raw reaction. You want a refined synthesis. Once you've got enough critiques under your belt, curate the positive examples into a collection. From interior designers to lab researchers to club DJs, creators recognize the value of a structured, referenceable collection. Sometimes a situation calls for urgency or direction, and an organized, well-researched collection can offer an existing solution. Sometimes, in the context of a comprehensive collection, the lack of a referenceable solution is itself a signal that it's time to invent. Curated collections become artifacts unto themselves. I've helped create a few, including 0ver.org, seealso.org, and the Awesome Python Applications list. There’s more awesome out there beyond Awesome Lists, like explorabl.es, the Cooperpress newsletters, or the “swipe file” phenomenon used among designers and content creators. There's respectable work in curation. Still, curation is more important as a stepping stone to our original higher calling. Less is more. Create Step 4: Return to your curations regularly to discover your creative path forward. Collections of a certain size tend to produce interesting findings. Patterns and gaps emerge that inspire creative next steps. As an example, while researching approaches to Python packaging, a pattern emerged that led to one of my most popular concepts/blog posts/talks, The Packaging Gradient. Whole projects can be born out of connections made with collections. My framework Clastic, which was eventually used by teams at PayPal and Wiki Loves Monuments, came out of the curated combination of pytest dependency-injection semantics with werkzeug primitives. Realistically, the majority of creation happens below the threshold of standalone artifacts. For instance, when adding a feature to an existing system, a parallel approach in a different project serves as a useful guide. I've lost track of the number of times I've swiped techniques from Awesome Python Applications, including ones used to port my dayjob's 300k SLOC codebase from Python 2 to 3. Most creative outputs have a similar lineage. Only now we have an explicit process. Debugging the process It's easy to see creations we appreciate as towering achievements that sprung fully-formed from their creators' genius. But creation comes in fits and starts. If creation comes slowly, here are a few strategies to consider: Search for a natural split in an existing collection that's getting too big, and explore what makes it interesting. Revisit an old, contentious critique and re-react. What did you get right/wrong? Pick a particular exemplar and turn it into a case study. One beautiful aspect of FOSS projects is that going deep can mean getting involved. There's nothing like proximity to a problem to inspire creative thinking. More generally, be wary of one-size-fits-all solutions; while prescriptive techniques such as the Zettelkasten Method may work for some, creation is idiosyncratic. Embrace your own process. Putting it into practice When inspiration hits, connections can form so quickly that we take for granted what goes on. When inspiration proves less willing to strike, we can keep ourselves primed for creativity by ensuring all four activities continue in balance. There are a few notable benefits of intentional creation: When you've built something, the influences are well-documented. It can be easier to involve others when there's a clear creative thread to pull on. Sharing your critiques and curations invites collaboration with other creators and curators. Self-awareness. If you're not finding your critiques crystallizing into new thoughts and ideas for projects, that's a sign you're looking at the wrong stuff. Are you following your interests or passively consuming trending content? Practically, intentional creation means consciously spending less time on consumer sites, from Twitter to Hacker News, and more time taking notes, tagging bookmarks, and creating your own knowledge base. Attempt activities that are less entertainment and more you, ultimately closing the gap between you and your creative goals. If it sounds too simple, that's because it is. You're still accountable to you, that's the hard part. But hopefully you'll find some value in this simple hierarchy that lets you check in on your own activities and make adjustments toward a more creative end. Spend less time consuming, and more time on the other three Cs. Consume only enough to allow yourself to critique, curate, and create. If you made it this far, then start now. Step 2. Use any tool or service you like, from spreadsheets to YAML, and answer this: What's your critique?
2020 was a year of reckonings. And for all that was beyond one’s control, as the year went on, I found myself pouring more and more into the one thing that felt within reach: futureproofing of the large enterprise web application I helped build, SimpleLegal. Now complete, this replatforming easily ranks in my most complex projects, and right now, holds the top spot for the happiest ending. That happiness comes at a cost, but with some the right approach that cost may not be as high as you think. Contents The Bottom Line The Setup The Outset The Traction Issues The Sentry Pivot The New Road Committing to transactions The truly atomic request Transactional test setup Better than best practices The utility of namespaces Coverage tools Flattening database migrations Easing onto the stack The Rollout The Aftermath The Bottom Line We took SimpleLegal’s primary product, a 300,000 line Django-1.11-Python 2.7-Redis-Postgres-10 codebase, to a Django 2.2-Python 3.8-Postgres-12 stack, on-schedule and without major site incidents. And it feels amazing. Speaking as tech lead on the project, what did it look like? For me, something like this: But as Director of Engineering, what did it cost? 3.5 dev years and just about $2 per line of code. And I'm especially proud of that result, because along the way, we also substantially improved the speed and reliability of both the site and development process itself. The product now has a bright future ahead, ready to shine in sales RFPs and compliance questionnaires. Most importantly, there’ll be no worrying about when to delicately break it to a candidate that they’ll be working with unsupported technology. In short, a large, solid investment that’s already paying for itself. If you just came here for the estimate we wish we had, you've got it. This post is all about how your team can achieve the same result, if not better. The Setup The story begins in 2013, when a freshly YC-incubated SimpleLegal made all the right decisions for a new SaaS LegalTech company: Python, Django, Postgres, Redis. In classic startup fashion, features came first, unless technology was a blocker. Packages were only upgraded incidentally. By 2019, the end of this technical runway had drawn near. While Python 2 may be getting extended support from various vendors, there were precious few volunteers in sight to do Django 1 CVE patches in 2021. A web framework’s a riskier attack surface, so we finally had our compliance forcing function, and it was time to pay off our tech debt. The Outset So began our Tech Refresh replatforming initiative, in Q4 2019. The goal: Upgrade the stack while still shipping features, like changing the tires of a moving car. We wanted to do it carefully, and that would take time. Here are some helpful ground rules for long-running projects: Any project that gets worked on 10+ hours per week deserves a 30-minute weekly sync. Every recurring meeting deserves a log. Put it in the invite. Use that Project Log to record progress, blockers, and decisions. It’s a marathon, not a sprint. Avoid relying on working nights, weekends, and holidays. We started with a sketch of a plan that, generously interpreted, ended up being about halfway correct. Some early guesses that turned into successes: Move to pip-tools and unpin dependencies based on extensive changelog analysis. Identify packages without py23 compatible versions. (Though we’ve since moved to poetry.) Add line coverage reporting to CI Revamp internal testing framework to allow devs to quickly write tests More on these below. Other plans weren’t so realistic: Take our CI from ~60% to 95% line coverage in 6 months Parallelized conversion of app packages over the course of 3 months Use low traffic times around USA holidays (Thanksgiving, Christmas, New Years) to gradually roll onto the new app before 2021. We were young! As naïve as we were, at least we knew it would be a lot of work. To help shoulder the burden, we scouted, hired, and trained three dedicated off-shore developers. The Traction Issues Even with added developers, by mid-2020 it was becoming obvious we were dreaming about 95% coverage, let alone 100%. Total coverage may be best practice, but 3.5 developers couldn’t cover enough ground. We were getting valuable tests, and even finding old bugs, but if we stuck with the letter of the plan, Django 2 would end up being a 2022 project. At 70%, we decided it was time to pivot. We realized that CI is more sensitive than most users for most of the site. So we focused in on testing the highest impact code. What’s high-impact? 1) the code that fails most visibly and 2) the code that’s hardest to retry. You can build an inventory of high-impact code in under a week by looking at traffic stats, batch job schedules, and asking your support staff. Around 80% of the codebase falls outside that high-traffic/high-impact list. What to do about that 80%? Lean in on error detection and fast time-to-fix. The Sentry Pivot One nice thing about startup life is that it’s easy to try new tools. One practice we’ve embraced at SimpleLegal is to reserve every 5th week for developers to work on the development process itself, like a coordinated 20% time. Even the best chef can’t cook five-star food in a messy kitchen. This was our way of cleaning up the shop and ultimately speeding up the ship. During one such period, someone had the genius idea to add dedicated error reporting to the system, using Sentry. Within a day or two, we had a site you could visit and get stack traces. It was pretty magical, and it wasn’t until Tech Refresh that we realized that while integration takes one dev-day, full adoption can take a team months. You see, adding Sentry to a mature-but-fast-moving system means one thing: noise. Our live site was erroring all the time. Most errors weren’t visible or didn’t block users, who in some cases had quietly learned to work around longstanding site quirks. Pretty quickly, our developers learned to treat Sentry as a repository of debugging information. A Sentry event on its own wasn’t something to be taken seriously in 2019. That changed in 2020, with the team responsible for delivering a seamless replatform needing Sentry to be something else: a responsive site quality tool. How did we get there? First step, enhance the data flowing into Sentry by following these best practices: Split up your products into separate Sentry projects. This includes your frontend and backend. Tag your releases. Don’t tag dev env deployments with the branch, it clutters up the Releases UI. Add a separate branch tag for searches. Split up your environments. This is critical for directing alerts. Our Sentry client environment is configured by domain conventions and Django’s sites framework. If it helps, here's a baseline, we use these environments: Production: Current official release. DevOps monitored. Sandbox: Current official release (some companies do next release). Used by customers to test changes. DevOps monitored. Demo/Sales: Previous official release. Mostly internal traffic, but external visibility at prospect demo time. DevOps monitored. Canary: Next official release. Otherwise known as staging. Internal traffic. Dev monitored. ProdQA: Current official release. Used internally to reproduce support issues. Dev monitored. QA: Dev branches, dev release, internal traffic. Unmonitored debugging data. Local test/CI: Not published to Sentry by default. With issues finally properly tagged and searchable, we used Sentry’s new Discover tool to export issues weekly, and prioritize legacy errors. To start, we focused on high-visibility production errors with non-internal human users. Our specific query: has:user !transaction:/api/* event.type:error !user.username:*@simplelegal.* We triaged into 4 categories: Quick fix (minor bug), Quick error (turn an opaque 500 error into a actionable 400 of some form), Spike (larger bug, requires research), and Silence (using Sentry’s ignore feature). Over 6 weeks we went from over 2500 weekly events down to less than 500. Further efforts have gotten us under 100 events per week, spread across a handful of issues, which is more than manageable for even a lean team. While "Sentry Zero" remains the ideal, we achieved and maintained the real goal of a responsive flow, in large part thanks to the Slack integration. Our team no longer hears about server errors from our Support team. In fact, these days, we let them know when a client is having trouble and we’ve got a ticket underway. And it really is important to develop close ties with your support team. Embedded in our strategy above was that CI is much more sensitive than a real user. While perfection is tempting, it’s not unrealistic to ask a bit of patience from an enterprise user, provided your support team is prepared. Sync with them weekly so surprise is minimized. If they’re feeling ambitious, you can teach them some Sentry basics, too. The New Road With noise virtually eliminated, we were ready to move fast. While the lean-in on fast-fixing Sentry issues was necessary, a strong reactive game is only useful if there are proactive changes being pushed. Here are some highlights we learned when making those changes: Committing to transactions Used properly, rollbacks can make it like errors never happened, the perfect complement to a fast-fix strategy. The truly atomic request Get as much as possible into the transactions. Turn on ATOMIC_REQUESTS, if you haven’t already. Some requests do more than change the database, though, like sending notifications and enqueuing background tasks. At SimpleLegal, we rearchitected to defer all side effects (except logging) until a successful response was being returned. Middleware can help, but mainly we achieved this by getting rid of our Redis queue, and switching to a PostgreSQL-backed task queue/broker. This arrangement ensures that if an error occurs, the transaction is rolled back, no tasks are enqueued, and the user gets a clean failure. We spot the breakage in Sentry, toggle over to the old site to unblock, and their next retry succeeds. Transactional test setup Transactionality also proved key to our testing strategy. SimpleLegal had long outgrown Django’s primitive fixture system. Most tests required complex Python to set up, making tests slow to write and slow to run. To speed up both writing and running, we wrapped the whole test session in a transaction, then, before any test cases run, we set up exemplary base states. Test cases used these base states as fixtures, and rolled back to the base state after every test case. See this conftest.py excerpt for details. Better than best practices Software scenarios vary so widely, there’s an art to knowing which advice isn’t for you. Here’s an assortment of cul de sacs we learned about firsthand. The utility of namespaces Given how code is divided into modules, packages, Django apps, etc., it may be tempting to treat those as units of work. Don’t start there. Code divisions can be pretty arbitrary, and it’s hard to know when you’ve pulled on a risky thread. Assuming there are automated refactorings, as in a 2to3 conversion, start by porting by type of transformation. That way, one need only review a command and a list of paths affected. Plus, automated fixes necessarily follow a pattern, meaning more people can fix bugs arising from the refactor. Coverage tools Coverage was a mixed bag for us. Obviously our coverage-first strategy wasn’t tenable, but it was still useful for prioritization and status checks. On a per-change basis, we found coverage tools to be somewhat unreliable. We never got to the bottom of why coverage acted nondeterministically, and we left the conclusion at, “off-the-shelf tools like codecov are probably not targeted at monorepos of our scale.” In running into coverage walls, we ended up exploring many other interpretations of coverage. For us, much higher-priority than line coverage were “route coverage” (i.e., every URL has at least one integration test) and “model repr coverage” (i.e., every model object had a useful text representation, useful for debugging in Sentry). With more time, we would have liked to build tools around those, and even around online-profiling based coverage statistics, to prioritize the highest traffic lines, not just the highest traffic routes. If you’ve heard of approaches to these ends, we’d love to discuss them with you. Flattening database migrations On the surface, reducing the number of files we needed to upgrade seems logical. Turns out, flattening migrations is a low-payoff strategy to get rid of files. Changing historical migration file structure complicated our rollout, while upgrading migrations we didn’t flatten was straightforward. Not to mention, if you just wanted the CI speedup, you can take the same page from the Open EdX Platform that we did: build a base DB cache that you check in every couple months. Turns out, you can learn a lot from open-source applications. Easing onto the stack If you have more than one application, use the smaller, simpler application to pilot changes. We were lucky enough to have a separate app whose tests ran faster, making for a tighter development loop we coul learn from. Likewise, if you have more than one production environment, start rollouts with the one with the least impact. Clone your CI jobs for the new stack, too. They’ll all fail, but resist the urge to mark them as optional. Instead, build a single-file inventory of all tests and their current testing state. We built a small extension for our test runner, pytest, which bulk skipped tests based on a status inventory file. Then, ratchet: unskip and fix a test, update the file, check that tests pass, and repeat. Much more convenient and scannable than pytest mark decorators spread throughout the codebase. See this conftest.py excerpt for details. The Rollout In Q4 2020, we doubled up on infrastructure to run the old and new sites in parallel, backed by the same database. We got into a loop of enabling traffic to the new stack, building a queue of Sentry issues to fix, and switching it back off, while tracking the time. After around 120 hours of new stack, strategically spread around the clock and week, enough organizational confidence had been built that we could leave the site on during our most critical hours: Mondays and Tuesdays at the beginning of the month. The sole hiccup was an AWS outage Thanksgiving week. At this point we were ahead of schedule, and enough confidence had been built in our fast-fix workflow that we didn’t need our original holiday testing windows. And for that, many thanks were given. We kept at the fast-fix crank until we were done. Done isn't when the new system has no errors, it's when traffic on the new system has fewer events than the old system. Then, fix forward, and start scheduling time to delete the scaffolding. The Aftermath So, once you’re on current LTS versions of Django, Python, Linux, and Postgres, job complete, right? Thankfully, tech debt never quite hits 0. While updating and replacing core technologies on a schedule is no small feat, replacing a rusty part with a shiny one doesn’t change a design. Architectural tech debt -- mistakes in abstractions, including the lack thereof -- can present an even greater challenge. Solutions to those problems don’t generalize between projects as cleanly, but they do benefit from up-to-date and error-free foundations. For all the projects looking to add tread to their technical tires, we hope this retrospective helps you confidently and pragmatically retrofit your stack for years to come. Finally, big thanks to Uvik for the talent connection, and the talent: Yaroslav, Serhii, and Oleh. Shoutouts to Kurt, Justin, and Chris, my fellow leads. And the cheers to business leadership at SimpleLegal and everywhere, for seeing the value in maintainability.
Thought I'd take a Sunday afternoon to reflect on, oh I don't know, a decade. Been a long ten years, but it's flown past. This particular decade happens to coincide with my first years of full-time professional software engineering. The Quantity I can't possibly summarize it all, and if I tried, it'd still be colored by what's on my mind right now. But I can point to the artifacts I tried to leave along the way: Twitter FWIW1 (2008+) ~20 Open-Source Projects (2012+) ~15 Hatnote Projects (2013+, follow us) ~25 entries on this blog (2015+) +7 here (2014-2016) Not including pythondoeswhat.com or blog.hatnote.com (or other posts on the blogs only real heads know) ~10 Talks (2016+) Lest I forget: O'Reilly's Enterprise Software with Python (2016) And several podcast/media appearances calver.org (2016) and 0ver.org (2018) (Versioning is a fun pastime) Pyninsula (2017+) - YouTube, Meetup, Email Announce Taking a chronological look at each of the above, I'm relieved to see obvious growth. If I were to highlight one resource, it would probably be the talks. Despite the stress of preparation and delivery, I'm least concerned with having a massive miscommunication when we're all in the room and I can see the points hitting home. It's impossible to pick a favorite, but Ask the Ecosystem (2019), the Restructuring Data lightning talk (2018), and The Packaging Gradient (2017) seem like audience faves from where I'm sitting. The Quality Each project, post, and talk had its own reward, but I guess I've got more than just those to show for the decade. On the more profit-driven side, I built tools and teams at PayPal, but once I could manage the risk, I got to dip into startups for the last few years. Lucky for me, it wasn't a total bust, and the wife and I bought a place in my favorite neighborhood (in the USA). Not a millionaire, but I'm hoping and working for a world where no one has to be. More recently, the Python Software Foundation made me a Fellow. This isn't something I can be nonchalant about, and I'm not going to understate how much this means, to me, working in a field like software, where concrete symbols of progress are alternatingly elusive and vanishing. Plus it's Python, and reciprocated love is nice. I have hundreds of people to thank for helping me reach this point, and I have to thank the PSF for dedicating the time to ramping up these awards. They've convinced me more than ever that we need more institutions to build this sort of advancement. To all of you, thank you. The Struggle I like to think I managed to do all of the above while staying away from industry hype, on the principle that massive speculative capital influx isn't where real value is added to society, and doesn't generate the kind of innovation that excites me. I may have been naïve, but I came to Silicon Valley with an idea about the transformative power of software. Changing times may illustrate a grittier interpretation than the one I had and have, but I continue to hold dear software's potential for positive impact. If you've felt that vision waver, let me tell you, you're not alone. In the past decade, I've seen too many engineers sucked in by new technologies and ventures, only to find themselves alienated from their work. Episodes ranging from an afternoon lost to debugging Docker/k8s clusters, to years of work disappearing at the end of a VC runway. Nothing has been harder to watch than those bedraggled-but-persistent idealists regroup, each time a bit more cynical than the last. Even if its seeming intractibility has taken it from the center stage, the burnout conversation continues to smolder, because there's no issue realer. I know; I released more ceramics than software back in 2014. Some problems can be solved by paying the maintainers, but I think the vastly bigger issue is around losing the human connection between the real effort software takes and the real benefits it brings, combined with FOSS's dearth of collaborators in supporting roles (QA, product/project/release management). That's why I'm incredibly thankful for the Wikimedia community for always being there, patient with schedules and issues, as long as the software got the job done. It can be a challenge to juggle projects, but I tell every budding engineer: find that direct connection to people who will appreciate your work, and avoid cynicism at all costs. There are some interesting prospects in the works, but I'm keeping this post retro. Besides, if 2029 rolls around and all I did was break even with 2009-19, I don't see how I can be disappointed. Thanks again for everything in 201X, and for sticking with me in 202X. Despite using Twitter for over a decade, the process of tweeting feels so perfunctory, and the service itself so tenuous, that I still can't bring myself to invest the time. I mostly use it to crosspost my blog posts or help friends promote their posts/projects. But until I start an email newsletter, or really get on top of yak.party, it's still the best I got for announcing where I'm speaking next. ↩
More in programming
Reading my recent computing retrospective, I realised there was a big section missing: the people in my life that made an impact and helped shape my career. Outside my immediate family, one person made an outsized contribution, and I’m fairly certain that without his influence my life would have taken a very different path. The fact that I’m still here in 2026, still writing code and being fortunate enough to have a career in something I love is testament to him. So I’d like to take a few moments to talk about my old secondary school teacher, George Dryden. Denied Back in 1995, I had a problem. I knew I wanted to study computing at university and build a career out of my passion, but there was a snag. For those unfamiliar with the UK schooling system, when you’re 15-16 you take a set of GCSE exams in a broad range of subjects. After that, you pick around 3 subjects to really focus on over a period of 2 years. These are called A Levels, and they are a big step up and are meant to prepare you for a degree-level course at university. Admission to university is also governed by these results - if you want to study computing, you’re going to need a computing A-Level, and most universities will only accept you (or “make an offer”) if you achieve a certain grade. And whilst I had taken computing at a GCSE level, my school did not offer a computing A-Level course. I instead had to settle on “Design & Technology”, which just didn’t inspire me. Instead of working on my portfolio and projects, I spent most of my time daydreaming and writing code on the Acorn Archimedes computers that were the staple of every 90s UK school. No disrespect to the teachers - they were all awesome - but it just wasn’t for me. I was miserable, and by the end of my first year, I was well on my way to failing outright with my entire future plans seemingly going up in smoke. Someone noticed That’s when George stepped in. He’d taught me computing right the way through my GCSEs, and with no A-Level course on offer, that was officially where his involvement was supposed to have ended. It didn’t. He had noticed my constant presence in the computing labs - before and after school, during lunch breaks, free “study” periods - working on some little pet project or digging into RISC OS internals. I remember him as warm, with a wicked, dry sense of humour, and a refreshingly spiky attitude to authority - I always got the sense he’d worked out for himself which rules were worth taking seriously and which ones weren’t. And he always had time for me. I spent years pestering him with questions that had nothing to do with anything on the syllabus, and he’d always find a way to answer them that actually made sense. He was just as supportive of my odd little obsessions. At one point I’d got deep into the BBS scene, which I thought was the coolest thing ever, and decided what the school really needed was an internal BBS running on its own network. So I wrote one. It was deeply cringeworthy, obviously - but George helped me put posters up around the school advertising it, and even gave it a mention in assembly one morning. I think about five people in total ever checked it out. It didn’t matter: a teacher had stood up in front of the entire school and treated my weird little project like it was worth something, and that was a hugely validating moment for me. Off the books He recognised the passion, and eventually he made a suggestion: What if I quit the Design and Technology course, and instead attempt the A-level course myself? Personally, I also suspect he was enjoying himself. There was some internal school politics behind why computing wasn’t offered at A-Level in the first place - I never knew the details - and I think the prospect of one of his students simply going out and getting the qualification anyway appealed to him on two separate levels. It would get me where I wanted to go, and it would wind up exactly the right people. It wouldn’t be easy, he warned. The school would be against it, plus it was a two-year course which I’d have to cram into one year. I’d have to do it all myself - studying, lesson planning, coursework - he couldn’t help me in an official capacity, but he said he’d advocate for me and help where he could. If I had assignments, he’d send them off to be graded and would give feedback in his own time. He’d enter me in for the exams and also gave me a set of keys to the computer lab so I could use it whenever I needed. It was the first time anybody outside my own family had really shown faith in my abilities and encouraged me to take a stand. It was a pivotal moment for me - I realised if I wanted something badly enough I would have to fight for it, but I could still make it happen. I didn’t have to take “NO” for an answer - a lesson I think I picked up from watching him as much as from anything he ever actually said to me. After a few weeks of dithering, I took the jump. I remember a few awkward meetings with the school administration but thanks to his behind-the-scenes work, I was soon following my dream. An intense year And yes, it was bloody hard work. I had to condense an entire two year course into under a year, be disciplined enough to produce my own study plan, and be critical enough of my own shortcomings that I could focus my study where it was needed. I pretty much lived and breathed it for months straight and was more-or-less a permanent fixture in the labs or school library poring over my course books. I’d make lists of questions and chat to George over lunch, and he’d provide guidance and encouragement. It was a lonely way to learn with no classmates to compare notes with, no lessons to turn up to, and right up until the end I had no real idea whether any of it was good enough - but bit by bit, it started to feel like something I could actually pull off. And sure enough, in the late spring of 1996, I sat down in an exam hall with my fellow students, the only one with an A-Level computing question paper in front of me. The final exam went by in a blur - I can only remember a few of the questions now (and a peculiar obsession with the Pascal language) - but I do remember the euphoria as the invigilator called “time’s up, pens down, close your papers NOW”. I had done it. A few nerve-wracking months later, my Mum drove me into school to pick up my results. I ripped open the envelope and saw it - I’d passed with an A grade! I literally ran up the stairs to George’s office next to the computing labs to thank him personally. I’d taken my camera into school to take a few last photos for memory’s sake and snapped this photo of him before I walked out the school gates for the last time: A different path Because of him, I managed to get into my university of choice, studying computing with a focus on networks. Because of that, I landed my first job working as a “webmaster”, and my career since has been one of the highlights of my life. All these years later, it’s a real privilege to be able to get up each morning and actively look forward to working in an industry I love. Without George stepping up for me and encouraging me to believe in myself, none of that would have happened. I wouldn’t have had the career I have, and I wouldn’t be where I am now. I met my wife when we both worked at a software company - she sat at the desk behind me - so even my home and family life can be traced back to that spring of 1996. And the A-Level itself was only half of what I took away from that year. The qualification opened the door to university, but the lesson that came with it was every bit as important: that a “no” isn’t always the end of it, and that sometimes the answer can be argued with. I’m so proud of what I managed to achieve all those years ago, and even more thankful to have had someone like George in my life to put me on the right track. Mr. Dryden I did see him again after I left. He drank in one of my local pubs - a pub I’d been going to for a good while before I was technically old enough to be in it - and I’d say hello if I spotted him in there, mostly in the months before I moved away to university. After that it was only a handful of times. For years I’d find myself scanning the room whenever I was back home and in for a pint, half expecting him to be at the bar. At some point I stopped seeing him altogether and eventually moved across the country. The trouble was I never really knew how to talk to him outside of school. He was always Mr. Dryden, or just “Sir”, I don’t think I ever once called him George to his face! I was (and still am if I’m honest) fairly socially awkward, and I never worked out how to phrase the thing I actually wanted to say: that he had changed the entire direction of my life, and I wasn’t sure he knew it. So instead I’d say hello, and ask how he was, talk about nothing much, and go back to my friends. Epilogue Sadly, 3 years ago now, I opened the latest issue of my old school alumni newsletter to read that he’d passed away. The photo at the start of this article was taken from his obituary article and I read that he’d had a long illness and had suffered from dementia at the end. I did write to him years ago by email - I don’t know if he ever got it, or was in any capacity to understand what he’d done for me, but I hope so. There’s an old saying by one of my favourite authors (Terry Pratchett) that “no one is finally dead until the ripples they cause in the world die away”. In one of his books, a character keeps the memory of his son alive by passing his name along a series of telegraph towers. It’s in that spirit that I’m writing this post - I debated it for many years as it’s very personal to me and I also have no contact with any of George’s family so I have no idea what they would make of it all. But even though it’s 30+ years ago now, I will never forget him or what he did for me - and at least now, if someone searches his name it’ll be recorded here for as long as I’m alive and running this site. Thank you, Sir. George Dryden 1942-2023
Let’s step inside the kernel and understand how it implements copy-on-write and what are its implications for the performance of user-space systems
Yesterday, I received this email as a response to You Can't Vibe Code Love. It's such a remarkable and powerful statement that I asked permission to share it here, in its entirety, with personal information redacted: Hey Jeff, Hope you and your family are doing well.
A frustrated Reddit post about being a condom between an AI and production made the rounds in our team. Here is why I think the opposite is true and what it means for how we review code, plan work and think.
And here we are three years after I wrote about the Google Pixel Fold being announced, followed now with the announcement of the iPhone Duo...(I have questions about the naming by the way). Four years ago I was talking about web primitives in the platform for the Surface Duo. My how time flies. There are CSS media features, a Viewport Segments API, a Device Posture API but Chromium based browsers are the only ones currently supporting these things. I haven't been able to find any signal yet on whether Safari will support these things in the web platform as the developer docs focus on application development. If you're interested in trying out the platform features, you can emulate the Surface Duo and Galaxy Z Fold in the developer tools. And if you're thinking, do I really have to have my website adapt to two screens? The answer is no. Adding a design to an application or dual screen makes sense if you have an experience that has two simulataneous contexts that are useful e.g. a list of email messages/inbox on one screen, an open message, email thread or email composer on the other. Here's one of my talks from 2022 if you're interested in learning more about what's available in the browser for dual screen/foldable devices. Happy building :)