Full Width [alt+shift+f] Shortcuts [alt+shift+k]
Sign Up [alt+shift+s] Log In [alt+shift+l]
44
When I started this blog in 2007, a running theme was "Can interactive experiences like video games be written in a functional style?" These are programs heavily based around mutable state. They evolve, often drastically, during development, so there isn't a perfect up-front design to architect around. These were issues curiously avoided by the functional programming proponents of the 1980s and 1990s. It's still not given much attention in 2016 in either. I regularly see excited tutorials about mapping and folding and closures and immutable variables, and even JavaScript has these things now, but there's a next step that's rarely discussed and much more difficult: how to keep the benefits of immutability in large and messy programs that could gain the most from functional solutions--like video games. Before getting to that, here are the more skeptical functional programming articles I wrote, so it doesn't look like I'm a raving advocate: I took a straightforward, arguably naive,...
over a year ago

Improve your reading experience

Logged in users get linked directly to articles resulting in a better reading experience. Please login for free, it takes less than 1 minute.

More from Programming in the 21st Century

So Long, Prog21

I always intended "Programming in the 21st Century" to have a limited run. I knew since the entry from January 1, 2010, that I needed to end it. It just took a while.Recovering Programmer And now, an explanation. I started this blog to talk about issues tangentially related to programming, about soft topics like creativity and inspiration and how code is a medium for implementing creative visions. Instead I worked through more technical topics that I'd been kicking around over the years. That was fun! is something I would have loved to read in 1998. More than once I've googled around and ended up back at one of my essays.Purely Functional Retrogames As I started shifting gears and getting back toward what I originally wanted to do, there was one thing that kept bothering me: the word in the title.programming I don't think of myself as a programmer. I write code, and I often enjoy it when I do, but that term is both limiting and distracting. I don't want to program for its own sake, not being interested in the overall experience of what I'm creating. If I start thinking too much about programming as a distinct entity then I lose sight of that. Now that I've exhausted what I wanted to write about, I can clear those topics out of my head and focus more on using technology to make fun things.programmer Thanks for reading! It's hard to sum up 200+ articles, but here's a start. This is not even close to a full index. See the if you want everything. (There are some bits in there.)archivesodd Also see the for all of the functional programming articles.previous entry is something I wrote in 2004 which used to be linked from every prog21 entry.Programming as if Performance Mattered widely linked popular on creativity others that I like Erlang retro Things That Turbo Pascal is Smaller Than Do You Really Want to be Doing This When You're 50? Organizational Skills Beat Algorithmic Wizardry Retiring Python as a Teaching Language Computer Science Courses that Don't Exist, But Should Five Memorable Books About Programming A Spellchecker Used to Be a Major Feat of Software Engineering Want to Write a Compiler? Just Read These Two Papers. On Being Sufficiently Smart Optimizing for Fan Noise Free Your Technical Aesthetic from the 1970s Advice to Aimless, Excited Programmers Write Code Like You Just Learned How to Program Don't Distract New Programmers with OOP Recovering From a Computer Science Education Don't Fall in Love With Your Technology A Complete Understanding is No Longer Possible Solving the Wrong Problem This is Why You Spent All that Time Learning to Program We Who Value Simplicity Have Built Incomprehensible Machines Your Coding Philosophies are Irrelevant The Silent Majority of Experts Hopefully More Controversial Programming Opinions How much memory does malloc(0) allocate? The Pure Tech Side is the Dark Side Flickr as a Business Simulator How to Think Like a Pioneer What Do People Like? Accidental Innovation, Part 1 If You're Not Gonna Use It, Why Are You Building It? It's Like That Because It Has Always Been Like That Trapped by Exposure to Pre-Existing Ideas Get Good at Idea Generation You Can't Sit on the Sidelines and Become a Philosopher The Software Developer's Sketchbook Design is Expensive Why Doesn't Creativity Matter in Tech Recruiting? Deriving Forth Tales of a Former Disassembly Addict Living Inside Your Own Black Box Tricky When You Least Expect It The Most Important Decisions are Non-Technical Things to Optimize Besides Speed and Memory All that Stand Between You and a Successful Project are 500 Experiments The UNIX Philosophy and a Fear of Pixels Dangling by a Trivial Feature Documenting the Undocumentable You Don't Want to Think Like a Programmer You Don't Read Code, You Explore It If You Haven't Done It Before, All Bets Are Off What Can You Put in a Refrigerator? The Same User Interface Mistakes Over and Over Fun vs. Computer Science A Deeper Look at Tail Recursion in Erlang My Road to Erlang Garbage Collection in Erlang How to Crash Erlang Eleven Years of Erlang A Ramble Through Erlang IO Lists A Concurrent Language for Non-Concurrent Software A Peek Inside the Erlang Compiler Evolution of an Erlang Style A Personal History of Compilation Speed Slow Languages Battle Across Time How Much Processing Power Does it Take to be Fast? 8-Bit Scheme: A Revisionist History Stumbling Into the Cold Expanse of Real Programming Why Do Dedicated Game Consoles Exist? Lost Lessons from 8-Bit BASIC Programming Modern Systems Like It Was 1984

over a year ago 32 votes
Progress Bars are Surprisingly Difficult

We've all seen progress bars that move slowly for twenty minutes, then rapidly fill up in the last 30 seconds. Or the reverse, where a once speedy bar takes 50% of the time covering the last few pixels. And bars that occasionally jump backward in time are not the rarity you'd expect them to be. Even this past month, when I installed the macOS Sierra update, the process completed when the progress bar was only two-thirds full. DOOM 2016 has a circular progress meter for level loads, with the percent-complete in the center. It often sits for a while at 0%, gets stuck at 74% and 99%, and sometimes finishes in the 90s before reaching 100%. Clearly this is not a trivial problem, or these quirks would be behind us. Conceptually, a perfect progress bar is easy to build. All you need to know is exactly how long the total computation will take, then update the bar in its own thread so it animates smoothly. Simple! Why do developers have trouble with this? Again, all you need to know is ...exactly how long Oh. You could time it with a stopwatch and use that value, but that assumes your system is the standard, and that other people won't have faster or slower processors, drives, or internet connections. You could run a little benchmark and adjust the timing based on that, but there are too many factors. You could refine the estimate mid-flight, but this is exactly the road that leads to the bar making sudden jumps into the past. It's all dancing around that you can't know ahead of time exactly how long it should take for the progress bar to go from empty to full. There's a similar problem in process scheduling, where there are a number of programs to run sequentially in batch mode. One program at a time is selected to run to completion, then the next. If the goal is to have the lowest average time for programs being completed, then best criteria for choosing the next program to run is the one with the shortest execution time (see ). But this requires knowing how long each program will take before running it, and that's not possible in the general case.shortest job next And so the perfect progress bar is forever out of reach, but they're still useful, as established by Brad Allan Meyers in his 1985 paper ("The importance of percent-done progress indicators for computer-human interfaces"). But "percent-done" of what? It's easy to map the loading of a dozen similarly sized files to an overall percentage complete. Not so much when all kinds of downloading and local processing is combined together into a single progress number. At that point the progress bar loses all meaning except as an indication that there's some sort of movement toward a goal, and that mostly likely the application hasn't hasn't locked up. (If you liked this, you might enjoy .)An Irrational Fear of Files on the Desktop

over a year ago 40 votes

More in programming

An Analysis of Links From The White House’s “Wire” Website

A little while back I heard about the White House launching their version of a Drudge Report style website called White House Wire. According to Axios, a White House official said the site’s purpose was to serve as “a place for supporters of the president’s agenda to get the real news all in one place”. So a link blog, if you will. As a self-professed connoisseur of websites and link blogs, this got me thinking: “I wonder what kind of links they’re considering as ‘real news’ and what they’re linking to?” So I decided to do quick analysis using Quadratic, a programmable spreadsheet where you can write code and return values to a 2d interface of rows and columns. I wrote some JavaScript to: Fetch the HTML page at whitehouse.gov/wire Parse it with cheerio Select all the external links on the page Return a list of links and their headline text In a few minutes I had a quick analysis of what kind of links were on the page: This immediately sparked my curiosity to know more about the meta information around the links, like: If you grouped all the links together, which sites get linked to the most? What kind of interesting data could you pull from the headlines they’re writing, like the most frequently used words? What if you did this analysis, but with snapshots of the website over time (rather than just the current moment)? So I got to building. Quadratic today doesn’t yet have the ability for your spreadsheet to run in the background on a schedule and append data. So I had to look elsewhere for a little extra functionality. My mind went to val.town which lets you write little scripts that can 1) run on a schedule (cron), 2) store information (blobs), and 3) retrieve stored information via their API. After a quick read of their docs, I figured out how to write a little script that’ll run once a day, scrape the site, and save the resulting HTML page in their key/value storage. From there, I was back to Quadratic writing code to talk to val.town’s API and retrieve my HTML, parse it, and turn it into good, structured data. There were some things I had to do, like: Fine-tune how I select all the editorial links on the page from the source HTML (I didn’t want, for example, to include external links to the White House’s social pages which appear on every page). This required a little finessing, but I eventually got a collection of links that corresponded to what I was seeing on the page. Parse the links and pull out the top-level domains so I could group links by domain occurrence. Create charts and graphs to visualize the structured data I had created. Selfish plug: Quadratic made this all super easy, as I could program in JavaScript and use third-party tools like tldts to do the analysis, all while visualizing my output on a 2d grid in real-time which made for a super fast feedback loop! Once I got all that done, I just had to sit back and wait for the HTML snapshots to begin accumulating! It’s been about a month and a half since I started this and I have about fifty days worth of data. The results? Here’s the top 10 domains that the White House Wire links to (by occurrence), from May 8 to June 24, 2025: youtube.com (133) foxnews.com (72) thepostmillennial.com (67) foxbusiness.com (66) breitbart.com (64) x.com (63) reuters.com (51) truthsocial.com (48) nypost.com (47) dailywire.com (36) From the links, here’s a word cloud of the most commonly recurring words in the link headlines: “trump” (343) “president” (145) “us” (134) “big” (131) “bill” (127) “beautiful” (113) “trumps” (92) “one” (72) “million” (57) “house” (56) The data and these graphs are all in my spreadsheet, so I can open it up whenever I want to see the latest data and re-run my script to pull the latest from val.town. In response to the new data that comes in, the spreadsheet automatically parses it, turn it into links, and updates the graphs. Cool! If you want to check out the spreadsheet — sorry! My API key for val.town is in it (“secrets management” is on the roadmap). But I created a duplicate where I inlined the data from the API (rather than the code which dynamically pulls it) which you can check out here at your convenience. Email · Mastodon · Bluesky

22 hours ago 2 votes
Building a container orchestrator

Kubernetes is not exactly the most fun piece of technology around. Learning it isn’t easy, and learning the surrounding ecosystem is even harder. Even those who have managed to tame it are still afraid of getting paged by an ETCD cluster corruption, a Kubelet certificate expiration, or the DNS breaking down (and somehow, it’s always the DNS). Samuel Sianipar If you’re like me, the thought of making your own orchestrator has crossed your mind a few times. The result would, of course, be a magical piece of technology that is both simple to learn and wouldn’t break down every weekend. Sadly, the task seems daunting. Kubernetes is a multi-million lines of code project which has been worked on for more than a decade. The good thing is someone wrote a book that can serve as a good starting point to explore the idea of building our own container orchestrator. This book is named “Build an Orchestrator in Go”, written by Tim Boring, published by Manning. The tasks The basic unit of our container orchestrator is called a “task”. A task represents a single container. It contains configuration data, like the container’s name, image and exposed ports. Most importantly, it indicates the container state, and so acts as a state machine. The state of a task can be Pending, Scheduled, Running, Completed or Failed. Each task will need to interact with a container runtime, through a client. In the book, we use Docker (aka Moby). The client will get its configuration from the task and then proceed to pull the image, create the container and start it. When it is time to finish the task, it will stop the container and remove it. The workers Above the task, we have workers. Each machine in the cluster runs a worker. Workers expose an API through which they receive commands. Those commands are added to a queue to be processed asynchronously. When the queue gets processed, the worker will start or stop tasks using the container client. In addition to exposing the ability to start and stop tasks, the worker must be able to list all the tasks running on it. This demands keeping a task database in the worker’s memory and updating it every time a task change’s state. The worker also needs to be able to provide information about its resources, like the available CPU and memory. The book suggests reading the /proc Linux file system using goprocinfo, but since I use a Mac, I used gopsutil. The manager On top of our cluster of workers, we have the manager. The manager also exposes an API, which allows us to start, stop, and list tasks on the cluster. Every time we want to create a new task, the manager will call a scheduler component. The scheduler has to list the workers that can accept more tasks, assign them a score by suitability and return the best one. When this is done, the manager will send the work to be done using the worker’s API. In the book, the author also suggests that the manager component should keep track of every tasks state by performing regular health checks. Health checks typically consist of querying an HTTP endpoint (i.e. /ready) and checking if it returns 200. In case a health check fails, the manager asks the worker to restart the task. I’m not sure if I agree with this idea. This could lead to the manager and worker having differing opinions about a task state. It will also cause scaling issues: the manager workload will have to grow linearly as we add tasks, and not just when we add workers. As far as I know, in Kubernetes, Kubelet (the equivalent of the worker here) is responsible for performing health checks. The CLI The last part of the project is to create a CLI to make sure our new orchestrator can be used without having to resort to firing up curl. The CLI needs to implement the following features: start a worker start a manager run a task in the cluster stop a task get the task status get the worker node status Using cobra makes this part fairly straightforward. It lets you create very modern feeling command-line apps, with properly formatted help commands and easy argument parsing. Once this is done, we almost have a fully functional orchestrator. We just need to add authentication. And maybe some kind of DaemonSet implementation would be nice. And a way to handle mounting volumes…

10 hours ago 2 votes
Digital hygiene: Emails

Email is your most important online account, so keep it clean.

6 hours ago 1 votes
AmigaGuide Reference Library

As I slowly but surely work towards the next release of my setcmd project for the Amiga (see the 68k branch for the gory details and my total noob-like C flailing around), I’ve made heavy use of documentation in the AmigaGuide format. Despite it’s age, it’s a great Amiga-native format and there’s a wealth of great information out there for things like the C API, as well as language guides and tutorials for tools like the Installer utility - and the AmigaGuide markup syntax itself. The only snag is, I had to have access to an Amiga (real or emulated), or install one of the various viewer programs on my laptops. Because like many, I spend a lot of time in a web browser and occasionally want to check something on my mobile phone, this is less than convenient. Fortunately, there’s a great AmigaGuideJS online viewer which renders AmigaGuide format documents using Javascript. I’ve started building up a collection of useful developer guides and other files in my own reference library so that I can access this documentation whenever I’m not at my Amiga or am coding in my “modern” dev environment. It’s really just for my own personal use, but I’ll be adding to it whenever I come across a useful piece of documentation so I hope it’s of some use to others as well! And on a related note, I now have a “unified” code-base so that SetCmd now builds and runs on 68k-based OS 3.x systems as well as OS 4.x PPC systems like my X5000. I need to: Tidy up my code and fix all the “TODO” stuff Update the Installer to run on OS 3.x systems Update the documentation Build a new package and upload to Aminet/OS4Depot Hopefully I’ll get that done in the next month or so. With the pressures of work and family life (and my other hobbies), progress has been a lot slower these last few years but I’m still really enjoying working on Amiga code and it’s great to have a fun personal project that’s there for me whenever I want to hack away at something for the sheer hell of it. I’ve learned a lot along the way and the AmigaOS is still an absolute joy to develop for. I even brought my X5000 to the most recent Kickstart Amiga User Group BBQ/meetup and had a fun day working on the code with fellow Amigans and enjoying some classic gaming & demos - there was also a MorphOS machine there, which I think will be my next target as the codebase is slowly becoming more portable. Just got to find some room in the “retro cave” now… This stuff is addictive :)

yesterday 4 votes
That boolean should probably be something else

One of the first types we learn about is the boolean. It's pretty natural to use, because boolean logic underpins much of modern computing. And yet, it's one of the types we should probably be using a lot less of. In almost every single instance when you use a boolean, it should be something else. The trick is figuring out what "something else" is. Doing this is worth the effort. It tells you a lot about your system, and it will improve your design (even if you end up using a boolean). There are a few possible types that come up often, hiding as booleans. Let's take a look at each of these, as well as the case where using a boolean does make sense. This isn't exhaustive—[1]there are surely other types that can make sense, too. Datetimes A lot of boolean data is representing a temporal event having happened. For example, websites often have you confirm your email. This may be stored as a boolean column, is_confirmed, in the database. It makes a lot of sense. But, you're throwing away data: when the confirmation happened. You can instead store when the user confirmed their email in a nullable column. You can still get the same information by checking whether the column is null. But you also get richer data for other purposes. Maybe you find out down the road that there was a bug in your confirmation process. You can use these timestamps to check which users would be affected by that, based on when their confirmation was stored. This is the one I've seen discussed the most of all these. We run into it with almost every database we design, after all. You can detect it by asking if an action has to occur for the boolean to change values, and if values can only change one time. If you have both of these, then it really looks like it is a datetime being transformed into a boolean. Store the datetime! Enums Much of the remaining boolean data indicates either what type something is, or its status. Is a user an admin or not? Check the is_admin column! Did that job fail? Check the failed column! Is the user allowed to take this action? Return a boolean for that, yes or no! These usually make more sense as an enum. Consider the admin case: this is really a user role, and you should have an enum for it. If it's a boolean, you're going to eventually need more columns, and you'll keep adding on other statuses. Oh, we had users and admins, but now we also need guest users and we need super-admins. With an enum, you can add those easily. enum UserRole { User, Admin, Guest, SuperAdmin, } And then you can usually use your tooling to make sure that all the new cases are covered in your code. With a boolean, you have to add more booleans, and then you have to make sure you find all the places where the old booleans were used and make sure they handle these new cases, too. Enums help you avoid these bugs. Job status is one that's pretty clearly an enum as well. If you use booleans, you'll have is_failed, is_started, is_queued, and on and on. Or you could just have one single field, status, which is an enum with the various statuses. (Note, though, that you probably do want timestamp fields for each of these events—but you're still best having the status stored explicitly as well.) This begins to resemble a state machine once you store the status, and it means that you can make much cleaner code and analyze things along state transition lines. And it's not just for storing in a database, either. If you're checking a user's permissions, you often return a boolean for that. fn check_permissions(user: User) -> bool { false // no one is allowed to do anything i guess } In this case, true means the user can do it and false means they can't. Usually. I think. But you can really start to have doubts here, and with any boolean, because the application logic meaning of the value cannot be inferred from the type. Instead, this can be represented as an enum, even when there are just two choices. enum PermissionCheck { Allowed, NotPermitted(reason: String), } As a bonus, though, if you use an enum? You can end up with richer information, like returning a reason for a permission check failing. And you are safe for future expansions of the enum, just like with roles. You can detect when something should be an enum a proliferation of booleans which are mutually exclusive or depend on one another. You'll see multiple columns which are all changed at the same time. Or you'll see a boolean which is returned and used for a long time. It's important to use enums here to keep your program maintainable and understandable. Conditionals But when should we use a boolean? I've mainly run into one case where it makes sense: when you're (temporarily) storing the result of a conditional expression for evaluation. This is in some ways an optimization, either for the computer (reuse a variable[2]) or for the programmer (make it more comprehensible by giving a name to a big conditional) by storing an intermediate value. Here's a contrived example where using a boolean as an intermediate value. fn calculate_user_data(user: User, records: RecordStore) { // this would be some nice long conditional, // but I don't have one. So variables it is! let user_can_do_this: bool = (a && b) && (c || !d); if user_can_do_this && records.ready() { // do the thing } else if user_can_do_this && records.in_progress() { // do another thing } else { // and something else! } } But even here in this contrived example, some enums would make more sense. I'd keep the boolean, probably, simply to give a name to what we're calculating. But the rest of it should be a match on an enum! * * * Sure, not every boolean should go away. There's probably no single rule in software design that is always true. But, we should be paying a lot more attention to booleans. They're sneaky. They feel like they make sense for our data, but they make sense for our logic. The data is usually something different underneath. By storing a boolean as our data, we're coupling that data tightly to our application logic. Instead, we should remain critical and ask what data the boolean depends on, and should we maybe store that instead? It comes easier with practice. Really, all good design does. A little thinking up front saves you a lot of time in the long run. I know that using an em-dash is treated as a sign of using LLMs. LLMs are never used for my writing. I just really like em-dashes and have a dedicated key for them on one of my keyboard layers. ↩ This one is probably best left to the compiler. ↩

yesterday 4 votes