Full Width [alt+shift+f] Shortcuts [alt+shift+k]
Sign Up [alt+shift+s] Log In [alt+shift+l]
128

AI - SkyNet Is Not Coming to Kill You

from Renegade Otter [alt+shift+b] in programming

.highlight pre { background-color: #efecec; border-color: var(--theme-secondary-background-color); border-radius: 10px; } The firehose of data is turned on In the beginning, the Internet was a small, cozy place. Most people weren’t online, and most businesses weren’t really online. The old Internet was for nerds willing to suffer through the less-than-straightforward technical setup, before the soul-scraping screech of a 28K baud modem resulted in a successful connection to the interwebs. Finally - we could now slowly download, bar by bar, images of Cindy Margolis. It was an innocent time, with tacky page view counters, guestbooks, “dancing baby” animated gifs, scrolling marquees, and just terrible background color choices. Back then we discovered things on the Web through an array of search engines - AltaVista, Excite, Lycos, Yahoo… None of them particularly stood out on their own. Yahoo was a more thorough, actual directory of websites maintained by fellow life forms. The Internet was small enough that it could be categorized — just like a library. With time, the amount of data grew, and the usefulness of the existing search engines noticeably took a dive. Search engine companies were pushing the limits of vertical scaling, so when Google crashed the party with surprisingly good search results combined with a simple, uncluttered homepage, it was clear that the days of legacy search companies were numbered. Fast-forwarding through the rest of Internet history: the amount of data kept growing exponentially. “Social meeds” and mobile devices arrived at once, and now any nincompoop with a phone could whip out their gadget and add even more empty informational calories to the already massive pile of data dung. Big Data was invented, aggregating everything from our detailed marketing profiles to how we moved the mouse pointer around a page. “Data scientist” becomes one of the hottest career tracks. Then, nothing interesting really changed for...
22nd Apr 2024

Stay updated

Get a weekly newsletter with the top 5 articles worth reading every week.

More from Renegade Otter

A Lannister Always Pays His Technical Debts

A tale of two rewrites Jamie Zawinski is kind of a tech legend. He came up with the name “Mozilla”, invented that whole thing where you can send HTML in emails, and more. In his harrowing work diary of how Mosaic/Netscape came to be, Jamie described the burnout rodeo that was the Mosaic development (the top disclaimer has its own history — ignore it): I slept at work again last night; two and a half hours curled up in a quilt underneath my desk, from 11am to 1:30pm or so. That was when I woke up with a start, realizing that I was late for a meeting we were scheduled to have to argue about colormaps and dithering, and how we should deal with all the nefarious 8-bit color management issues. But it was no big deal, we just had the meeting later. It’s hard for someone to hold it against you when you miss a meeting because you’ve been at work so long that you’ve passed out from exhaustion. Netscape’s wild ride is well-depicted in the dramatized Discovery mini-series Valley of the Boom, and the company eventually collapsed with the death march rewrite of what seemed to be just seriously unmaintainable code. It was the subject of one of the more famous articles by ex-Microsoft engineer and then entrepreneur Joel Spolsky - Things You Should Never Do. While the infamous Netscape codebase is long gone, the people that it enriched are still shaping the world to this day. There have been big, successful rewrites. Twitter moved away from Ruby-on-Rails to JVM over a decade ago but the first, year-long full rewrite effort completely failed. Following architecture by fiat from the top, the engineering team said nothing, speaking out only days before the launch. The whole thing would crash out of the gate, they claimed, so Twitter had to go back to the drawing board and rewrite again. I'd love to hear from you. What didn’t work for Netscape worked for Twitter. Why? Netscape had major heat coming from ruthless Microsoft competition, very little time for major moves, and a team aleady exhausted from “office heroics”. Twitter, however, is a unique product that is incredibly hard to dislodge, even with the almost purposefully incompetent and reckless management. It’s hard to abandon your social media account after accumulating algorithmic reputation and followers for years, and yet one can switch browsers faster than they can switch socks. Companies often do not survive this kind of adventure without having an almost unfair moat. Those that do survive, they probably caught some battle scars. Friendly Fire: Notify in Slack directly Skip reviewers who are not available File pattern matching Individual code review reminders No access to your codebase needed The road to hell is paved with TODO comments All of this is to say that you should probably never let your system rot so badly until a code rewrite is even discussed. It never just happens. Your code doesn’t just become unmaintainable overnight. It gets there by the constant cutting of corners, hard-coding things, and crop-dusting your work with long-forgotten //FIXME comments. Fix who? We used to call it technical debt - a term that is now being frowned upon. The concept of “technical debt” got popular around the time when we were getting obsessed with “proh-cess” and Agile, as we got tired of death march projects, arbitrary deadlines, and general lack of structure and visibility into our work. Every software project felt like a tour — you came up for air and then went back into the 💩 for months. Agile meant that the stakeholders could be present in our planning meetings. We had to explain to them - somehow - that it took time to upgrade the web framework from v1 to v5 because no one has been using v1 for years, and in general, it slowed everyone down. Since we didn’t know how to explain this to a non-coder, someone came up with the condescending “technical debt” — “those spreadsheet monkeys wouldn’t understand what we do here!” While “technical debt” has most likely run its course as a manipulative verbal device, it is absolutely the right term to use amongst ourselves to reason about risks and to properly triage them. The three type of technical debt The word “debt” has negative connotations for sure, but just like with actual monetary debt, it’s never great but not always horrible. To mutilate the famous saying - you have to spend code to make code. I would categorize technical debt into three types — Aesthetic, Deferrable, and Toxic. A mark of a good engineer is knowing when to create technical debt, what kind of debt, and when to repay it. Aesthetic debt This is the kind of stuff that triggers your OCD but does not really affect your users or your velocity in any way. Maybe the imports are not sorted the way you want, and maybe there is a naming convention that is grinding your gears. It’s something that can be addressed with relatively low effort when you are good and ready, in many cases with proper automated code analysis and tools. Deferrable debt Deferrable debt is what should be refactored at some point, but it’s fairly contained and will not be a problem in the immediate future. The kind of debt that you need to minimize by methodically striking it off your list, and as long as it seeps through into your sprint work, you can probably avoid a scenario where it all gets out of control. Sometimes this sort of thing is really contained - a lone hacky file, written in the Mesozoic Era by a sleep-deprived Jamie Zawinski because someone was breathing down his neck. No one really understands what the code does, but it’s been humming along for the last 7 years, so why take your chances by waking the sleeping dragons? Slap the Safety Pig on it, claim a victory, and go shake down a vending machine. Toxic debt This is the kind of debt that needs to be addressed before it’s too late. How do you identify “toxic” debt? It’s that thing that you did half-way and now it’s become a workaround magnet. “We have to do it like this now until we fix it - someday”. The workarounds then become the foundation of new features, creating new and exciting debugging side quests. The future work required grows bigger with every new feature and a line of code. This is the toxic debt. Lack of tests is toxic debt Not having automated tests, or insufficient testing of critical paths, is tech debt in its own right. The more untested code you are adding, the more miserable your life is going to get over time. Tests are important to fight the debt itself. It’s much easier to take a sledgehammer to your codebase when a solid integration test suite’s got your back. We don’t like it, it’s upfront work that slows us down, but at some point after your Minimal Viable Prototype starts running away from you, you need to switch into Test Mode and tie it all down — before things get really nasty. Lack of documentation is toxic debt I am not talking about a War & Peace sized manual or detailed and severely out of date architecture diagrams in your Google Docs. Just a a set of critical READMEs and runbooks on how to start the system locally and perform basic tasks. What variables and secrets do I need? What else do I need installed? If there is a bug report, how do I configure my local environment to reproduce it, and so on. The time taken to reverse-engineer a system every time has an actual dollar value attached to it, plus the opportunity cost of not doing useful work. Put. It. In. A. Card. I have been guilty of this myself. I love TODOs. They are easy to add without breaking the flow, and they are configured in my IDE to be bright and loud. It’s a TODO — I will do it someday. During the Annual TODO Week, obviously. Let’s be frank — marking items as “TODO” is saying to yourself that you should really do this thing, but probably never will. This is relevant because TODO items can represent any level of technical debt described above, and so you should really make these actual stories on your Kanban/Agile boards. Mark technical debt as such You should be able to easily scan your “debt stories” and figure out which ones have payment due. This can be either a tag in your issue-tracking system or a column in your Kanban-style board like Trello. An approach like this will let you gauge better the ratio of new feature stories vs the growing technical debt. Your debt column will never be empty — that goal is as futile as Zero Inbox, but it should never grow out of control either. // TODO: conclusion

8th Feb 2024 • 80 votes
Your database skills are not ‘good to have’

A MySQL war story It’s 2006, and the New York Magazine digital team set out to create a new search experience for its Fashion Week portal. It was one of those projects where technical feasibility was not even discussed with the tech team - a common occurrence back then. Agile was still new, let alone in publishing. It was just a vision, a real friggin’ moonshot, and 10 to 12 weeks to develop the wireframed version of the product. There would be almost no time left for proper QA. Fashion Week does not start slowly but rather goes from zero to sixty in a blink. The vision? Thousands of near-real-time fashion show images, each one with its sub-items categorized: “2006”, “bag”, “red”, “ leather”, and so on. A user will land on the search page and have the ability to “drill down” and narrow the results based on those properties. To make things much harder, all of these properties would come with exact counts. The workflow was going to be intense. Photographers will courier their digital cartridges from downtown NYC to our offices on Madison Avenue, where the images will be processed, tagged by interns, and then indexed every hour by our Perl script, reading the tags from the embedded EXIF information. Failure to build the search product on our side would have collapsed the entire ecosystem already in place, primed and ready to rumble. “Oh! Just use the facets in Solr, dude”. Yeah, not so fast - dude. In 2006 that kind of technology didn’t even exist yet. I sat through multiple enterprise search engine demos with our CTO, and none of the products (which cost a LOT of money) could do a deep faceted search. We already had an Autonomy license and my first try proved that… it just couldn’t do it. It was supposed to be able to, but the counts were all wrong. Endeca (now owned by Oracle), came out of stealth when the design part of the project was already underway. Too new, too raw, too risky. The idea was just a little too ambitious for its time, especially for a tiny team in a non-tech company. So here we were, a team of three, myself and two consultants, writing Perl for the indexing script, query-parsing logic, and modeling the data - in MySQL 4. It was one of those projects where one single insurmountable technical risk would have sunk the whole thing. I will cut the story short and spare you the excitement. We did it, and then we went out to celebrate at a karaoke bar (where I got my very first work-stress-related severe hangover) 🤮 For someone who was in charge of the SQL model and queries, it was days and days of tuning those, timing every query and studying the EXPLAIN output to see what else I could do to squeeze another 50ms out of the database. There were no free nights or weekends. In the end, it was a combination of trial and error, digging deep into MySQL server settings, and crafting GROUP BY queries that would make you nauseous. The MySQL query analyzer was fidgety back then, and sometimes re-arranging the fields in the SELECT clause could change a query’s performance. Imagine if SELECT field1, field2 FROM my_table was faster than SELECT field2, field1 FROM my_table. Why would it do that? I have no idea to this day, and I don’t even want to know. Unfortunately, I lost examples of this work, but the Way Back Machine has proof of our final product. The point here is - if you really know your database, you can do pretty crazy things with it, and with the modern generation of storage technologies and beefier hardware, you don’t even need to push the limits - it should easily handle what I refer to as “common-scale”. Renegade Otter is the developer of Friendly Fire - Smarter pull request assignment for GitHub: Connect GitHub users to Slack and notify directly Skip reviewers who are not available File pattern matching Individual code review reminders No access to your codebase needed The fading art of SQL In the past few years I have been noticing an unsettling trend - software engineers are eager to use exotic “planet-scale” databases for pretty rudimentary problems, while at the same time not having a good grasp of the very powerful relational database engine they are likely already using, let alone understanding the technology’s more advanced and useful capabilities. The SQL layer is buried so deep beneath libraries and too clever by a half ORMs that it all just becomes high-level code. Why is it slow? No idea - let's add Cassandra to it! Modern hardware certainly allows us to go way up from the CPU into the higher abstraction layers, while it wasn’t that uncommon in the past to convert certain functions to assembly code in order to squeeze every bit of performance out of the processor. Now compute and storage is cheaper - it’s true - but abusing this abundance has trained us laziness and complacency. Suddenly, that Cloud bill is a wee too high, and heavens knows how much energy the world is burning by just running billions of these inefficient ORM queries every second against mammoth database instances. The morning of my first job interview in 2004, I was on a subway train memorizing the nine levels of database normalization. Or is it five levels? I don’t remember, and It doesn’t even matter - no one will ever ask you this now in a software engineer interview. Just skimming through the table of contents of your database of choice, say the now freshly in vogue Postgres, you will find an absolute treasure trove of features fit to handle everything but the most gruesome planet-scale computer science problems. Petabyte-sized Postgres boxes, replicated, are effortlessly running now as you are reading this. The trick is to not expect your database or your ORM to read your mind. Speaking of… ORMs are the frenemy I was a new hire at an e-commerce outfit once, and right off the bat I was thrown into fixing serious performance issues with the company’s product catalog pages. Just a straight-forward, paginated grid of product images. How hard could it be? Believe it or not - it be. The pages took over 10 seconds to load, sometimes longer, the database was struggling, and the solution was to “just cache it”. One last datapoint - this was not a high-traffic site. The pages were dead-slow even if there was no traffic at all. That’s a rotten sign that something is seriously off. After looking a bit closer, I realized that I hit the motherlode - all top three major database and coding mistakes in one. ❌ Mistake #1: There is no index The column that was hit in every single mission-critical query had no index. None. After adding the much-needed index in production, you could practically hear MySQL exhaling in relief. Still, the performance was not quite there yet, so I had to dig deeper, now in the code. ❌ Mistake #2: Assuming each ORM call is free Activating the query logs locally and reloading a product listing page, I see… 200, 300, 500 queries fired off just to load one single page. What the shit? Turns out, this was the result of a classic ORM abuse of going through every record in a loop, to the effect of: for product_id in product_ids: product = amazing_orm.products.get(id=product_id) products.append(product) The high number of queries was also due the fact that some of this logic was nested. The obvious solution is to keep the number of queries in each request to a minimum, leveraging SQL to join and combine the data into one single blob. This is what relational databases do - it’s in the name. Each separate query needs to travel to the database, get parsed, transformed, analyzed, planned, executed, and then travel back to the caller. It is one of the most expensive operations you can do, and ORMs will happily do the worst possible thing for you in terms of performance. One wonders what those algorithm and data structure interview questions are good for, considering you are more likely to run into a sluggish database call than a B-tree implementation (common structure used for database indexes). ❌ Mistake #3: Pulling in the world To make matters worse, the amount of data here was relatively small, but there were dozens and dozens of columns. What do ORMs usually do by default in order to make your life “easier”? They send the whole thing, all the columns, clogging your network pipes with the data that you don’t even need. It is a form of toxic technical debt, where the speed of development will eventually start eating into performance. I spent hours within the same project hacking the dark corners of the Dango admin, overriding default ORM queries to be less “eager”. This led to a much better office-facing experience. Performance IS a feature Serious, mission-critical systems have been running on classic and boring relational databases for decades, serving thousands of requests per second. These systems have become more advanced, more capable, and more relevant. They are wonders of computer science, one can claim. You would think that an ancient database like Postgres (in development since 1982) is in some kind of legacy maintenance mode at this point, but the opposite is true. In fact, the work has been only accelerating, with the scale and features becoming pretty impressive. What took multiple queries just a few years ago now takes a single one. Why is this significant? It has been known for a long time, as discovered by Amazon, that every additional 100ms of a user waiting for a page to load loses a business money. We also know now that from a user’s perspective, the maximum target response time for a web page is around 100 milliseconds: A delay of less than 100 milliseconds feels instant to a user, but a delay between 100 and 300 milliseconds is perceptible. A delay between 300 and 1,000 milliseconds makes the user feel like a machine is working, but if the delay is above 1,000 milliseconds, your user will likely start to mentally context-switch. The “just add more CPU and RAM if it’s slow” approach may have worked for a while, but many are finding out the hard way that this kind of laziness is not sustainable in a frugal business environment where costs matter. Database anti-patterns Knowing what not to do is as important as knowing what to do. Some of the below mistakes are all too common: ❌ Anti-pattern #1. Using exotic databases for the wrong reasons Technologies like DynamoDB are designed to handle scale at which Postgres and MySQL begin to fail. This is achieved by denormalizing, duplicating the data aggressively, where the database is not doing much real-time data manipulation or joining. Your data is now modeled after how it is queried, not after how it is related. Regular relational concepts disintegrate at this insane level of scale. Needless to say, if you are resorting to this kind of storage for “common-scale” problems, you are already solving problems you don’t have. ❌ Anti-pattern #2. Caching things unnecessarily Caching is a necessary evil - but it’s not always necessary. There is an entire class of bugs and on-call issues that stem from stale cached data. Read-only database replicas are a classic architecture pattern that is still very much not outdated, and it will buy you insane levels of performance before you have to worry about anything. It should not be a surprise that mature relational databases already have query caching in place - it just has to be tuned for your specific needs. Cache invalidation is hard. It adds more complexity and states of uncertainty to your system. It makes debugging more difficult. I received more emails from content teams than I care for throughout my career that wondered “why is the data not there, I updated it 30 minutes ago?!” Caching should not act as a bandaid for bad architecture and non-performant code. ❌ Anti-pattern #3. Storing everything and a kitchen sink As much punishment as an industry-standard database can take, it’s probably not a good idea to not care at all about what’s going into it, treating it like a data landfill of sorts. Management, querying, backups, migrations - all becomes painful once the DB grows substantially. Even if that is of no concern as you are using a managed cloud DB - the costs should be. An RDBMS is a sophisticated piece of technology, and storing data in it is expensive. Figure out common-scale first It is fairly easy to make a beefy Postgres or a MySQL database grind to a halt if you expect it to do magic without any extra work. “It’s not web-scale, boss. Our 2 million records seem to be too much of a lift. We need DynamoDB, Kafka, and event sourcing!” A relational database is not some antiquated technology that only us tech fossils choose to be experts in, a thing that can be waved off like an annoying insect. “Here we React and GraphQL all the things, old man”. In legal speak, a modern RDBMS is innocent until proven guilty, and the burden of proof should be extremely high - and almost entirely on you. Finally, if I have to figure out “why it’s slow”, my approximate runbook is: Compile a list of unique queries, from logging, slow query log, etc. Look at the most frequent queries first Use EXPLAIN to check slow query plans for index usage Select only the data that needs to travel across the wire If an ORM is doing something silly without a workaround, pop the hood and get dirty with the raw SQL plumbing Most importantly, study your database (and SQL). Learn it, love it, use it, abuse it. Spending a couple of days just leafing through that Postgres manual to see what it can do will probably make you a better engineer than spending more time on the next flavor-of-the-month React hotness. Again. Related posts I am not your Cloud person Renegade Otter is the developer of Friendly Fire - Smarter pull request assignment for GitHub: Connect GitHub users to Slack and notify directly Skip reviewers who are not available File pattern matching Individual code review reminders No access to your codebase needed

12th Nov 2023 • 77 votes
Death by a thousand microservices

The Church of Complexity There is a pretty well-known sketch in which an engineer is explaining to the project manager how an overly complicated maze of microservices works in order to get a user’s birthday - and fails to do so anyway. The scene accurately describes the absurdity of the state of the current tech culture. We laugh, and yet bringing this up in a serious conversation is tantamount to professional heresy, rendering you borderline un-hirable. How did we get here? How did our aim become not addressing the task at hand but instead setting a pile of cash on fire by solving problems we don’t have? Trigger warning: Some people understandably got salty when I name-checked JavaScript and NodeJS as a source of the problem, but my point really was more about the dangers of hermetically sealed software ecosystems that seem hell-bent on re-learning the lessons that we just had finished learning. We ran into the complexity wall before and reset - otherwise we'd still be using CORBA and SOAP. These air-tight developer bubbles are a wrecking ball on the entire industry, and it takes about a full decade to swing. The perfect storm There are a few events in recent history that may have contributed to the current state of things. First, a whole army of developers writing JavaScript for the browser started self-identifying as “full-stack”, diving into server development and asynchronous code. JavaScript is JavaScript, right? What difference does it make what you create using it - user interfaces, servers, games, or embedded systems. Right? Node was still kind of a learning project of one person, and the early JavaScript was a deeply problematic choice for server development. Pointing this out to still green server-side developers usually resulted in a lot of huffing and puffing. This is all they knew, after all. The world outside of Node effectively did not exist, the Node way was the only way, and so this was the genesis of the stubborn, dogmatic thinking that we are dealing with to this day. And then, a steady stream of FAANG veterans started merging into the river of startups, mentoring the newly-minted and highly impressionable young JavaScript server-side engineers. The apostles of the Church of Complexity would assertively claim that “how they did things over at Google” was unquestionable and correct - even if it made no sense with the given context and size. What do you mean you don’t have a separate User Preferences Service? That just will not scale, bro! But, it’s easy to blame the veterans and the newcomers for all of this. What else was happening? Oh yeah - easy money. What do you do when you are flush with venture capital? You don’t go for revenue, surely! On more than one occasion I received an email from management, asking everyone to be in the office, tidy up their desks and look busy, as a clouder of Patagonia vests was about to be paraded through the space. Investors needed to see explosive growth, but not in profitability, no. They just needed to see how quickly the company could hire ultra-expensive software engineers to do … something. And now that you have these developers, what do you do with them? Well, they could build a simpler system that is easier to grow and maintain, or they could conjure up a monstrous constellation of “microservices” that no one really understands. Microservices - the new way of writing scalable software! Are we just going to pretend that the concept of “distributed systems” never existed? (Let’s skip the whole parsing of nuances about microservices not being real distributed systems). Back in the days when the tech industry was not such a bloated farce, distributed systems were respected, feared, and generally avoided - reserved only as the weapon of last resort for particularly gnarly problems. Everything with a distributed system becomes more challenging and time-consuming - development, debugging, deployment, testing, resilience. But I don’t know - maybe it’s all super easy now because toooollling. There is no standard tooling for microservices-based development - there is no common framework. Working on distributed systems has gotten only marginally easier in 2020s. The Dockers and the Kuberneteses of the world did not magically take away the inherent complexity of a distributed setup. I love referring to this summary of 5 years of startup audits, as it is packed with common-sense conclusions: … the startups we audited that are now doing the best usually had an almost brazenly ‘Keep It Simple’ approach to engineering. Cleverness for cleverness sake was abhorred. On the flip side, the companies where we were like ”woah, these folks are smart as hell” for the most part kind of faded. Generally, the major foot-gun that got a lot of places in trouble was the premature move to microservices, architectures that relied on distributed computing, and messaging-heavy designs. Literally - “complexity kills”. The audit revealed an interesting pattern, where many startups experienced a sort of collective imposter syndrome while building straight-forward, simple, performant systems. There is a dogma attached to not starting out with microservices on day one - no matter the problem. “Everyone is doing microservices, yet we have a single Django monolith maintained by just a few engineers, and a MySQL instance - what are we doing wrong?”. The answer is almost always “nothing”. Likewise, it’s very often that seasoned engineers experience hesitation and inadequacy in today’s tech world, and the good news is that, no - it’s probably not you. It’s common for teams to pretend like they are doing “web scale”, hiding behind libraries, ORMs, and cache - confident in their expertise (they crushed that Leetcode!), yet they may not even be aware of database indexing basics. You are operating in a sea of unjustified overconfidence, waste, and Dunning-Kruger, so who is really the imposter here? Renegade Otter is the developer of Friendly Fire - Smarter pull request assignment for GitHub: Slack notifications Out-of-office support File matching Try it now. There is nothing wrong with a monolith The idea that you cannot grow without a system that looks like the infamous slide of Afghanistan war strategy is a myth. Dropbox, Twitter, Netflix, Facebook, GitHub, Instagram, Shopify, StackOverflow - these companies and others started out as monolithic code bases. Many have a monolith at their core to this day. StackOverflow makes it a point of pride how little hardware they need to run the massive site. Shopify is still a Rails monolith, leveraging the tried and true Resque to process billions of tasks. WhatsApp went supernova with their Erlang monolith and a relatively small team. How? WhatsApp consciously keeps the engineering staff small to only about 50 engineers. Individual engineering teams are also small, consisting of 1 - 3 engineers and teams are each given a great deal of autonomy. In terms of servers, WhatsApp prefers to use a smaller number of servers and vertically scale each server to the highest extent possible. Instagram was acquired for billions - with a crew of 12. And do you imagine Threads as an effort involving a whole Meta campus? Nope. They followed the Instagram model, and this is the entire Threads team: Perhaps claiming that your particular problem domain requires a massively complicated distributed system and an open office stuffed to the gills with turbo-geniuses is just crossing over into arrogance rather than brilliance? Don’t solve problems you don’t have It’s a simple question - what problem are you solving? Is it scale? How do you know how to break it all up for scale and performance? Do you have enough data to show what needs to be a separate service and why? Distributed systems are built for size and resilience. Can your system scale and be resilient at the same time? What happens if one of the services goes down or comes to a crawl? Just scale it up? What about the other services that are going to get hit with traffic? Did you war-game the endless permutations of things that can and will go wrong? Is there backpressure? Circuit breakers? Queues? Jitter? Sensible timeouts on every endpoint? Are there fool-proof guards to make sure a simple change does not bring everything down? The knobs you need to be aware of and tune are endless, and they are all specific to your system’s particular signature of usage and load. The truth is that most companies will never reach the massive size that will actually require building a true distributed system. Your cosplaying Amazon and Google - without their scale, expertise, and endless resources - is very likely just an egregious waste of money and time. Religiously following all the steps from an article called “Ten morning habits of very successful people” is not going to make you a billionaire. The only thing harder than a distributed system is a BAD distributed system. “But each team… but separate… but API” Trying to shove a distributed topology into your company’s structure is a noble effort, but it almost always backfires. It’s a common approach to break up a problem into smaller pieces and then solve those one by one. So, the thinking goes, if you break up one service into multiple ones, everything becomes easier. The theory is sweet and elegant - each microservice is being maintained rigorously by a dedicated team, walled off behind a beautiful, backward-compatible, versioned API. In fact, this is so solid that you rarely even have to communicate with that team - as if the microservice was maintained by a 3rd party vendor. It’s simple! If that doesn’t sound familiar, that’s because this rarely happens. In reality, our Slack channels are flooded with messages from teams communicating about releases, bugs, configuration updates, breaking changes, and PSAs. Everyone needs to be on top of everything, all the time. And if that wasn’t great, it’s normal for one already-slammed team to half-ass multiple microservices instead of doing a great job on a single one, often changing ownership as people come and go. In order to win the race, we don’t build one good race car - we build a fleet of shitty golf carts. What you lose There are multiple pitfalls to building with microservices, and often that minefield is either not fully appreciated or simply ignored. Teams spend months writing highly customized tooling and learning lessons not related at all to the core product. Here are just some often overlooked aspects… Say goodbye to DRY After decades of teaching developers to write Don’t Repeat Yourself code, it seems we just stopped talking about it altogether. Microservices by default are not DRY, with every service stuffed with redundant boilerplate. Very often the overhead of such “plumbing” is so heavy, and the size of the microservices is so small, that the average instance of a service has more “service” than “product”. So what about the common code that can be factored out? Have a common library? How does the common library get updated? Keep different versions everywhere? Force updates regularly, creating dozens of pull requests across all repositories? Keep it all in a monorepo? That comes with its own set of problems. Allow for some code duplication? Forget it, each team gets to reinvent the wheel every time. Each company going this route faces these choices, and there are no good “ergonomic” options - you have to choose your version of the pain. Developer ergonomics will crater “Developer ergonomics” is the friction, the amount of effort a developer must go through in order to get something done, be it working on a new feature or resolving a bug. With microservices, an engineer has to have a mental map of the entire system in order to know what services to bring up for any particular task, what teams to talk to, whom to talk to, and what about. The “you have to know everything before doing anything” principle. How do you keep on top of it? Spotify, a multi-billion dollar company, spent probably not negligible internal resources to build Backstage, software for cataloging its endless systems and services. This should at least give you a clue that this game is not for everyone, and the price of the ride is high. So what about the tooooling? The Not Spotifies of the world are left with MacGyvering their own solutions, robustness and portability of which you can probably guess. And how many teams actually streamline the process of starting a YASS - “yet another stupid service”? This includes: Developer privileges in GitHub/GitLab Default environment variables and configuration CI/CD Code quality checkers Code review settings Branch rules and protections Monitoring and observability Test harness Infrastructure-as-code And of course, multiply this list by the number of programming languages used throughout the company. Maybe you have a usable template or a runbook? Maybe a frictionless, one-click system to launch a new service from scratch? It takes months to iron out all the kinks with this kind of automation. So, you can either work on your product, or you can be working on toooooling. Integration tests - LOL As if the everyday microservices grind was not enough, you also forfeit the peace of mind offered by solid integration tests. Your single-service and unit tests are passing, but are your critical paths still intact after each commit? Who is in charge of the overall integration test suite, in Postman or wherever else? Is there one? Integration testing a distributed setup is a nearly-impossible problem, so we pretty much gave up on that and replaced it with another one - Observability. Just like “microservices” are the new “distributed systems”, “observability” is the new “debugging in production”. Surely, you are not writing real software if you are not doing…. observability! Observability has become its own sector, and you will pay in both pretty penny and in developer time for it. It doesn’t come as plug-and-pay either - you need to understand and implement canary releases, feature flags, etc. Who is doing that? One already overwhelmed engineer? As you can see, breaking up your problem does not make solving it easier - all you get is another set of even harder problems. What about just “services”? Why do your services need to be “micro”? What’s wrong with just services? Some startups have gone as far as create a service for each function, and yes, “isn’t that just like Lambda” is a valid question. This gives you an idea of how far gone this unchecked cargo cult is. So what do we do? Starting with a monolith is one obvious choice. A pattern that could also work in many instances is “trunk & branches”, where the main “meat and potatoes” monolith is helped by “branch” services. A branch service can be one that takes care of a clearly-identifiable and separately-scalable load. A CPU-hungry Image-Resizing Service makes way more sense than a User Registration Service. Or do you get so many registrations per second that it requires independent horizontal scaling? Side note: In version control, back in the days of CVS and Subversion, we rarely used "master" branches. We had "trunk and branches" because, you know - *trees*. "Master" branches appeared somewhere along the way, and when GitHub decided to do away with the rather unfortunate naming convention, the average engineer was too young to remember about "trunk" - and so the generic "main" default came to be. The pendulum is swinging back The hype, however, seems to be dying down. The VC cash faucet is tightening, and so the businesses have been market-corrected into exercising common-sense decisions, recognizing that perhaps splurging on web-scale architectures when they don’t have web-scale problems is not sustainable. Ultimately, when faced with the need to travel from New York to Philadelphia, you have two options. You can either attempt to construct a highly intricate spaceship for an orbital descent to your destination, or you can simply purchase an Amtrak train ticket for a 90-minute ride. That is the problem at hand. Additional reading & listening How to recover from microservices You want modules, not microservices XML is the future Gasp! You might not need microservices Podcast: How we keep Stack Overflow’s codebase clean and modern Goodbye Microservices: From 100s of problem children to 1 superstar It’s the future Renegade Otter is the developer of Friendly Fire - Smarter pull request assignment for GitHub: Slack notifications Out-of-office support File matching Try it now.

10th Sep 2023 • 46 votes
Getting our focus back

“We was robbed…” Attention is arguably the most precious resource of the 21st century. Technology companies have expended incredible efforts to improve the ways in which they capture our attention and convert it into revenue. That fight for the nine unmonetized glances. An “addictive” app or device is not merely a figure of speech. Your compulsion to check your phone without any specific reason is not significantly different from any other form of dependence. Substance addiction is psychoactive, whereas this is behavioral, but the effect is strikingly similar. “Addictive” technology has transformed us all into seekers of micro-dopamine hits, with attention spans akin to that of a squirrel. Your brain - now rewired Johann Hari starts Stolen Focus with a story of how he realized - after going device and Internet-free as an experiment - that he is no longer able to sit down and finish a chunk of a book without his mind wondering off. Hari has been called out for cherry-picking scientific studies, and yet the book could stand on anecdotal evidence alone, as the concepts he discusses are profoundly relatable. Even as a fidgety teenager, I remember being laser-focused for hours on chapters of the next science fiction book, to the point where my parents were getting worried about me not playing enough outside. Now? I am not able to get through half a page without thinking of checking something. Similarly, at the (home) office, getting deep work done has gotten extremely challenging. As proof that Hari’s (and mine) observations are not that unique, Nicholas Carr noticed the same thing: Over the past few years I’ve had an uncomfortable sense that someone, or something, has been tinkering with my brain, remapping the neural circuitry, reprogramming the memory. My mind isn’t going—so far as I can tell—but it’s changing. I’m not thinking the way I used to think. I’m not the only one. When I mention my troubles with reading to friends and acquaintances—literary types, most of them—many say they’re having similar experiences. The more they use the Web, the more they have to fight to stay focused on long pieces of writing. Some of the bloggers I follow have also begun mentioning the phenomenon. The kicker? This was written in 2008. Why is that notable? Because just before - 2007 - was a pivotal year in the evolution of the internet: Twitter and Facebook go into hyper-growth Steve Jobs launches the iPhone Hadoop brings the rise of Big Data Nicholas Carr and his “literary friends” noticed their focus slipping away even back when they used the classic web, before we had the Internet on our phones all day long, buzzing with notifications, and before Slack or Microsoft Teams gnawed at our productivity. The boiling frog effect has made us unaware of just how bad it has gotten. Social media, device notifications, and work group chats have changed the way we process information. Think about your regular session on an app such as X-itter. You doom-scroll through items, and they are very different items, forcing your brain to switch context rapidly and violently: A troll says something repulsive - "triggered" A dog does something funny - "lol" A study shows that X amount of Y every day may be good for you - "interesting!" A large company gets fined for violating the law - "yay justice!" The market is down - "ugh" All of THAT will happen within 1 or 2 minutes, your brain taken for a wild ride of news and whimsy. Much of the content will have articles attached, but you don’t click and read those - you are “power browsing”. This ritual is taxing and exhausting, sapping your power to focus. This is why taking a break from work on social media does not count as rest. Do this long enough, and your brain will learn to pay attention to things only in small doses. It will seek small things to process - group chat pings, text notifications, headline alerts. Like hamsters, we are now trained to be in this relentless loop of FOMO. The situation with work chat has reached obnoxious levels as well, where it’s impossible to get 5 minutes of peace without being distracted. There is a channel for everything. We are yet to come to terms with the tab incurred from the distraction economy, but the data is beginning to come in - and it’s not good. Rani Molla writes in “The productivity pit: how Slack is ruining work”: Much like the ubiquitous open-floor plan, this type of software is meant to get different parts of a company working together, to break down hierarchies, to spark chance interactions and innovations. In practice, it can be hell. Renegade Otter is the developer of Friendly Fire - Smarter pull request assignment for GitHub: Slack notifications Out-of-office support File matching Try it now. There is a pill for that Is it any surprise that (in the United States), the consumption of focus-enhancing stimulants has skyrocketed? Adults who previously had no issues with focus now find themselves struggling, and students have gone as far as getting professors dismissed due to their inability to concentrate on courses that have been successfully taught for decades. While ADHD is a genuine condition, it’s worth asking - did some of us develop ADHD suddenly at the age of 37? Is it necessary to alter our brain chemistry, or has the excessive fast-scrolling on Instagram turned more than five sentences at once so hard to process? Maybe, before we resort to new “miracle” pills for our focus problems or Ozempic for our impulsive snacking, should we try and mount a counterattack against this barrage of distractions in “organic” ways first? What can we do? For some of us, the reflex to switch context for no reason is so strong that self-limiting does not work. Just like food diets do not work, a screen diet doesn’t either. It’s not a quick fix - it’s a lifestyle change (or a lifestyle rollback?). So what can we do? There are basic things to try, but these are pretty well-known: Uninstall addictive apps that sap your time Ruthlessly disable notifications for apps that keep bothering you Mute noisy work chat channels Do not check your phone before going to sleep, in the middle of the night, or right after you wake up This baseline TODO, however, still requires discipline and many for us will quickly go back to our old ways of being easily distracted. From my personal experimentation, perhaps you will find some of these mental (and physical) tricks useful… Oxygen One mental device I came up with is to imagine my attention (which is a finite resource) as oxygen in a spaceship. Once I “open” the airlock, I lose that oxygen/attention at a very rapid pace. So every time my mind wonders off in search of a distraction, I ask myself: “do I want to open the airlock and lose my attention oxygen now?” Separate from your device Why not just leave your phone behind entirely for a non-critical amount of time when going outside? Before the smartphones and before people were used to texting, I would disappear in the NYC subway and no one would know where I was for hours, unless I dropped a quarter and called on a NYNEX pay phone. Ask yourself - what could be so end-of-the-world catastrophic that you must respond now and not in, say, 45 minutes? Try and leave your phone behind when you get out of the office for lunch, for example - alone or with your colleagues. After the initial freak-out, a sense of zen is going to settle in - and it will feel glorious. If not alone, your colleagues will also appreciate you not planting your face in that phone while they are trying to have a conversation over lunch - an added benefit. Say NO to “mission control” work setup While that fourth monitor is going to look ultra-cool, perhaps you should go back - to one. It’s not clear how the constant movement of logs, messages, and popups that you notice with the corner of your eye is supposed to help you do deep work. There is a reason IDEs now offer “focus mode”, where all you see is your editor and nothing else. This is a silly trend that should seriously be questioned. There are some kinds of work where multiple monitor setup is helpful. Specifically, where context switching and distraction is the nature of the job, but for focus work? Perhaps you need that second screen for fast preview of content? Operating systems offer quick virtual desktop switching, which is just as good. The screens are getting so huge that even that is not necessary - just go splitsies on the screen real estate. Producer/consumer mode No matter how many Marvel moves you watch, the truth is that none of us are super-human - our brain is not capable of multitasking. It can only do somewhat fast yet taxing context switching. It’s fine if your day consists of small tasks. In knowledge work, however, it makes the process of getting into the flow very fragile. Do you notice how, after you finally get into the flow, even you are impressed with how much you can crank out? The next time you sit down at your work station, think about what mode you want to be in: are you a producer, or an attention-wasting consumer? You cannot be both. Background café noise I am personally not a fan of working in coffee shops (I cannot focus with all the comings and goings), but some people do find those helpful for concentration. Curiously, a simulation of café ambience does help me get in the zone when I am working from home, at low volume, using a playlist like this (Spotify). Pink, white, and brown noise If your woe is a very noisy open-office work environment where the constant jibber-jabber is making you, uh, stabby (noise-cancelling headphones are not that effective against human voice frequencies), the combo of pink white and brown noises will put you at 30 thousand feet and inside a humming passenger plane. The hum drowns out human voices, helping you get in the zone. Your device has a Focus Mode Both Android and iOS operating systems now have Focus Modes. This mutes all of your notifications but allows for exceptions, in order to let emergencies get through. Dedicate a “distraction device” You may also want to dedicate an old laptop or a tablet as your “distraction device”. It can certainly not be your phone. Introduce a certain barrier of entry - don’t let distraction be within a simple reach of a hand. The goal is to force yourself to make a conscious decision - “I am going to walk away from my work station now and waste time”. The password “confidant” If you are being a totally difficult customer who has zero impulse control, use a password confidant - a person that you can trust with your account. Here is how this works: Transfer your [social media] account to the other person’s email and phone number Use the problem app until you know you should stop Change the password but do not memorize or save it! Log out When you seriously need the fix, reset the password and bother the confidant for the confirmation code The benefit of this system is that you have to actually ping someone else about the password reset, and you cannot abuse that trust, as your confidant can only be disturbed only so many times before they start getting tired of your antics. They own your account, so be nice. What to read I mentioned the “FOMO loop” before. It’s not really that - it’s the habit loop. I have read many books on compulsive behavior, but The Power of Habit is really the one that explains to the laymen why we are locked in the doom loop of bad behavior - be it drinking, checking the fridge for a snack out of boredom, or biting your nails. “The Power of Habit” doesn’t really go into compulsive device use, and it doesn’t have to - the mechanics of these behaviors are identical. It’s key to understanding and undoing the re-wiring done to us. What to watch Tacky dramatizations aside, The Social Dilemma is a pretty good crash course in how large tech companies malt your attention (and you) into a product. None of this is new information, but it is presented in a digestible manner. In short: to an algorithm - we are just a bunch of marketing dimensions, constantly refined, until the algorithm knows about what we are going to want and need before we do. It’s an ongoing fight Regaining and keeping your focus is hard. You will almost inevitably relapse into time-wasting routines, but hopefully this can be a starting point and a toolkit. It’s a struggle, and while we may not be winning the war yet, it’s time to start winning some battles. Renegade Otter is the developer of Friendly Fire - Smarter pull request assignment for GitHub: Slack notifications Out-of-office support File matching Try it now.

24th Aug 2023 • 38 votes

More in programming

Writing a Demoscene Intro for the ZX Spectrum

Download Now View on pouet.net Intro In the late 80s, I spent an embarrassing number of hours hunched over my ZX Spectrum, typing in Sinclair BASIC programs and rudimentary assembler routines copied from magazines, trying to re-create the Amiga crack intros I’d seen round at a friend’s house. Obviously, they never came even slightly close - but now, many decades later, I’ve finally managed it! I’m now the proud author of a hand-rolled retro demoscene intro that runs on a stock 128Kb Spectrum. It features all those late 80s tropes: A smooth scrolltext (with ‘leet custom fonts!), a three-channel AY soundtrack, bouncing VU meters and the essential parallax starfield. It’s currently 10287 bytes of code, or 19552 bytes as a complete tape image - which, given I’ve been working on this since December 13th last year, means my average rate of code-production is a paltry ~ 455 bytes per week! This blog post is a technical overview of how I got there, some cool tricks I picked up, and everything I had to learn (and re-learn) about the Speccy and Z80 assembly along the way. It simultaneously rates as one of my favourite side-projects ever, and also the one that most often nearly triggered a foot put through my monitor in frustration. I’ll reflect on all that at the end, but for now if you just want to see the finished product, the download button is at the top of the article and the included .tap cassette tape image should load into any emulator. There’s also a YouTube capture, links to the production on pouet.net, and you can run it from within a web browser by visiting my TNFS site portal - check option 4 for recent uploads. Be Kind, Rewind But let’s skip back briefly to the late 80s. Coming from the world of an 8-bit, cassette tape loading computer, seeing an Amiga for the first time was like encountering something from outer space. Like everyone who saw one back in the day, I was blown away by the amazing graphics and sound on display, not to mention the library of ground-breaking games. But I found myself even more captivated by the mysterious (and totally badass) “intros” displayed at the beginning of pirated games. Created by the software piracy groups that had cracked the copy protection and distributed the floppy-disk based games, they were mini works of art in their own right - music, graphics and usually some cool-looking effect to show off the group’s prowess. There’s a great collection of these intros (or “cracktros”) archived on YouTube. Watch a few of those videos and you’ll quickly notice the “design” tropes that almost every cracktro shares: a cool, edgy logo at the top of the screen (bonus points if it’s dripping blood or in any way “metal”), some boastful text displayed on top of an effect of some sort - usually a scrolling parallax starfield, or rotating 3d object - and a further scroll-text full of shout outs, greetings to fellow pseudonymous crackers and adverts for illegal BBS systems galore. Needless to say, to a kid in the 80s/early 90s this was possibly the coolest thing ever. I thought this one was the absolute bomb when I first saw it. Sine-scroller, pirate BBS adverts, cool logo with a sword - and of course, a starfield. There’s something for everyone here. The music from this is burned into my brain. I think pretty much everyone of my classmates knew this intro given the popularity of Lemmings in the early 90s. Skid Row as a group are still active to this day and this is probably the quintessential Amiga cracktro of the period. Endless Piracy. Edgy. Another cool logo, sine-wave scroller, starfield and boasting text. The classics never go out of style. Another great combination of all the essential ingredients. Cool logo, even cooler text font, scrolling starfield and a scroll-text running along the bottom. Vintage stuff. Logo. Text. Scroller. But no starfield - otherwise a solid effort from Anthrox here :) The demoscene grew out of these intros. The cracking eventually faded into the background, and coders, “graphicians” and musicians all contributed to feature-length productions that showcased their abilities, competed with other groups and pushed the machine to its limits. In amazing feats of ingenuity and pure “code as an art form”, the scene lives on today, with groups producing breathtaking productions like this and this - bear in mind, both of those are running on a 7Mhz home computer equipped with a floppy disk, 1Mb of memory and a sound chip from 1985. But much as I admire the sheer skill and dedication that takes, it’s the early “single screen” scrolly-text-and-starfield cracktros I’ve always found captivating. They took on a sort of mystical quality for me, hinting at a scene and a culture I could only ever experience from the outside - which is exactly why I spent all those hours in Sinclair BASIC, trying in vain to re-create them, and dreaming of the day I’d eventually have an Amiga of my own. The Jump Back I eventually did trade up to a range of Amiga systems, joined a few “scene” groups myself and threw myself into a variety of activities until exams and the end of my teenage years beckoned. I dabbled with Z80 code back then, and at one point knew enough 68k assembly to bluff my way through - partly from the Amiga’s heyday, partly from a few “embedded systems” modules at University where I had to program a set of traffic lights or lift-controllers. Given there’s not much call for that sort of thing at my $DAYJOB, though, it had all pretty much faded from my memory. I have kept working on a bunch of projects in my spare time, and one of my favourites is my TNFS site - sort of an old-school BBS system or interactive “website” for the Spectrum, which grew out of a fun experiment to see how far I could push Sinclair BASIC. It relies on a modern network card fitted to the Spectrum, which is how I came to be chatting to “Desertkun”, another TNFS-protocol enthusiast. He’s now producing a great new WiFi network card for the Speccy, and was kind enough to send me a few of the prototypes to test out for him. I was massively inspired by his hardware and low-level coding chops, and during one of our video calls where we’d been comparing our development setups, he asked why I didn’t write any Z80 assembly myself. A throw-away comment, but the challenge was set! Now, I couldn’t face re-writing the whole site in asm, but I figured I could produce a self-contained intro to promote it in the style of the early “BBStros”, and finally be one of the cool kids with a scene production of my own :) I mean, how hard could it be ? (ooooh - foreshadowing!) Dev Environment My development environment through this was pretty straightforward, and is the same stack I use for developing my TNFS site. I just used a Makefile-driven build system which ran my code through the Pasmo assembler, my fork of zmakebas to build BASIC files, and the FUSE emulator for testing. For final testing on the real hardware, I uploaded my .tap files to my local TNFS server where I could run them on my +3. A few additional tools I have to call out because they ended up making a big difference - one is the amazing web-based “asm-editor” I discovered while looking up Z80 instructions. Not only does this site have a great reference of all the Z80 instructions, but it also has an editor where you can run example code and see how the memory and/or various registers change state. Take for example, the djnz instruction - you can build it and step through the example loop construct one instruction at a time. Or try my example of an add instruction in the screenshot to see how the carry flag works when a bit gets shifted out of a register. Although it uses a slightly different syntax to Pasmo (so I couldn’t drop my code directly in it), it massively helped when manually stepping through things. And lastly, probably the most significant debugging tools I used on this project - a notepad and pen, and my two dogs (say “Hi!” to Biscuit & Pebble…) I’ll write some more about this in the wrap-up, but this really was a highly unusual project for me, coming from decades of much higher-level languages and constructs. Most of my z80 code I only ever really understood after drawing out the bytes and bitmasks, keeping a table of register flags and drawing chunks of screen memory out by hand. And I found my evening walks with the two dogs provided ideal rubber duck exercises, although I’m not sure what they both made of my protracted mutterings about the hl register pair. Either way, debugging-by-dog is highly recommended! The Starting Point So, let’s get started. I followed a great “Hello, world!” example from futurice.com to validate my build environment. Beyond that, the first thing I wanted to do was load in my logo and background image I’d use as a “stage” for the remainder of the planned intro. I wasn’t really sure what I was going to do at this point, just that I wanted my logo at the top, and a scrolltext along the bottom row of the screen. As my TNFS site header already looked kinda demoscene-y, I made a quick edit putting in place two blue guide-bars inside which I’d put further content. Memory map courtesy of breakintoprogram.co.uk A Spectrum SCREEN$ file (usually displayed as loading screens) is just a dump of screen memory, starting from byte 16384, and running for 6912 bytes. The weird layout of this is very relevant later, but in order to display it all you need to do is load your file into that memory location. After a few hour’s work of reviewing Speccy internals and following various guides, I had a very simple starting point for my demo that served as the basis for everything else that came after. org $8000 screen EQU $4000 ; address of the first byte of screen memory, 16384 in dec scrlen EQU 6912 ; total length of the screen memory inc. attributes ula EQU $FE ; I/O port for the ULA to control the border black EQU 0 ; Spectrum colour number, the exact same ones BASIC uses start: ld a,black ; Set the border colour to black using the ULA out (ula),a ; Copy my logo header picture to screen memory ld hl,header ; HL = location of my screen file in this code ld de,screen ; DE = start of screen memory ld bc,scrlen ; BC = 6912 bytes to copy ldir ; block copy, see https://jnz.dk/z80/ldir.html ; busy loop - rest of intro goes here :P hold: halt ; sleep until the next interrupt (1/50 second) jr hold ; then jump back and sleep again, for ever header: incbin "assets/header.scr" ; my logo header/background end start Not much to look at now, but this simple starting point introduced me to important concepts like interfacing with the Spectrum’s hardware (ULA chip to start with, and AY sound chip later) via the out instruction, how to pull in binary files using Pasmo’s INCBIN directive, using the Z80’s block copy ldir instruction, the use of register pairs for 16-bit addressing, and simple branching to labels. Hello World The next step was to print something on the screen and explore some of the Speccy’s built-in ROM routines, which are somewhat analagous to syscalls on modern operating systems. There are plenty of disassemblies and guides to the Spectrum ROM available on the internet, one of the best commented and linked one is at https://skoolkid.github.io/rom/. Using this as a reference alongside the various “Hello, world!” tutorials showed me how this sort of thing is built up: at EQU 22 ; "at" control character used to position the cursor row EQU 11 ; coordinates where to print the message col EQU 9 prstring EQU $203C ; where the pr_string routine lives in ROM ld de,msg ; DE = address of the first character ld bc,msglen ; BC = how many characters to print call prstring ; print the lot ; busy loop is the same... msg: db at,row,col ; equivalent of "PRINT AT 11,9" in BASIC db "Hello, world!" msglen EQU $-msg ; 16 bytes in all, pasmo shortcut, $=program counter I ended up mostly not using the ROM routines in my code as they dragged in all sorts of things I didn’t need, but it was massively useful to understand how they worked as they introduced me to yet more commonly-used constructs. For example, the PR_STRING routine defined here shows how you can build a loop over each character. If you’re just starting out, I highly recommend going over the 8 lines in that routine with a notepad and pen, working out the contents of the registers and following the loop constructs. I also made a version of it in asm-editor you can try. If you click that link, you’ll see a version of the code loaded into the web editor. Click “Build”, and then step through the code using the “Step” button - although no text will be actually displayed (I skipped over the call to the Spectrum ROM “restart” vector that does this), you’ll see the various states of the registers and for example the BC register counting down as each byte of the string is processed. Adding Some Style Problem is, the Speccy’s built-in font is not exactly dripping with demoscene credibility. I wanted to use some stylish fonts and this turned out to be super easy - a quick search turned up Jim Blimey’s awesome font collection, which included a ready-to-use set of assembler files and some instructions which soon led me to these additions in my code: ; CHARS is the address the ROM's printing routine reads to find the font. ; Picking our own font is simply a matter of over-writing this chars EQU $5C36 ; from Jim's docs - POKE 23607=5C36 in hex ld hl,myfont-256 ; HL = font address - 256 as per docs ld (chars),hl ; from now on the ROM draws with it! ; rest of code is the same myfont: include "textfont.asm" ; "Cartoon" font from jimblimey.com There we go! Now that’s looking a little more elite. Sorry, I mean 31337. Strap in folks, we’re leaving lamer city… So at this point, a month or so in to my dabblings and I’d started to get a reasonable understanding of common idioms and was starting to feel like this was a project that might actually go somewhere. The problem started when I moved on to trying to implement actual demo effects: It became very apparent, very quickly, just how much I didn’t know about how this all worked. Not to mention the fact that at this level, you’re really closely exposed to the physical hardware intricacies, including the design decisions and trade-offs that the hardware designers had to make when working with the extremely limited specs of the early 1980s. Cursed Screen Layout The first such problem - and the cause of probably 90% of my pain - is the Speccy’s extremely weird screen layout. Rather than standard bitmapped display you might expect on more modern hardware, it is instead split into 3 sections, each with a set of interleaved rows (so if you keep adding 1 to a screen address, eventually instead of reaching the next row, you’ll instead end up 8 rows down, and so on until you hit the bottom of the section and then loop back to the row underneath where you started!) followed by a block of “attribute” bytes which determine the colours of each 8x8 block. The attribute blocks are the reason why each 8x8 character cell can only have two colours (a PAPER and INK attribute) which led to the garish “colour-clash” of spectrum games and distinctive style where artists try to keep inside 8x8 blocks as much as possible. But it was the layout of the bitmap portion that is such a mind twister. Download the video You can see this illustrated whenever a game reads in a loading screen from tape. You can see each section built up with the interleaved rows one at a time, and then finally the colour and other attributes (like flashing ). There’s a lot of detail at the fantastic Break Into Program site. In the defense of the designers, they were working with extremely limited hardware and I don’t think the Speccy was ever imagined as a games machine. When you start shuffling bytes around, you realise that this approach is really optimised for displaying text which is neatly arranged into 8x8 character boundaries. For example, using this “scrambled” bit order, if you have a screen address in the hl register pair, you can quickly get to the next pixel line of a given address by simply incrementing the “high” byte. There’s a bunch of other “cheap moves” you can make by simply shifting bits around: Move Instruction Right 8 pixels inc l Left 8 pixels dec l Down 1 pixel line inc h Up 1 pixel line dec h Down 1 character row add 32 to l It was a reasonable choice for a machine that would probably spend a lot of it’s time displaying BASIC program listings - it’s only when you start trying to write demo or sprite routines that it starts to feel really weird. This also means you’ll pretty quickly need a routine to convert X,Y addresses to the relevant memory addresses. Again, L Break Into Program has a useful routine and if you disassemble more or less any spectrum game or demo, you’ll find a version of this procedure somewhere. And yes, this all became horribly relevant later on… The Scroller Download the video Admittedly, not the most interesting demonstration - but this was the first time I had made something move on the Spectrum’s screen with assembly! I watched this with pride more times than I care to admit :) I had a snazzy logo and “Hello World” text, but that doesn’t make for a very interesting intro! The first effect I wanted was the scrolltext - without that, it’s not really an intro. Now, the Amiga had all kinds of hardware sprites, blitter operations, custom chips but the Spectrum had none of that. I was going to have to work out how to do things the hard way - “banging on the metal”, literally by shifting bits around the screen memory by hand and having the CPU do all the work. This is where understanding the Speccy’s weird screen and character cell layout is so important. It turns out there’s a neat trick as long as I stay inside one character row - there’s a RL instruction, which takes a byte and “rotates” it left. It inserts whatever is in the carry flag into the rightmost bit, and whatever gets shifted out ends up in the carry bit again. That means if you string a bunch of them together, you can move a set of pixels across the screen - the moment they “fall off” one character cell, they end up in the carry flag and then the next rl instruction inserts them at the beginning of the next character cell to the left. Repeat 32 times, and you’ve just moved a pixel (or set of pixels) across a line of the screen: linestart EQU $4880 ; leftmost byte of the line 12 rows down lineend EQU linestart+31 ; rightmost byte column ($489F) linelen EQU 32 ; one whole line is 32 bytes (character cells) across dot EQU %00000001 ; the dot "shape", just one pixel lit at the far right end newdot: ld a,dot ; load the dot shape (only rightmost pixel lit) into a ld (lineend),a ; store it in the rightmost byte of the line, pixel is now lit frame: halt ; wait for the next frame, otherwise it whizzes across too fast to see ld hl,lineend ; hl = last (rightmost) byte of the line ld b,linelen ; loop counter, 32 bytes to work through or a ; trick to clear the carry flag, or against itself leaves a register ; unchanged but clears various bits including the carry flag ; see https://jnz.dk/z80/or_r.html shift: rl (hl) ; rotate this byte left (00000001 -> 00000010) dec hl ; point hl at the next byte to the left djnz shift ; loop construct: b=b-1 and then jump back to shift label until b=0 jr c,newdot ; now we've shifted the whole line, so if the carry flag is set now it means ; the dot has just left the screen, so we jump back to the start jr frame ; otherwise it is still somewhere on the screen, so keep shifting! Download the video Hell yeah. It’s almost a proper scrolltext now! Wrapping this in another loop that increments h each time (remember - that gets us moving down a scanline) gets us moving an entire character cell. Here you can see it moving an entire row of my “Hello world” message. There was however a lot more work needed to make a proper scrolltext, with each letter appearing smoothly. I started off by just printing an entire character at a time on the right-hand edge of the screen - it worked, but was pretty ugly as you saw each new character appear at once in the right-hand side of the screen, scrolling 8 pixels over and then the next character flashed up. Instead, I wanted to make each character appear pixel by pixel and that required a bit more bit juggling. The routine I ended up with effectively streams each character in, line by line, pixel by pixel using the font data directly instead of calling the ROM print character routine. It does this by looking up the ASCII code for a character, and then fetching the corresponding font representation into a 8-byte buffer, one byte for each line of data. The font data starts with the space character (character code 32), so a lookup is simply font code base address + (ascii code - 32) * 8 to get to the bits for that letter. Arranged into an 8x8 grid, the bits for the letter H in the font data look like this. For each row of bits, I shift the line left using a neat trick. The byte is loaded into the a register, and then by adding itself to itself - add a,a we shift the value left (in binary, multiplying by 2 moves everything over one bit) so that the pixel I want to draw ends up dropping into the carry flag. This is an idiomatic way of doing it in Z80 code as it’s faster and smaller than the sla instruction. And every cycle quickly adds up when you’re re-drawing 256 bytes every frame! The character buffer then looks like this, with the top line shifted. Now, with the hl register pair pointing at the correct location on the screen, I can then use the rl trick discussed above. This shifts a scanline from the bottom character row of the screen left by one pixel and draws my pixel in the rightmost position by moving it in from the carry flag. Looping 32 times over this moves the entire row by 1 pixel with my new pixel inserted at the right. I then repeat this step again using inc h to move down one scan line until the entire character column has been drawn. At this point, the character buffer looks like this, and all 8 lines of the bottom screen row have all been shifted left by one bit by looping over all 32 character columns. The whole thing then repeats another 7 times until the entire character has been smoothly scrolled onto the screen. So just to re-state this because I think it’s worth the emphasis - just to scroll a message across the bottom of the screen, we’re having to loop over and re-draw 256 pixels every frame! It shows how much faster assembly is than the BASIC interpreter built into the spectrum’s ROM. I found it amazing to see my code finally working and imagining all the bits flying around and registers flapping away. It really makes me appreciate just how much effort went into creating the games I played back then. Music The original Amiga module I converted. A proper 1980s ear-worm! Of course, any good intro needs some music. On the Amiga, the standard was the “tracker” module format, which has a long and involved history of different forks, versions and tweaks. A song is built up from a series of patterns, each made up of rows of instructions. Each instruction usually includes note or pitch information, which sample to play and various parameters such as volume, effects - or in the case of some instructions, which position in the song to jump to. In many ways, it’s almost like a programming language for music. Using ProTracker as an example, a typical note cell is 4 bytes, shown on-screen as something like C-2 01 C20. That breaks down into three fields: the note and octave (C-2), the sample number (01), and the effect and its parameter (C20 - in this case, “set the volume to $20”, which is half of the maximum $40). A close-up of Protracker playing a tune. Note the VUmeters - another classic demo effect I would pinch later on! For an impressive demonstration of what’s possible with a tracker and to see what the note data looks like on-screen, check out the work of h0ffman, and in particular his winning “Professional Tracker” entry from the Revision 2014 competition. He’s an undisputed master of the science (or art!) of tracking, and his “Secrets of Protracker” talk gives a great overview of what’s possible. The Amiga’s sound hardware far eclipsed anything else on the market at the time, and the ZX Spectrum is paltry by comparison. The original 48K models only offered a rudimentary “beeper”, but the later 128K models included a 3-channel AY-3-8912 sound chip. This was very popular in 8-bit micros of the time, and a more-or-less identical variant was also used in the Atari ST. While it only offers basic waveforms and sound processing, it does enable multi-channel “chip music”, and so it became the mainstay of the ZX Spectrum demoscene. It also spawned a variety of SoundTracker clones on the Spectrum itself, although most musicians these days prefer to work in modern tools like Vortex Tracker II, which runs natively under Windows. The Conversion Pex looking back on his demos - this starts at the point where he features my letter and plays a bit of my cover version! For my intro, I wanted to port an Amiga module - and I had the perfect one in mind. One of my favourite Amiga music disks was “Sounds Of Gnome”, released in 1989 by the Swedish duo “Mahoney & Kaktus”. Mahoney also wrote NoiseTracker, one of the first really widely used trackers on the Amiga, and the program that served as my teenage introduction to making computer music of my own. Back in 2018 I made a rock cover of one of the other tunes on that disk, and wrote to Mahoney (real name “Pex”) about it. He ended up showing my “music video” at a demo conference, which I am eternally grateful for! So it seemed only fitting that I mangle another song from that collection with a Spectrum conversion… Sorry, guys! I picked the track “Milkman” which was written by “Glue Master”, an associate of Mahoney’s who had a few of his tracks featured on the compilation. It’s an ideal track for converting: It’s short (only 7 patterns in total), most of those patterns share the same chord sequence, and it looked fairly easy to squeeze the original 4 channels down into the Spectrum’s 3. Plus, it’s an incredibly catchy tune and I knew it well - I’ve been listening to it for 35+ years and often jam along to it on guitar or bass. The conversion started off pretty easily: all I really had to do was copy the patterns note-for-note from the Amiga mod, which I had open in OpenMPT as my Windows tracker. Below is a session with both trackers open on the same pattern - the original Amiga mod on the left, and my 3-channel Spectrum conversion on the right. Note the 11.7 in the 3rd channel of the Spectrum version. This is a “pitch slide down” effect in Vortex Tracker syntax, which drops the pitch by about a quarter-tone. It’s repeated on the next line, so the overall effect is a slide down from the A# that leads in to the next bass note. The original Amiga mod does the same thing using its own syntax - those are the 202s you can see on the same notes, at the bottom of the 4th track. Using just a single “beep” sound for everything was enough to check everything was working, and after a few weeks of hacking on it and re-arranging things in the evenings, I had a basic version which sounded surprisingly good for a first draft: Download the MP3 instead. I have to give a special mention to this forum post which was a massive help getting me started. It’s probably the best introduction to Vortex Tracker I’ve found - many, many thanks to the original author for taking the time to write it all up! The real challenge was making it sound as close to the original as possible. That meant building up a library of better-sounding samples for the different parts, and making extensive use of Vortex Tracker’s “ornaments”. Vortex Tracker comes with a great selection of sounds, and fortunately I’m also a weirdo who likes to listen to chiptunes in the background while I work! So whenever I found something particularly enticing, I filed it away for later pilfering inspiration. A Vortex Tracker sample isn’t really a “sample” in the Amiga sense of a recorded sound - it’s a definition of how a note sounds, one line per frame. Each line can mix in a tone or the white noise generator (useful for snare effects and suchlike), along with various other parameters. Ornaments, on the other hand, give you extra control over what pitch a note plays - and this enables really useful tricks like “arpeggio chords”. For example, an ornament of 0,4 makes the channel switch rapidly (25 times a second) between the note being played and the one 4 semitones above it. It happens so quickly that it sounds much like two notes played at the same time, which means you can build up full-sounding triad chords using only two channels. Take the opening chord in my fully converted version, Cm: It plays C in the first channel and the minor 3rd (D#) in the second, then I apply an ornament to the second channel to bring it up 4 semitones to the 5th (G). That gives all three notes of the chord, while keeping the 3rd channel free for other parts. As well as making much more efficient use of the limited 3 channels, this also gives Spectrum music that characteristic “warbling” sound, which I find really appealing in a retro kind of way! Between tricks like this and samples that mix tones with white noise (to make a combined “bass-snare” sound, for example), I managed to condense the full track down to 3 channels. Many months later, I have to say I’m really pleased with the final result. Playback The original site that was home to Vortex Tracker, documentation and other resources appears to be offline at the moment. I managed to grab a copy of everything I needed from archive.org - you’ll want the PTxTools.7z file for the playback routines discussed here. The playback was simple in theory: As this overview notes, you basically just pull in the player routine, include your module file and call the playback routine once per frame in your main loop. The catch is, none of the source code versions supplied in the archive worked with my Pasmo assembler. Fortunately, there is a pre-compiled binary object shipped alongside the source (VTII10bG in the PTxTools/PT3Play directory). This just has to be loaded in at address $C000 - this cannot be changed, as it has lots of absolute addresses baked in. My code to handle module playback now looks something like this, with the binary module loaded in place: ; Define the start address of the player routine and functions we need player EQU $C000 ; hard-coded org address of binary player plinit EQU player+3 ; init function in docs, module address in hl plplay EQU player+5 ; play and move on one frame ; from the docs - this "analyzes PT3 module header and prepares ; corresponding note and volume tables" ld hl,moduleimg call plinit ; main loop hold: halt call plplay call scroll ; other effects / routines go here... jr hold moduleimg: incbin "assets/music.pt3" Starfield Download the video Well, that didn’t work as hoped. TNFStro - tortoise edition. This brings me to the main event. No self-respecting cracktro is complete without a starfield! My first, naive approach to the classic starfield effect was to scatter random stars across the screen, then move down each scan-line applying the same rl trick I’d used for the scroll-text. I figured I could get parallax by shifting each row a different amount: One pixel for the first row, two for the second, three for the third, then back to one again, all the way down the screen to give the impression of several “fields” of stars moving at different speeds. It turned out there were two major problems with that approach. Firstly, shifting an entire row smeared whatever else was on it along with the stars - which I’d expected, and had filed away as a problem for future me. The bigger issue was speed: by the time I’d finished the first three rows I’d already performed hundreds of shift operations, and it was nowhere near fast enough to fit inside a single frame. Everything crawled across the screen, and the music playback slowed down so much that you could actually hear the arpeggio ornaments warbling away as separate notes. Clearly I was going to have to keep track of the individual stars instead. I found this routine, which was more-or-less what I had in mind and started me off with the idea of a structure to hold the state of each star. It also uses a screen address lookup much like the one from “Break Into Program”. Download the video Dammit. Back with another funeral dirge rendition :( That got things moving, but the co-ordinate lookup and tracking proved expensive to do (it has to run for every star, every frame as well as keep track of the old position for erasing) and I’d been ignoring the bigger question how to display stuff on top of the starfield. My original idea was simply to draw the stars and then paint text over them each frame, which blew the frame budget just as comprehensively as my first attempt. If I wanted something more interesting than an empty starfield, I’d have to keep track of which cells were occupied by text and avoid drawing stars in them in the first place, rather than over-painting them every time. After lots of scribbling in my notepad and visualising rows of bits, I eventually solved both challenges by “cheating” fairly liberally. So now, I don’t generate a random starfield at all. Instead, I built a table of pre-calculated Y and X position pairs: stdata: defb 109, 25, 88,182, 42, 96, 155,226 defb 59,254, 137,169, 160, 84, 125,193 ; ... and so on, for 32 stars ... This saves faffing about with an RNG, and once I have a layout that looks good I know it’ll always look good. I can also “fudge” things and ensure no two stars share the same y co-ordinate which sidesteps a whole class of problems like stars sharing a scanline, and having to deal with them overtaking each other. The “random” layout is generated by a simple Ruby script: #!/usr/bin/env ruby PAIRS = 32 PER_LINE = 4 # pairs per line Y_RANGE = 40..175 # exclude rows taken up by my logo & bars X_RANGE = 0..255 raise "y range too small" if Y_RANGE.size < PAIRS # no duplicate y values ys = Y_RANGE.to_a.sample(PAIRS) pairs = ys.map { |y| [y, rand(X_RANGE)] } pairs.each_slice(PER_LINE) do |row| body = row.map { |y, x| format('%3d,%3d,', y, x) }.join(' ') puts "defb #{body.chomp(',')}" end When the routine initialises, I convert those co-ordinate pairs into a table that tracks the state of each star. So rather than storing a position and translating it to a screen address every frame, each of my star records now looks like this (offsets from the base address): STSCR EQU 0 ; screen address of the byte the star is in, e.g. $4123 STOCC EQU 2 ; a pointer to the "occupancy check" table cell STMASK EQU 4 ; the star's bit pattern, e.g. %00010000 STSTEP EQU 5 ; "speed" - num. of pixels this star travels per frame The key thing is that there are no X/Y co-ordinates at all. For each star I store the screen byte it sits in and the bit mask within that byte, so it’s already in the form the screen writes need. That means no address conversion per frame, just simple bit shifting and rotation. Moving a star is then dead simple. I rotate its mask left with rlca, and when the bit rotates out of bit 7, it lands back in bit 0 with the carry flag set. That’s the signal I need to move the star one character cell to the left, which is one of the “cheap moves” from earlier. Because I’m tracking which cell each star is in, the collision check comes almost for free. That’s what the second pointer is for: STOCC (Star Occupancy) moves in step with STSCR through a table holding one byte per character cell. The “typewriter” routine sets a byte to 1 for every cell containing a text character or the cursor, so checking whether a star is hidden behind text is a single or instruction. If the cell is occupied, I skip drawing the star but carry on rotating and stepping it along until it reaches a cell where it can be displayed: ld l,(ix+STOCC) ld h,(ix+STOCC+1) ld a,(hl) ; A = the occupancy check for this cell or a ; test A against zero jr nz,stmove ; it's occupied, so leave the screen alone Parallax is still handled by moving each star a different number of pixels per frame, but the cost is a lot lower this time as I’m only touching the cells the stars actually occupy, rather than every byte of every row. Download the video That’s looking more like it! The typewriter routine is a simple copy of the font data from memory, and is very cheap as we can use the “cheap moves” to quickly fetch the 8x8 block. I also skip drawing any space character so it leaves the “occupancy cell” unmarked. This means the stars keep drifting through the gaps in the text - you can see here the odd star passing in between the words. I switched to ALL CAPS writing for the final version which helps the effect even more as the font is quite blocky, so you don’t really notice stars skipping the spaces inside a glyph at all. Misc Notes A few odds and ends that didn’t warrant a section of their own… VU Meters Some sort of graphic display bouncing along with the audio playback was another common early demo trope. I did originally consider a real-time graphic equaliser, but that would have been far too complex, probably in terms of spare cycles per frame, and definitely in terms of my limited mental capacity at this point! So I settled for a simple set of bouncing meter bars instead. It’s worth a mention mostly because it gives me an excuse to link to the awesome chibiakumas website! I just love the late 90s web design on display - one can argue that readability and accessibility take a hit, but it’s just so much fun to encounter a site that still looks like this. It’s a clear labour of love and there’s a wealth of content there as well, if your eyeballs are up to it. Their guide on Making Sound with the AY-3-8910 was what showed me how to get the data I needed to draw the meter bars. The examples write a register number to $FFFD to select a register, and I realised that once a register is selected, I could use the in instruction to read its value back out instead of writing a new one. As the page says, R8, R9 and R10 hold the channel amplitudes, so they feed nicely into the height parameter for drawing a bar. Getting the amplitude for channel A is then as simple as: ld a,8 ; Channel A register call ayread ; ... rest of code here ... ayread: ld bc,$FFFD ; Register select port out (c),a in a,(c) ret Twinkle, Twinkle, Little Star This is a simple effect that makes the header logo a little more interesting. I’m not sure where I originally got the idea - I’ve definitely seen something like it on a few late 80s/early 90s demos, though. All I wanted was to pick random star dots in the logo background and animate them. Here’s the header image loaded in ZXPaintbrush with the character cell grid showing. By happy chance, there are four stars which each sit neatly inside a single character cell with enough surrounding space, so they can be used to give an animated “twinkle” effect by drawing a cross out from the centre point and back in again. These dots map to the following co-ordinates, which target the top scanline of each fully-grown star (two pixels up from the centre): twkdata: defb 1,5 ; char row 0, col 5, centre y=3 ink 5 defb 1,20 ; char row 0, col 20, centre y=3 ink 1 bright defb 26,7 ; char row 3, col 7, centre y=4 ink 4 defb 25,22 ; char row 3, col 22, centre y=3 ink 6 bright The “twinkle” routine itself is simple enough: it adds a pixel on either axis to grow the star, waits a few frames, then shrinks it back down again. The interesting bit was how I made the twinkle order pseudo-random. Remember that my starfield was entirely pre-calculated from a data table generated by a Ruby script, which meant I’d never actually needed a random number generator in assembly. From Jonathan Cauldwell’s “How To Write Spectrum Games” e-book, I found another great “hack” which simply walks through the start of the Spectrum’s ROM, picking out random-looking bytes as it goes. As he says, he uses it in most of his games, and it does the job in a really simple way - as long as you stay below the block of structured, repeating data that sits 8Kb into the ROM. twkrand: push hl ld hl,(twkseed) ld a,h and 31 ; keep it within the first 8k of ROM ld h,a ld a,(hl) ; the "random" byte inc hl ld (twkseed),hl pop hl ret Out Of Space About halfway through writing the intro, I hit a fairly predictable problem: I had run out of memory. In an attempt to keep everything self-contained in one block of code, I was pulling all of the assets (the screen background, the player binary, the music file etc.) into the program itself, and then using the ldir block copy instruction to move each one into its correct location. Since that meant effectively storing every block twice, and the player routine had a hard-coded start address of $C000, I quickly ran out of room and my code started crashing into the playback routine. The simplest option was to split things out into separate code blocks, load each one directly to the address it needed to live at, and wrap the lot in a multi-stage BASIC tape loader: 10 CLEAR 32767 20 BORDER 0: PAPER 0: INK 0: CLS 30 LOAD "" CODE 40 LOAD "" CODE 50 LOAD "" CODE 60 RANDOMIZE USR 32768 This loads the main program, then the player routine directly into $C000, and finally the background screen straight into screen memory. It does mean the background starts appearing before the demo has fully loaded, but as I imagine most people will be seeing this running in an emulator or on my TNFS site, it doesn’t really matter. Plus, at this point I was in danger of disappearing down yet more rabbit holes (I still really want to work out how to do multi-colour border effects!) and I’d never have got this thing released. The Zen Of Z80 This whole project is a really weird one to look back at. On the one hand, I’ve fulfilled a childhood ambition and have something I’m very proud of. It’s also pretty unusual these days to end up with a binary where you can say pretty much what every single byte is doing and the story of how it got there. But on the other hand - I now have an answer for why I don’t code in assembly: It’s because I value (what’s left) of my sanity! The thing is, in order to write code like this you have to start thinking like the machine itself. You have to visualise the registers and the status flags flapping away, graphics as lines of bits slotted into strange orders, loops and conditionals as binary logic and so on. Getting this close to the hardware definitely brought about a greater sense of attachment to the hardware of my childhood, and it’s easy to see why assembly coders and sceners get so passionate about it. That said, I’m in no hurry to start re-writing my body of work in assembly! The projects I really enjoy are those that give me an excuse to learn something for the sheer hell of it, and to slow down a little from the ever-increasing velocity and demands of the modern work world. Going back to Z80 code is probably about the furthest I could have taken this, short of going into hardware hacking. I found it at times an intriguing puzzle, often a frustrating exercise that nearly had me throwing in the towel at several stages. This project has certainly taken the longest to go from the “idea” stage of any given feature to any kind of meaningful output. True, I didn’t spend every day of the last year working on it - my actual coding time was probably only a few hours over a couple of evenings a week. But I’ve certainly been thinking about it and pondering it most days (as my dogs will attest). The biggest breakthroughs and “aha!” moments usually didn’t come from behind the keyboard; they came from scribbles in my notebook and thinking about processor states while doing something unrelated. It has however given me massive appreciation and respect for the coders who pull this stuff off and make it look easy. And that goes double for those kids and bedroom coding superstars who managed all of this back in the 80s, and had to work it all out by experiment themselves. I only managed to get this far because of numerous books, online tutorials, GitHub repositories with sample code, YouTube videos… I can’t imagine how much longer this would have all taken me if the only resources I had were my Spectrum, a cassette tape recorder and a notepad. So, as I draw a line under this project - for now! - I raise a glass, and in the words of h0ffman: “Shout out to the hackers, graphicians and protrackers.”

20 hours ago • 1 votes
CalVer 26.0: 10th Anniversary Edition

A decade ago, a little bit of history was made. I didn't realize it, but a colleague made a great point, one of those real mind-changing points that seem too obvious to admit same-day. But, the next day, calver.org was born. At the time my team maintained the Python infrastructure for eBay and PayPal, and we were stuck deciding whether we were really ready for a "major" 1.0 release. Semantic Versioning was the only game in town and "major" means "big", right?! Thankfully, a wiser colleague mentioned: Ubuntu and Twisted don't struggle with version number debates. They slap a date on it and keep shipping. In fact, their date-based versions were even better because you always knew where it stood, in terms of updatedness and support. The only problem is that no one really knew about it. Somehow, this problem solving versioning alternative, arguably as old as history itself, had gone nameless for millenia, conspiring to make me feel foolish in an office meeting. Never again! Ten years of adoption Fast forward 10 years, we've seen CalVer adopted by Apple, Nvidia, JetBrains, and countless others. (We have a timeline!) The site may have more inbound links than any other project of mine. Apple made the biggest jump, at WWDC 2025: iOS went from 18 to 26 macOS from 15 to 26 watchOS from 11 to 26 and visionOS from 2 to 26 All landing on one, consistent number like a car's model year. I still remember the texts from the Venn diagram fanbase of my friends who love Apple and reasonable versioning. No such texts from when NVIDIA announced calendar versions across the GPU Operator, RAPIDS, and its monthly NGC containers, but still very cool. Open source, too: Home Assistant, pip, CockroachDB, and yt-dlp all ship on dates, with plenty more on the users page. The conversation even reached the language core; PEP 2026 proposed versioning CPython as 3.YY, and it almost happened, too. And it's never too late, time marches on! Fixing the notation But I don't think I got every detail right from day 1. That's the main motivator for CalVer 26. It's high time to start righting a couple idiosyncratic token design choices, starting with some additions: Meaning Before 26.0 26.0 Full year YYYY YYYY Short year (6, 16) YY YY Zero-padded year (06, 16) 0Y 0Y Short month (1 ... 12) MM M Zero-padded month (01 ... 12) 0M 0M Short week (1 ... 52) WW W Zero-padded week (01 ... 52) 0W 0W Short day (1 ... 31) DD D Zero-padded day (01 ... 31) 0D 0D Seeing double First, the doubled letters. From the first version (16.6), MM and DD meant the unpadded month and day, which reads backwards to anyone who knows date formats (ISO 8601's YYYY-MM-DD, Java, moment.js, day.js), as some community members correctly pointed out. I was ready to flip them, until I checked what people actually use: most projects with a YY.MM.MICRO badge (conda, Twisted, Ansible's tooling) don't pad, and more than a dozen other version management tools (like bumpver and bump-my-version) implement the old meaning. So, it's too late to flip MM's meaning. Instead, 26.0 deprecates it and offers a more explicit and hopefully clearer option: M is the short month, 0M the padded one, and MM is a technically-retired synonym for M. In case you're wondering, the explicit 0M was me being overinspired by Ubuntu's approach, perhaps: 6.06 pads its month but not its year, and YY.0M says exactly that. To pad or not to pad I think it's worth a detour into why padding is even a thing anyways. It's become important now that new ecosystems have emerged that enforced SemVer formatting semantics, and I wanted clear guidance about on the spec site. SemVer forbids leading zeros outright, so Cargo rejects 26.04.0 and Go modules reject v26.04.0. Even Python's packaging spec normalizes leading zeros away, so you can tag 2026.08.19 if you want, but PyPI will still show 2026.8.19. NVIDIA's GPU Operator docs put it this way: "Zero padding is omitted for month to be still compatible with semantic versioning." CalVer was always intended to drop in where SemVer was used. So 26.0 recommends unpadded (YYYY.M.D) as a sane "pure" default for software libraries. But libraries are not the only objects of versioning schemes. The exception is a version that becomes a filename, an image tag, or an object-store key that gets listed lexically. There, padding keeps 26.10 sorted after 26.09, which is why Ubuntu, NixOS, and NVIDIA's own NGC containers pad. More evidence of teams designing their versions. We love to see it. Our FAQ has a longer discussion of the padding issue, as well. Optional segments There was never any rule against them, but 26.0 makes optional trailing segments more explicit with square brackets. Now, yt-dlp's scheme can finally be written down: YYYY.0M.0D[.MICRO]. For the CalVer badges I could find on GitHub, they all stay valid for now. I've got a note on the deprecated spellings and a new copy-paste badge section for new ones. What else is new? It's always a great time to add more citations to the site. A spec changelog; the spec now versions itself: 16.6, 19.7, 26.0. A FAQ: breaking changes, same-day releases, and padding. The users page, rebuilt by category, with past users of note (schemes change; that's fine) and tooling. Case studies: Apple and NVIDIA in, yt-dlp replacing youtube-dl. Much of the thinking behind these changes happened in the GitHub issue tracker over the years, and 19 or so issues close with this release. That's where ideas for CalVer should go, so by all means, open an issue, and we'll get it sorted! In due time, of course. In closing, I can't believe I still love belaboring these numbers so much. Thanks to all (but especially Mark, Glyph, Hugo, issue reporters, translators, and maintainers) for the discussion, and ultimately making the most timely versioning system a timeless classic. See also 2016 announcement Designing a version My Yap on Why CalVer beats Semver

2 days ago • 1 votes
"You didn't do it right"

Four sincere attempts at OKRs, four failures, and every time: "you didn't do it right." Maybe the framework doesn't fit you -- so how do you find one that does?

2 days ago • 1 votes
We solved SQLite's single-writer limitation

Multiple concurrent writers. Multiple processes. Same SQLite. No modifications.

4 days ago • 1 votes
If your team is happy, are you doing a good job?

Brilliant jerks, ZIRP-era managers, and how psychological safety lost the plot. Part 3 of my conversation with Dr. Cat Hicks.

5 days ago • 1 votes
📚 BoredReading

You seem to be enjoying this.

Join free to unlock everything.

Create free account

Already have an account? Sign in