Full Width [alt+shift+f] FOCUS MODE Shortcuts [alt+shift+k]
Sign Up [alt+shift+s] Log In [alt+shift+l]
57
A week ago, somebody added malicious code to the tj-actions/changed-files GitHub Action. If you used the compromised action, it would leak secrets to your build log. Those build logs are public for public repositories, so anybody could see your secrets. Scary! Mutable vs immutable references This attack was possible because it’s common practice to refer to tags in a GitHub Actions workflow, for example: jobs: changed_files: ... steps: - name: Get changed files id: changed-files uses: tj-actions/changed-files@v2 ... At a glance, this looks like an immutable reference to an already-released “version 2” of this action, but actually this is a mutable Git tag. If somebody changes the v2 tag in the tj-actions/changed-files repo to point to a different commit, this action will run different code the next time it runs. If you specify a Git commit ID instead (e.g. a5b3abf), that’s an immutable reference that will run the same code every time. Tags vs commit...
5 months ago

Comments

Improve your reading experience

Logged in users get linked directly to articles resulting in a better reading experience. Please login for free, it takes less than 1 minute.

More from alexwlchan

Handling JSON objects with duplicate names in Python

Consider the following JSON object: { "sides": 4, "colour": "red", "sides": 5, "colour": "blue" } Notice that sides and colour both appear twice. This looks invalid, but I learnt recently that this is actually legal JSON syntax! It’s unusual and discouraged, but it’s not completely forbidden. This was a big surprise to me. I think of JSON objects as key/value pairs, and I associate them with data structures like a dict in Python or a Hash in Ruby – both of which only allow unique keys. JSON has no such restriction, and I started thinking about how to handle it. What does the JSON spec say about duplicate names? JSON is described by several standards, which Wikipedia helpfully explains for us: After RFC 4627 had been available as its “informational” specification since 2006, JSON was first standardized in 2013, as ECMA‑404. RFC 8259, published in 2017, is the current version of the Internet Standard STD 90, and it remains consistent with ECMA‑404. That same year, JSON was also standardized as ISO/IEC 21778:2017. The ECMA and ISO/IEC standards describe only the allowed syntax, whereas the RFC covers some security and interoperability considerations. All three of these standards explicitly allow the use of duplicate names in objects. ECMA‑404 and ISO/IEC 21778:2017 have identical text to describe the syntax of JSON objects, and they say (emphasis mine): An object structure is represented as a pair of curly bracket tokens surrounding zero or more name/value pairs. […] The JSON syntax does not impose any restrictions on the strings used as names, does not require that name strings be unique, and does not assign any significance to the ordering of name/value pairs. These are all semantic considerations that may be defined by JSON processors or in specifications defining specific uses of JSON for data interchange. RFC 8259 goes further and strongly recommends against duplicate names, but the use of SHOULD means it isn’t completely forbidden: The names within an object SHOULD be unique. The same document warns about the consequences of ignoring this recommendation: An object whose names are all unique is interoperable in the sense that all software implementations receiving that object will agree on the name-value mappings. When the names within an object are not unique, the behavior of software that receives such an object is unpredictable. Many implementations report the last name/value pair only. Other implementations report an error or fail to parse the object, and some implementations report all of the name/value pairs, including duplicates. So it’s technically valid, but it’s unusual and discouraged. I’ve never heard of a use case for JSON objects with duplicate names. I’m sure there was a good reason for it being allowed by the spec, but I can’t think of it. Most JSON parsers – including jq, JavaScript, and Python – will silently discard all but the last instance of a duplicate name. Here’s an example in Python: >>> import json >>> json.loads('{"sides": 4, "colour": "red", "sides": 5, "colour": "blue"}') {'colour': 'blue', 'sides': 5} What if I wanted to decode the whole object, or throw an exception if I see duplicate names? This happened to me recently. I was editing a JSON file by hand, and I’d copy/paste objects to update the data. I also had scripts which could update the file. I forgot to update the name on one of the JSON objects, so there were two name/value pairs with the same name. When I ran the script, it silently erased the first value. I was able to recover the deleted value from the Git history, but I wondered how I could prevent this happening again. How could I make the script fail, rather than silently delete data? Decoding duplicate names in Python When Python decodes a JSON object, it first parses the object as a list of name/value pairs, then it turns that list of name value pairs into a dictionary. We can see this by looking at the JSONObject function in the CPython source code: it builds a list pairs, and at the end of the function, it calls dict(pairs) to turn the list into a dictionary. This relies on the fact that dict() can take an iterable of key/value tuples and create a dictionary: >>> dict([('sides', 4), ('colour', 'red')]) {'colour': 'red', 'sides': 4} The docs for dict() tell us that it` will discard duplicate keys: “if a key occurs more than once, the last value for that key becomes the corresponding value in the new dictionary”. >>> dict([('sides', 4), ('colour', 'red'), ('sides', 5), ('colour', 'blue')]) {'colour': 'blue', 'sides': 5} We can customise what Python does with the list of name/value pairs. Rather than calling dict(), we can pass our own function to the object_pairs_hook parameter of json.loads(), and Python will call that function on the list of pairs. This allows us to parse objects in a different way. For example, we can just return the literal list of name/value pairs: >>> import json >>> json.loads( ... '{"sides": 4, "colour": "red", "sides": 5, "colour": "blue"}', ... object_pairs_hook=lambda pairs: pairs ... ) ... [('sides', 4), ('colour', 'red'), ('sides', 5), ('colour', 'blue')] We could also use the multidict library to get a dict-like data structure which supports multiple values per key. This is based on HTTP headers and URL query strings, two environments where it’s common to have multiple values for a single key: >>> from multidict import MultiDict >>> md = json.loads( ... '{"sides": 4, "colour": "red", "sides": 5, "colour": "blue"}', ... object_pairs_hook=lambda pairs: MultiDict(pairs) ... ) ... >>> md <MultiDict('sides': 4, 'colour': 'red', 'sides': 5, 'colour': 'blue')> >>> md['sides'] 4 >>> md.getall('sides') [4, 5] Preventing silent data loss If we want to throw an exception when we see duplicate names, we need a longer function. Here’s the code I wrote: import collections import typing def dict_with_unique_names(pairs: list[tuple[str, typing.Any]]) -> dict[str, typing.Any]: """ Convert a list of name/value pairs to a dict, but only if the names are unique. If there are non-unique names, this function throws a ValueError. """ # First try to parse the object as a dictionary; if it's the same # length as the pairs, then we know all the names were unique and # we can return immediately. pairs_as_dict = dict(pairs) if len(pairs_as_dict) == len(pairs): return pairs_as_dict # Otherwise, let's work out what the repeated name(s) were, so we # can throw an appropriate error message for the user. name_tally = collections.Counter(n for n, _ in pairs) repeated_names = [n for n, count in name_tally.items() if count > 1] assert len(repeated_names) > 0 if len(repeated_names) == 1: raise ValueError(f"Found repeated name in JSON object: {repeated_names[0]}") else: raise ValueError( f"Found repeated names in JSON object: {', '.join(repeated_names)}" ) If I use this as my object_pairs_hook when parsing an object which has all unique names, it returns the normal dict I’d expect: >>> json.loads( ... '{"sides": 4, "colour": "red"}', ... object_pairs_hook=dict_with_unique_names ... ) ... {'colour': 'red', 'sides': 4} But if I’m parsing an object with one or more repeated names, the parsing fails and throws a ValueError: >>> json.loads( ... '{"sides": 4, "colour": "red", "sides": 5}', ... object_pairs_hook=dict_with_unique_names ... ) Traceback (most recent call last): […] ValueError: Found repeated name in JSON object: sides >>> json.loads( ... '{"sides": 4, "colour": "red", "sides": 5, "colour": "blue"}', ... object_pairs_hook=dict_with_unique_names ... ) Traceback (most recent call last): […] ValueError: Found repeated names in JSON object: sides, colour This is precisely the behaviour I want – throwing an exception, not silently dropping data. Encoding non-unique names in Python It’s hard to think of a use case, but this post feels incomplete without at least a brief mention. If you want to encode custom data structures with Python’s JSON library, you can subclass JSONEncoder and define how those structures should be serialised. Here’s a rudimentary attempt at doing that for a MultiDict: class MultiDictEncoder(json.JSONEncoder): def encode(self, o: typing.Any) -> str: # If this is a MultiDict, we need to construct the JSON string # manually -- first encode each name/value pair, then construct # the JSON object literal. if isinstance(o, MultiDict): name_value_pairs = [ f'{super().encode(str(name))}: {self.encode(value)}' for name, value in o.items() ] return '{' + ', '.join(name_value_pairs) + '}' return super().encode(o) and here’s how you use it: >>> md = MultiDict([('sides', 4), ('colour', 'red'), ('sides', 5), ('colour', 'blue')]) >>> json.dumps(md, cls=MultiDictEncoder) {"sides": 4, "colour": "red", "sides": 5, "colour": "blue"} This is rough code, and you shouldn’t use it – it’s only an example. I’m constructing the JSON string manually, so it doesn’t handle edge cases like indentation or special characters. There are almost certainly bugs, and you’d need to be more careful if you wanted to use this for real. In practice, if I had to encode a multi-dict as JSON, I’d encode it as a list of objects which each have a key and a value field. For example: [ {"key": "sides", "value": 4 }, {"key": "colour", "value": "red" }, {"key": "sides", "value": 5 }, {"key": "colour", "value": "blue"}, ] This is a pretty standard pattern, and it won’t trip up JSON parsers which aren’t expecting duplicate names. Do you need to worry about this? This isn’t a big deal. JSON objects with duplicate names are pretty unusual – this is the first time I’ve ever encountered one, and it was a mistake. Trying to account for this edge case in every project that uses JSON would be overkill. It would add complexity to my code and probably never catch a single error. This started when I made a copy/paste error that introduced the initial duplication, and then a script modified the JSON file and caused some data loss. That’s a somewhat unusual workflow, because most JSON files are exclusively modified by computers, and this wouldn’t be an issue. I’ve added this error handling to my javascript-data-files library, but I don’t anticipate adding it to other projects. I use that library for my static website archives, which is where I had this issue. Although I won’t use this code exactly, it’s been good practice at writing custom encoders/decoders in Python. That is something I do all the time – I’m often encoding native Python types as JSON, and I want to get the same type back when I decode later. I’ve been writing my own subclasses of JSONEncoder and JSONDecoder for a while. Now I know a bit more about how Python decodes JSON, and object_pairs_hook is another tool I can consider using. This was a fun deep dive for me, and I hope you found it helpful too. [If the formatting of this post looks odd in your feed reader, visit the original article]

4 months ago 32 votes
A faster way to copy SQLite databases between computers

I store a lot of data in SQLite databases on remote servers, and I often want to copy them to my local machine for analysis or backup. When I’m starting a new project and the database is near-empty, this is a simple rsync operation: $ rsync --progress username@server:my_remote_database.db my_local_database.db As the project matures and the database grows, this gets slower and less reliable. Downloading a 250MB database from my web server takes about a minute over my home Internet connection, and that’s pretty small – most of my databases are multiple gigabytes in size. I’ve been trying to make these copies go faster, and I recently discovered a neat trick. What really slows me down is my indexes. I have a lot of indexes in my SQLite databases, which dramatically speed up my queries, but also make the database file larger and slower to copy. (In one database, there’s an index which single-handedly accounts for half the size on disk!) The indexes don’t store anything unique – they just duplicate data from other tables to make queries faster. Copying the indexes makes the transfer less efficient, because I’m copying the same data multiple times. I was thinking about ways to skip copying the indexes, and I realised that SQLite has built-in tools to make this easy. Dumping a database as a text file SQLite allows you to dump a database as a text file. If you use the .dump command, it prints the entire database as a series of SQL statements. This text file can often be significantly smaller than the original database. Here’s the command: $ sqlite3 my_database.db .dump > my_database.db.txt And here’s what the beginning of that file looks like: PRAGMA foreign_keys=OFF; BEGIN TRANSACTION; CREATE TABLE IF NOT EXISTS "tags" ( [name] TEXT PRIMARY KEY, [count_uses] INTEGER NOT NULL ); INSERT INTO tags VALUES('carving',260); INSERT INTO tags VALUES('grass',743); … Crucially, this reduces the large and disk-heavy indexes into a single line of text – it’s an instruction to create an index, not the index itself. CREATE INDEX [idx_photo_locations] ON [photos] ([longitude], [latitude]); This means that I’m only storing each value once, rather than the many times it may be stored across the original table and my indexes. This is how the text file can be smaller than the original database. If you want to reconstruct the database, you pipe this text file back to SQLite: $ cat my_database.db.txt | sqlite3 my_reconstructed_database.db Because the SQL statements are very repetitive, this text responds well to compression: $ sqlite3 explorer.db .dump | gzip -c > explorer.db.txt.gz To give you an idea of the potential savings, here’s the relative disk size for one of my databases. File Size on disk original SQLite database 3.4 GB text file (sqlite3 my_database.db .dump) 1.3 GB gzip-compressed text (sqlite3 my_database.db .dump | gzip -c) 240 MB The gzip-compressed text file is 14× smaller than the original SQLite database – that makes downloading the database much faster. My new ssh+rsync command Rather than copying the database directly, now I create a gzip-compressed text file on the server, copy that to my local machine, and reconstruct the database. Like so: # Create a gzip-compressed text file on the server ssh username@server "sqlite3 my_remote_database.db .dump | gzip -c > my_remote_database.db.txt.gz" # Copy the gzip-compressed text file to my local machine rsync --progress username@server:my_remote_database.db.txt.gz my_local_database.db.txt.gz # Remove the gzip-compressed text file from my server ssh username@server "rm my_remote_database.db.txt.gz" # Uncompress the text file gunzip my_local_database.db.txt.gz # Reconstruct the database from the text file cat my_local_database.db.txt | sqlite3 my_local_database.db # Remove the local text file rm my_local_database.db.txt A database dump is a stable copy source This approach fixes another issue I’ve had when copying SQLite databases. If it takes a long time to copy a database and it gets updated midway through, rsync may give me an invalid database file. The first half of the file is pre-update, the second half file is post-update, and they don’t match. When I try to open the database locally, I get an error: database disk image is malformed By creating a text dump before I start the copy operation, I’m giving rsync a stable copy source. That text dump isn’t going to change midway through the copy, so I’ll always get a complete and consistent text file. This approach has saved me hours when working with large databases, and made my downloads both faster and more reliable. If you have to copy around large SQLite databases, give it a try. [If the formatting of this post looks odd in your feed reader, visit the original article]

4 months ago 29 votes
A flash of light in the darkness

I support dark mode on this site, and as part of the dark theme, I have a colour-inverted copy of the default background texture. I like giving my website a subtle bit of texture, which I think makes it stand out from a web which is mostly solid-colour backgrounds. Both my textures are based on the “White Waves” pattern made by Stas Pimenov. I was setting these images as my background with two CSS rules, using the prefers-color-scheme: dark media feature to use the alternate image in dark mode: body { background: url('https://alexwlchan.net/theme/white-waves-transparent.png'); } @media (prefers-color-scheme: dark) { body { background: url('https://alexwlchan.net/theme/black-waves-transparent.png'); } } This works, mostly. But I prefer light mode, so while I wrote this CSS and I do some brief testing whenever I make changes, I’m not using the site in dark mode. I know how dark mode works in my local development environment, not how it feels as a day-to-day user. Late last night I was using my phone in dark mode to avoid waking the other people in the house, and I opened my site. I saw a brief flash of white, and then the dark background texture appeared. That flash of bright white is precisely what you don’t want when you’re using dark mode, but it happened anyway. I made a note to work it out in the morning, then I went to bed. Now I’m fully awake, it’s obvious what happened. Because my only background is the image URL, there’s a brief gap between the CSS being parsed and the background image being loaded. In that time, the browser doesn’t have anything to put in the background, so you just get pure white. This was briefly annoying in the moment, but it would be even more worse if the background texture never loaded. I have light text on black in dark mode, but without the background image it’s just light text on white, which is barely readable: I never noticed this in local development, because I’m usually working in a well-lit room where that white flash would be far less obvious. I’m also using a local version of the site, which loads near-instantly and where the background image is almost certainly saved in my browser cache. I’ve made two changes to prevent this happening again. I’ve added a colour to use as a fallback until the image loads. The CSS background property supports adding a colour, which is used until the image loads, or as a fallback if it doesn’t. I already use this in a few places, and now I’ve added it to my body background. body { background: url('https://…/white-waves-transparent.png') #fafafa; } @media (prefers-color-scheme: dark) { body { background: url('https://…/black-waves-transparent.png') #0d0d0d; } } This avoids the flash of unstyled background before the image loads – the browser will use a solid dark background until it gets the texture. I’ve added rel="preload" elements to the head of the page, so the browser will start loading the background textures faster. These elements are a clue to the browser that these resources are going to be useful when it renders the page, so it should start loading them as soon as possible: <link rel="preload" href="https://alexwlchan.net/theme/white-waves-transparent.png" as="image" type="image/png" media="(prefers-color-scheme: light)" /> <link rel="preload" href="https://alexwlchan.net/theme/black-waves-transparent.png" as="image" type="image/png" media="(prefers-color-scheme: dark)" /> This means the browser is downloading the appropriate texture at the same time as it’s downloading the CSS file. Previously it had to download the CSS file, parse it, and only then would it know to start downloading the texture. With the preload, it’s a bit faster! The difference is probably imperceptible if you’re on a fast connection, but it’s a small win and I can’t see any downside (as long as I scope the preload correctly, and don’t preload resources I don’t end up using). I’ve seen a lot of sites using <link rel="preload"> and I’ve only half-understood what it is and why it’s useful – I’m glad to have a chance to use it myself, so I can understand it better. This bug reminds me of a phenomenon called flash of unstyled text. Back when custom fonts were fairly new, you’d often see web pages appear briefly with the default font before custom fonts finished loading. There are well-understood techniques for preventing this, so it’s unusual to see that brief unstyled text on modern web pages – but the same issue is affecting me in dark mode I avoided using custom fonts on the web to avoid tackling this issue, but it got me anyway! In these dark times for the web, old bugs are new again. [If the formatting of this post looks odd in your feed reader, visit the original article]

4 months ago 55 votes
Beyond `None`: actionable error messages for `keyring.get_password()`

I’m a big fan of keyring, a Python module made by Jason R. Coombs for storing secrets in the system keyring. It works on multiple operating systems, and it knows what password store to use for each of them. For example, if you’re using macOS it puts secrets in the Keychain, but if you’re on Windows it uses Credential Locker. The keyring module is a safe and portable way to store passwords, more secure than using a plaintext config file or an environment variable. The same code will work on different platforms, because keyring handles the hard work of choosing which password store to use. It has a straightforward API: the keyring.set_password and keyring.get_password functions will handle a lot of use cases. >>> import keyring >>> keyring.set_password("xkcd", "alexwlchan", "correct-horse-battery-staple") >>> keyring.get_password("xkcd", "alexwlchan") "correct-horse-battery-staple" Although this API is simple, it’s not perfect – I have some frustrations with the get_password function. In a lot of my projects, I’m now using a small function that wraps get_password. What do I find frustrating about keyring.get_password? If you look up a password that isn’t in the system keyring, get_password returns None rather than throwing an exception: >>> print(keyring.get_password("xkcd", "the_invisible_man")) None I can see why this makes sense for the library overall – a non-existent password is very normal, and not exceptional behaviour – but in my projects, None is rarely a usable value. I normally use keyring to retrieve secrets that I need to access protected resources – for example, an API key to call an API that requires authentication. If I can’t get the right secrets, I know I can’t continue. Indeed, continuing often leads to more confusing errors when some other function unexpectedly gets None, rather than a string. For a while, I wrapped get_password in a function that would throw an exception if it couldn’t find the password: def get_required_password(service_name: str, username: str) -> str: """ Get password from the specified service. If a matching password is not found in the system keyring, this function will throw an exception. """ password = keyring.get_password(service_name, username) if password is None: raise RuntimeError(f"Could not retrieve password {(service_name, username)}") return password When I use this function, my code will fail as soon as it fails to retrieve a password, rather than when it tries to use None as the password. This worked well enough for my personal projects, but it wasn’t a great fit for shared projects. I could make sense of the error, but not everyone could do the same. What’s that password meant to be? A good error message explains what’s gone wrong, and gives the reader clear steps for fixing the issue. The error message above is only doing half the job. It tells you what’s gone wrong (it couldn’t get the password) but it doesn’t tell you how to fix it. As I started using this snippet in codebases that I work on with other developers, I got questions when other people hit this error. They could guess that they needed to set a password, but the error message doesn’t explain how, or what password they should be setting. For example, is this a secret they should pick themselves? Is it a password in our shared password vault? Or do they need an API key for a third-party service? If so, where do they find it? I still think my initial error was an improvement over letting None be used in the rest of the codebase, but I realised I could go further. This is my extended wrapper: def get_required_password(service_name: str, username: str, explanation: str) -> str: """ Get password from the specified service. If a matching password is not found in the system keyring, this function will throw an exception and explain to the user how to set the required password. """ password = keyring.get_password(service_name, username) if password is None: raise RuntimeError( "Unable to retrieve required password from the system keyring!\n" "\n" "You need to:\n" "\n" f"1/ Get the password. Here's how: {explanation}\n" "\n" "2/ Save the new password in the system keyring:\n" "\n" f" keyring set {service_name} {username}\n" ) return password The explanation argument allows me to explain what the password is for to a future reader, and what value it should have. That information can often be found in a code comment or in documentation, but putting it in an error message makes it more visible. Here’s one example: get_required_password( "flask_app", "secret_key", explanation=( "Pick a random value, e.g. with\n" "\n" " python3 -c 'import secrets; print(secrets.token_hex())'\n" "\n" "This password is used to securely sign the Flask session cookie. " "See https://flask.palletsprojects.com/en/stable/config/#SECRET_KEY" ), ) If you call this function and there’s no keyring entry for flask_app/secret_key, you get the following error: Unable to retrieve required password from the system keyring! You need to: 1/ Get the password. Here's how: Pick a random value, e.g. with python3 -c 'import secrets; print(secrets.token_hex())' This password is used to securely sign the Flask session cookie. See https://flask.palletsprojects.com/en/stable/config/#SECRET_KEY 2/ Save the new password in the system keyring: keyring set flask_app secret_key It’s longer, but this error message is far more informative. It tells you what’s wrong, how to save a password, and what the password should be. This is based on a real example where the previous error message led to a misunderstanding. A co-worker saw a missing password called “secret key” and thought it referred to a secret key for calling an API, and didn’t realise it was actually for signing Flask session cookies. Now I can write a more informative error message, I can prevent that misunderstanding happening again. (We also renamed the secret, for additional clarity.) It takes time to write this explanation, which will only ever be seen by a handful of people, but I think it’s important. If somebody sees it at all, it’ll be when they’re setting up the project for the first time. I want that setup process to be smooth and straightforward. I don’t use this wrapper in all my code, particularly small or throwaway toys that won’t last long enough for this to be an issue. But in larger codebases that will be used by other developers, and which I expect to last a long time, I use it extensively. Writing a good explanation now can avoid frustration later. [If the formatting of this post looks odd in your feed reader, visit the original article]

4 months ago 45 votes
Localising the `` with JavaScript

I’ve been writing some internal dashboards recently, and one hard part is displaying timestamps. Our server does everything in UTC, but the team is split across four different timezones, so the server timestamps aren’t always easy to read. For most people, it’s harder to understand a UTC timestamp than a timestamp in your local timezone. Did that event happen just now, an hour ago, or much further back? Was that at the beginning of your working day? Or at the end? Then I remembered that I tried to solve this five years ago at a previous job. I wrote a JavaScript snippet that converts UTC timestamps into human-friendly text. It displays times in your local time zone, and adds a short suffix if the time happened recently. For example: today @ 12:00 BST (1 hour ago) In my old project, I was using writing timestamps in a <div> and I had to opt into the human-readable text for every date on the page. It worked, but it was a bit fiddly. Doing it again, I thought of a more elegant solution. HTML has a <time> element for expressing datetimes, which is a more meaningful wrapper than a <div>. When I render the dashboard on the server, I don’t know the user’s timezone, so I include the UTC timestamp in the page like so: <time datetime="2025-04-15 19:45:00Z"> Tue, 15 Apr 2025 at 19:45 UTC </time> I put a machine-readable date and time string with a timezone offset string in the datetime attribute, and then a more human-readable string in the text of the element. Then I add this JavaScript snippet to the page: window.addEventListener("DOMContentLoaded", function() { document.querySelectorAll("time").forEach(function(timeElem) { // Set the `title` attribute to the original text, so a user // can hover over a timestamp to see the UTC time. timeElem.setAttribute("title", timeElem.innerText); // Replace the display text with a human-friendly date string // which is localised to the user's timezone. timeElem.innerText = getHumanFriendlyDateString( timeElem.getAttribute("datetime") ); }) }); This updates any <time> element on the page to use a human friendly date string, which is localised to the user’s timezone. For example, I’m in the UK so that becomes: <time datetime="2025-04-15 19:45:00Z" title="Tue, 15 Apr 2025 at 19:45 UTC"> Tue, 15 Apr 2025 at 20:45 BST </time> In my experience, these timestamps are easier and more intuitive for people to read. I always include a timezone string (e.g. BST, EST, PDT) so it’s obvious that I’m showing a localised timestamp. If you really need the UTC timestamp, it’s in the title attribute, so you can see it by hovering over it. (Sorry, mouseless users, but I don’t think any of my team are browsing our dashboards from their phone or tablet.) If the JavaScript doesn’t load, you see the plain old UTC timestamp. It’s not ideal, but the page still loads and you can see all the information – this behaviour is an enhancement, not an essential. To me, this is the unfulfilled promise of the <time> element. In my fantasy world, web page authors would write the time in a machine-readable format, and browsers would show it in a way that makes sense for the reader. They’d take into account their language, locale, and time zone. I understand why that hasn’t happened – it’s much easier said than done. You need so much context to know what’s the “right” thing to do when dealing with datetimes, and guessing without that context is at the heart of many datetime bugs. These sort of human-friendly, localised timestamps are very handy sometimes, and a complete mess at other times. In my staff-only dashboards, I have that context. I know what these timestamps mean, who’s going to be reading them, and I think they’re a helpful addition that makes the data easier to read. [If the formatting of this post looks odd in your feed reader, visit the original article]

4 months ago 64 votes

More in programming

chapter two

I had watched enough true crime to know that you should never talk to the police. And I wasn’t arrogant enough to believe that I was different. While I felt like I knew the interrogation tactics in and out, they were repeat customers of that interaction. I wasn’t going to call. I was going to ignore it. I’m not getting Reid techniqued. Why did they ask for me? This house was owned by my mother, how do they even know I live here? Wait who am I kidding, of course they know. I went to high school here, governments have records of that kind of thing. But still, why ask for me? Another thing was odd. We lived in Brooklyn, aka Kings County. Not Nassau County. These guys must have driven all the way here on a Saturday night. I felt like I was being watched. They wouldn’t drive all the way here to just leave a business card. I felt trapped in the house. Like they were a mountain lion on a rock perch and I was the prey in the valley below. They had the high ground and I didn’t know what they could see. But this was crazy, I didn’t do anything! Should I call them? Figure out what they want? No! That’s exactly what they want. They know I feel like this. This is exactly what they are going for. Another system carefully crafted based on years and years of “user feedback” designed to manipulate you into doing what it wants. But what if I’m doing what they want right now? Maybe they don’t want me to call. Maybe the real goal is to figure out what I do next. Watching and hoping I’ll go check on the body or something. But there wasn’t a body! If I did commit a crime this would all be a lot easier, I’d know why they were here and what they wanted and could plan my next move accordingly. I opened another Bud Light, took my clothes off, and got into bed. Even though there was nobody else home, I kept the sound off on the porn. Just in case they were listening. After I finished, I felt a bit more calm. Dude get a grip, all they did was leave a business card. Coming out of the paranoid spiral a bit, I realized what it must be about. It must have had to do with my Dad’s meeting. That was in Long Island, aka Nassau County. Probably some dumb financial crap. My mother was out with her friends in Manhattan, but she’d be home tonight and maybe she knew what the meeting was. It was now twenty to nine and I texted Brian. He’s like yea bro Dave just got here come through. And you still have that case of Bud Light? I put the beers in a backpack. Is this what the detective planned? Maybe I was playing right into the plot; arrest me for underage possession of alcohol and then get me to talk about what I knew. But I didn’t even know anything! This whole thing was stupid. I thought about how I got the beers, wondering if the whole thing was somehow a set-up. Totally nonsense thought. Kids buy beer with fake IDs all the time. When I got to Brian’s everything was normal. I walked around the back of his house and opened the screen door to his basement. There were three leather couches in a U-shape, two of which were sparsely occupied by Brian and Dave. I took my place on the third empty one and put my backpack on the center ottoman. “Pretty cool, right? Yea I found it in my Dad’s old stuff.” said Brian, referring to the inflated bag atop a device labeled Volcano sharing the ottoman with my backpack. “What is it?” “Bro it’s like an old vape. You put the weed in and plug it in to the wall.” He detached the cloudy bag from the device and demonstrated. If you pushed on the mouthpiece, it let air through and you could breathe in the vaporized drug. “It’s like a bong but chill.” I inhaled. This probably wasn’t smart with how paranoid I was from the interaction earlier, but I felt safe in the basement. It was a summer night, I was with friends, I had drank beer. Life was good. Dave showed us this reel. It was a mouse in a maze, and it started from the mouse’s perspective. Kind of like a skater cam, wow these things could scurry. Then it zoomed out so you could see the maze from the perspective of the experimenter. Then seeing the back of his head looking down at the maze, cutting to sped up dashcam video of him driving home from work. Zooming out again with a sparkling line showing his route through the grid of city streets. AI has done wonders for these video transitions. Maybe this whole video was AI. “What if we’re the mouse,” said Dave in the most stereotypical stoner voice. He’d always find shit like this, in that way that when you are high the thought seems really deep. But if you think about it more it’s nonsense, like that mouse is in a maze constructed by humans, and even if it doesn’t always feel like it, the society we live in is jointly constructed by all of us. Brian showed a video of two girls at some Mardi Gras bead type event licking one ice cream cone. He told us he wasn’t a virgin but I didn’t really believe him. It was a bit after midnight and it was time to go home. I hadn’t really thought about the interaction from earlier, but I started to again when I got outside. It was a half mile walk back home; I was grateful to hear all the noises of the city. Even though I couldn’t see it, it reminded me that there was a society out there. My mom’s car wasn’t in the driveway. Maybe she met a guy. Nothing too out of the ordinary. I unlocked the door, closed it behind me, locked both the knob and the deadbolt, went upstairs into my room, locked that door, and with the blanket of those three locks, a bunch of beers, and a couple hits of the Volcano, drifted off to sleep.

17 hours ago 2 votes
Dreams of Late Summer

Here on a summer night in the grass and lilac smell Drunk on the crickets and the starry sky, Oh what fine stories we could tell With this moonlight to tell them by. A summer night, and you, and paradise, So lovely and so filled with grace, Above your head, the universe has hung its … Continue reading Dreams of Late Summer →

2 days ago 7 votes
Thrice charmed at Rails World

The first Rails World in Amsterdam was a roaring success back in 2023. Tickets sold out in 45 minutes, the atmosphere was electric, and The Rails Foundation set a new standard for conference execution in the Ruby community. So when we decided to return to the Dutch Capital for the third edition of the conference this year, the expectations were towering. And yet, Amanda Perino, our executive director and event organizer extraordinaire, managed to outdo herself, and produced an even better show this year.  The venue we returned to was already at capacity the first time around, but Amanda managed to fit a third more attendees by literally using slimmer chairs! And I didn't hear any complaints the folks who had to sit a little closer together in order for more people to enjoy the gathering. The increased capacity didn't come close to satisfy the increased demand, though. This year, tickets sold out in less than two minutes. Crazy. But for the 800+ people who managed to secure a pass, I'm sure it felt worth the refresh-the-website scramble to buy a ticket.   And, as in years past, Amanda's recording crew managed to turn around post-production on my keynote in less than 24 hours, so anyone disappointed with missing out on a ticket could at least be in the loop on all the awesome new Rails stuff we were releasing up to and during the conference. Every other session was recorded too, and will soon be on the Rails YouTube channel. You can't stream the atmosphere, the enthusiasm, and the genuine love of Ruby on Rails, though. I was once again blown away by just how many incredible people and stories we have in this ecosystem. From entrepreneurs who've built million (or billion!) dollar businesses on Rails, to programmers who've been around the framework for decades, to people who just picked it up this year. It was a thrill to meet all of them, to take hundreds of selfies, and to talk about Ruby, Rails, and the Omarchy expansion pack for hours on the hallway track! I've basically stopped doing prepared presentations at conferences, but Rails World is the one exception. I really try my best to put on a good show, present the highlights of what we've been working on in the past year at 37signals, and transfer the never-ending enthusiasm I continue to feel for this framework, this programming language, and this ecosystem.  True, I may occasionally curse that commitment in the weeks leading up to the conference, but the responsibility is always rewarded during and after the execution with a deep sense of satisfaction. Not everyone is so lucky as I've been to find their life's work early in their career, and see it continue to blossom over the decades. I'm eternally grateful that I have. Of course, there's been ups and downs over the years — nothing is ever just a straight line of excitement up and to the right! — but we're oh-so-clearly on the up-up-up part of that curve at the moment. I don't know whether it's just the wind or the whims, but Rails is enjoying an influx of a new generation of programmers at the moment. No doubt it helps when I get to wax poetically about Ruby for an hour with Lex Fridman in front of an audience of millions. No doubt Shopify's continued success eating the world of ecommerce helps. No doubt the stability, professionalism, and execution from The Rails Foundation is an aid. There are many auxiliary reasons why we're riding a wave at the moment, but key to it all is also that Ruby on Rails is simply really, really good! Next year, with RailsConf finished, it's time to return to the US. Amanda has picked a great spot in Austin, we're planning to dramatically expand the capacity, but I also fully expect that demand will continue to rise, especially in the most prosperous and successful market for Rails. Thanks again to all The Rails Foundation members who believed in the vision for a new institution back in 2022. It looks like a no-brainer to join such a venture now, given the success of Rails World and everything else, but it actually took guts to sign on back then. I approached quite a few companies at that time who could see the value, but couldn't find the courage to support our work, as our industry was still held hostage to a band of bad ideas and terrible ideologies. All that nonsense is thankfully now long gone in the Rails world. We're enjoying a period of peak unity, excitement, progress, and determination to continue to push for end-to-end problem solving, open source, and freedom. I can't tell you how happy it makes me feel when I hear from yet another programmer who credits Ruby on Rails with finding joy and beauty in the writing web applications because of what I started over 22 years ago. It may sound trite, but it's true: It's an honor and a privilege. I hope to carry this meaningful burden for as long as my intellectual legs still let me stand. See you next year in Austin? I hope so!

2 days ago 6 votes
chapter one

I hadn’t lost my virginity yet. And it wasn’t for lack of trying; it seemed like the rest of my generation was no longer interested in sex. On some level, I understood where they were coming from, the whole act did seem kind of pointless. But after a few beers, that wasn’t how my mind was working. I turned 19 last week. Dad flew in from Idaho, and it was the first time he was in the house I shared with my mother. He left when I was 12, and it was always apparent that parenting wasn’t the top thing on his mind. There was some meeting on Long Island. That’s probably why he was there, in addition to the fact he knew mom wouldn’t make him sleep on the couch. He had many reasons to be in New York that weren’t me. My birthday was just a flimsy pretense. He’d worked on Wall Street the whole time he was around, a quant. He wrote programs that made other people rich. But something happened to him right before he left. A crisis of conscience perhaps; he was spiraling for weeks, cursing the capitalist system, calling my mother a gold-digging whore (which was mostly true), and saying things needed to change. Then he packed a single backpack and left for Idaho. I visited him out there once my sophomore year. He had a camouflaged one room cabin in the middle of a spruce forest, but instead of the hunting or fishing stuff you might expect, the walls were adorned with electrical test equipment and various things that looked like they were out of a biology or chemistry lab. I didn’t know much about this stuff and that wasn’t what he wanted to talk about anyway. He wanted to talk about “man shit” like nature and women and not being life’s bitch. I tried to act like I did, but I didn’t really listen. All I remember is how eerily quiet the night was, I could hear every animal movement outside. My dad said you get used to it. Brian was having a party tonight. Well okay, party is a lofty way to describe it. He’d replaced the fluorescent lights in his mom’s basement with blacklights, and we’d go over there to drink beer and smoke weed and sit around on our phones and scroll. And sometimes someone would laugh at something and share with the group. I had a case of Bud Light left over from the last party and drank two of them today. Hence the thinking about sex and not thinking that thinking about sex was stupid. People wouldn’t be going over there for a few more hours, so I laid in my bed, drank, and loosely beat off to YouTube. Celebrity gossip, internet gossip, speedrun videos, nothing even arousing. I liked the true crime videos about the hot female teachers who slept with their students. Yea yea yea terrible crime and they all act holier than thou about what if the genders were reversed, but the genders weren’t reversed. Maybe they just don’t want to get demonetized. There were never women at these parties. Okay maybe one or two. But nobody ever slept with them or much thought about them that way. They were the agendered mass like the rest of us. Fellow consumers, not providers. Fuck I should just go visit a hooker. I didn’t know much about that, were hookers real? I’d never met one, and there wasn’t a good way to find out about stuff like this anymore. The Internet was pretty much all “advertiser friendly” now, declawed, sanitized. Once the algorithms got good enough and it was technically easy to censor, there was nothing holding them back. It wasn’t actually censored, it would just redirect you elsewhere. And if you didn’t pay careful attention, you wouldn’t even notice it happening. I tried asking ChatGPT about hookers and it told me to call them sex workers. And this was kind of triggering. Who the fuck does this machine think it is? But then I was lost on this tangent, the algorithms got a rise out of me and I went back to comfort food YouTube. Look this guy beat Minecraft starting with only one block. The doorbell rang. This always gives me anxiety. And it was particularly anxiety inducing since I was the only one home. Normally I could just know that the door of my room was locked and someone else would get it and this would be a downstairs issue. But it was just me at home. My heart rate jumped. I waited for it to ring again, but prayed that it wouldn’t. Please just go away. But sure enough, it rang again. I went to my window, my room was on the second floor. There was a black Escalade in the driveway that I hadn’t seen before, and I could see two men at the door. They were wearing suits. I ducked as to make sure they wouldn’t look up at me, making as little noise as possible. Peering over the window sill I could see one opening the screen door, and it looked like he stuck something to the main door. My heart was beating even faster now. It was Saturday night, why were there two men in suits? And why were they here? It felt longer, but 3 minutes later they drove off. I waited another 3 for good measure, just watching the clock on my computer until it hit 6:57. I doubled checked out the window to make sure they were actually gone, and crept down the stairs to retrieve whatever they left on the door. It was a business card, belonging to a “Detective James Reese” of the Nassau County Police. And on the back of the card, there was handwriting. “John – call me” John was my name.

2 days ago 3 votes
Apologies and forgiveness

The first in a series of posts about doing things the right way

3 days ago 11 votes