I have a lot of digital cross-stitch patterns and until recently they were a mess. My “collection” was a series of several hundred PDFs, disorganised and spread across my disk. Recently, I was looking for a pattern and it took me several minutes to find it. I had a brief moment of panic, thinking that I’d lost it, and the website this pattern came from is long since gone. Fortunately it did turn up, but it was clear I had to organise my pattern collection. Some people would use apps like Pattern Keeper or Flosshaven, but I just created another static website on my media drive. This is what it looks like: The patterns are arranged in a masonry layout with a preview thumbnail for each pattern, which is normally the first page of the PDF. If a pattern has multiple files – for example, different files for colour and black-and-white printing – there’s a list of links below the thumbnail. I can also add a source link to where I bought the pattern, if I want to find more patterns by the same author. Internally, it’s structured like my other static websites. The metadata for the patterns is stored in a JavaScript file, and then I have an HTML page which renders that metadata in this visual design. My pattern collection doesn’t change very often, so I’m adding new files by hand. There are no sorting or filtering options, because my collection is small enough that they’re unnecessary. I can scroll through the entire list in less than a minute and find whatever I need. I’d like to start making my own clothes soon, and I expect to copy this design to store clothing patterns. This collection now sits alongside my saved bookmarks, podcasts, TV shows, and more. I can keep all my patterns organised, and I won’t lose them in the depths of my hard drive. Technical tricks There are a couple of technical pieces I’m quite pleased with in this process: Masonry layout. I love the cascading grid style of a masonry layout, and I think they’re great when you have variable-sized content – like my pattern cards. I’ve been using them for over a decade; screenshots of my early web apps show my experiments with David Desandro’s Masonry.js library. Masonry layouts are now a native feature of CSS called “Grid Lanes”, which is simpler and more performant than the JavaScript approach. I’m excited to use these layouts in even more places. Thumbnails of PDFs. Most digital postage patterns are shipped as PDFs, and I’m creating the thumbnail images with my create_thumbnail tool. I had to extend my tool to support PDFs, which turned out to be straightforward with the pdf_oxide crate. I’d not come across Oxide before, but it’s a cross-language library for working with PDFs, and I can imagine using it in future. A coloured stripe for the palette. In the metadata for each project, I have the complete list of DMC thread colours and the pattern dimensions. I have a lookup table that converts DMC thread colours to hex codes, and then I’m able to create a linear gradient in CSS to render a coloured stripe of the pattern colours. Here’s an example from a pattern with four colours: border-bottom: 12px solid transparent; border-image: linear-gradient( to right, #7a451f 0%, #7a451f 25%, #985e33 25%, #985e33 50%, #e4bb8e 50%, #e4bb8e 75%, #361f0e 75%, #361f0e 100% ) 1; Building the HTML, CSS, and JavaScript was very quick – I have notes on how I build my media sites, and I’ve done it plenty of times before. I threw together the initial prototype in less than 15 minutes. The longer part was finding all the PDFs scattered across my disk – but now that’s done, and I know exactly where to find my patterns. [If the formatting of this post looks odd in your feed reader, visit the original article]

I wrote a post for the Tailscale blog about a long-running series of corruption incidents, and how they eventually led us to find an SQLite bug that predates my entire programming career. I’m incredibly proud of this, both the work and the blog post. Before Tailscale, I was coming from smaller teams where I didn’t get to tackle problems of this scale or complexity. This was exactly the sort of tricky, deep technical challenge I wanted to be part of (though I’d rather it hadn’t been quite so stressful)! I’m glad I got to play a small part in these incidents, and I learnt so much from the more experienced engineers I worked with. I never want to hear the words “SQLite corruption” again, but if I do, I’d want to have Tailscalars at my side. Writing the blog post has a blast, too. The piece transformed from a rough draft into a solid, engaging piece of writing, thanks to thoughtful feedback from many people at Tailscale. Most of my writing is self-edited, and it’s always a pleasure to work with a dedicated editor. Please check out the blog post if you haven’t read it already – I think it’s a fascinating technical story, and one readers of this site are bound to enjoy. [If the formatting of this post looks odd in your feed reader, visit the original article]

Yesterday at work, a customer spotted a typo in our UI: “you can use the use the Tailscale CLI”. After the typo was fixed, I wanted to find other cases of accidentally repeated words or phrases. I used two regular expressions to search every codebase for unnecessary repetition. The first regex finds repeated words: \b([A-Za-z]+) \1\b Backfill product data from from Stripe Learn more about about inviting users Argument must be be one of host name, IP set name, IP prefix, or IP There’s a capturing group for a single word made up of letters ([A-Za-z]+), a space, then a backreference to the group. That expression is surrounded by word boundary assertions \b, which check that I’m at the start/end of a word – this avoids finding repeated character sequyences that within longer words, like “with the reason”. The second regex finds repeated phrases: \b([A-Za-z]+ [A-Za-z]+) \1\b Follow the steps in the in the "How to" section Log in to in to your account To configure federated identities federated identities using the Go SDK I’ve changed the capturing group, so now it looks for two words separated by a space. Sometimes repetition is useful, like when I really really went to emphasise a point, but often it’s just a typo. Cleaning up these mistakes has been a fun Friday cleanup task. [If the formatting of this post looks odd in your feed reader, visit the original article]

A month ago, I wrote about my Playwright fixture for testing static websites in a browser. I’ve been copying that fixture from project-to-project, but recently I decided to add it to chives, the utility library I use for all my static websites (or tiny archives). One of my rules for chives is that everything in it has to be tested – but how do you test a pytest fixture? Test code is just code, and it isn’t immune to bugs. Who tests the tests? Enter Pytester, a tool designed for testing pytest plugins. Pytester allows you to run isolated test suites, make assertions about the outcomes, and verify the behaviour of custom fixtures. In your top-level test suite, you always want everything to be passing, but with Pytester you can write a mixture of passing and failing tests, and check the results are what you expect. Pytester is disabled by default, so you first enable it in your top-level conftest.py file (the pytest configuration file where you configure plugins and fixtures): # conftest.py pytest_plugins = ["pytester"] Here’s an example of using Pytester where we create a test suite with two tests and check that one passes, one fails: from pytest import Pytester def test_with_pytester(pytester: Pytester): """ Run an isolated test suite with pytester. """ # Make a temporary pytest test file pytester.makepyfile( """ def test_arithmetic(): assert 2 + 2 == 4 def test_list_inclusion(): assert "yellow" in ["red", "green", "blue"] """ ) # Run the isolated test suite with pytest result = pytester.runpytest() # Check that one test passed, one failed result.assert_outcomes(passed=1, failed=1) I can imagine creating something similar with some complicated collection of nested functions, exec() and pytest.raises, but using Pytester is a cleaner interface than what I’d build. Under the hood, Pytester creates a temporary directory, writes specified files into it, then runs a fresh pytest subprocess against it. It has helper functions for writing files, including Python files (makepyfile), a conftest.py file (makeconftest), and plain text files (maketxtfile). When we’re testing a fixture, we can create a conftest.py file that imports that fixture, then reference it in the tests. Here’s a more complicated example, where we import one of my Playwright fixtures in my conftest.py, write an HTML file into the temporary directory, then use them both in the test: from pytest import Pytester def test_browser_fixture(pytester: Pytester): """ Try testing the browser fixture with pytester. """ # Make a conftest.py file pytester.makeconftest(""" from chives.browser_fixtures import browser """) # Make an HTML file (pytester.path / "greeting.html").write_text(""" <p>Hello world!</p> """) # Make a temporary pytest test file pytester.makepyfile( """ from chives.browser_fixtures import file_uri from playwright.sync_api import Browser, expect def test_browser_fixture(browser: Browser) -> None: uri = file_uri("greeting.html") p = browser.new_page() p.goto(uri) expect(p.get_by_text("Hello world!")).to_be_visible() """ ) # Run the isolated test suite with pytest result = pytester.runpytest() # Check that one test passed result.assert_outcomes(passed=1) This pattern is sufficient for many fixtures, but it doesn’t work for Playwright – if you run this test, the isolated test suite gives an error rather than a passing test. Playwright needs you to install a web browser to work (for example, playwright install webkit), and Pytester runs in a sufficiently isolated environment that Playwright can’t find the browsers you already have installed. We could run the install command inside the temporary directory, but that would be slow and inefficient – it would be better if we could tell Playwright to look for the already-installed browsers elsewhere. If we set the PLAYWRIGHT_BROWSERS_PATH environment variable inside our isolated test suite, Playwright will look there for browsers. First, we need to work out where browsers are installed – we could hard-code the location, or we could inspect the executable_path property property on a browser: from pathlib import Path from playwright.sync_api import sync_playwright import pytest @pytest.fixture(scope="session") def playwright_browsers_path() -> str: """ Return the cache directory where Playwright browsers are installed. """ with sync_playwright() as p: # In my local builds, this returns a path like: # # ~/Library/Caches/ms-playwright/webkit-2272/pw_run.sh # # Unwrap two levels to get to the `ms-playwright` folder. return str(Path(p.webkit.executable_path).parent.parent) Then we need to set this as an environment variable inside the Pytester test suite. I couldn’t find an easy way to set an environment variable; the best approach I came up with was to modify os.environ inside the conftest.py file. (Perhaps we could access the MonkeyPatch object and set more environment variables, but using private attributes is icky.) Here’s how the new test starts: def test_browser_fixture(pytester: Pytester, playwright_browsers_path: str): """ Test the browser fixture with pytester. """ # Make a conftest.py file pytester.makeconftest(f""" from chives.browser_fixtures import browser import os os.environ["PLAYWRIGHT_BROWSERS_PATH"] = {playwright_browsers_path!r} """) ... and now the overall test passes. Here’s the complete code for the new test: test_browser_fixture.py from pathlib import Path from playwright.sync_api import sync_playwright import pytest from pytest import Pytester @pytest.fixture(scope="session") def playwright_browsers_path() -> str: """ Return the cache directory where Playwright browsers are installed. """ with sync_playwright() as p: # In my local builds, this returns a path like: # # ~/Library/Caches/ms-playwright/webkit-2272/pw_run.sh # # Unwrap two levels to get to the `ms-playwright` folder. return str(Path(p.webkit.executable_path).parent.parent) def test_browser_fixture(pytester: Pytester, playwright_browsers_path: str): """ Test the browser fixture with pytester. """ # Make a conftest.py file pytester.makeconftest(f""" from chives.browser_fixtures import browser import os os.environ["PLAYWRIGHT_BROWSERS_PATH"] = {playwright_browsers_path!r} """) # Make an HTML file (pytester.path / "greeting.html").write_text(""" <p>Hello world!</p> """) # Make a temporary pytest test file pytester.makepyfile( """ from chives.browser_fixtures import file_uri from playwright.sync_api import Browser, expect def test_browser_fixture(browser: Browser) -> None: uri = file_uri("greeting.html") p = browser.new_page() p.goto(uri) expect(p.get_by_text("Hello world!")).to_be_visible() """ ) # Run the isolated test suite with pytest result = pytester.runpytest() # Check that one test passed result.assert_outcomes(passed=1) The full test suite is more extensive, and checks that certain scenarios fail or error – will the fixtures spot the mistakes I expect them to? For example, my Page fixture is meant to load a page and fail the test if there are any console warnings or errors; does it actually fail the test correctly? I don’t expect to use Pytester very often, because it’s rare for me to write fixtures complex enough to need their own test suite – but sometimes I do, and it’s good to know how to create another layer of safety net. [If the formatting of this post looks odd in your feed reader, visit the original article]