One of the most useful and janky internal tools we have in Buttondown’s codebase is a codegen pipeline called “autogen”. There is nothing “auto” about autogen: it is a series of scripts that munges a bunch of data into a bunch of different formats, to generate things like our API clients and code snippets and storybooks. Some of this data is stateful, and therefore requires a database, and therefore requires migrations — you see how this kind of thing can grow somewhat labrynthine. Each individual script is pretty simple, but as we’ve found more and more things to glom onto autogen. This, to be clear, is a good thing. It’s really nice to have automatic, consistent data and types everywhere, so that we literally cannot change the API without also pushing a concomitant change to the API docs. With each glom, though, the wall-clock time of running autogen increases — and so I found myself staring down the barrel at a 50second script running whenever we wanted to make any sort of non-trivial change to our schema. Fifty seconds was too many seconds. I set a budget of ten seconds — still a long time, but significantly less onerous — and began digging in at low-hanging fruit. There was a lot. A few that come to mind: We split up our vite config so we could only run the portion that we needed (cross-piling and minifying our CSS bundles; We disabled all Sentry and perf-tracing stuff that was getting enabled as part of the standard build; We no-oped all of the Python-land data generation if it was already there, since that stateful data didn’t change very often. This was all great, but we were still left with 15 seconds of wall clock time. Profiling each individual cog in the script revealed that the problem was essentially “it’s Python”: four items in the script ran Django commands, and just spinning up the Django process and running autodiscovery took around two seconds. Ouch! The impulse was to cut down that runtime. A great post by Adam led us to discover the biggest culprit was our Stripe imports, and we timeboxed a bit of time to try and get rid of them, either by deferring the imports or excising the library; neither seemed particularly feasible. Then, suddenly, the answer seemed obvious. If we have four scripts where the fixed cost of invoking Django is the long pole, why not simply combine the scripts? And that’s exactly what we did: if len(sys.argv) > 1 and "," in sys.argv[1]: commands = sys.argv[1].split(",") original_argv = sys.argv.copy() for command in commands: sys.argv[1] = command execute_from_command_line(sys.argv) sys.argv = original_argv else: execute_from_command_line(sys.argv)

Here is a confession: I am a very strong proponent of a robust test suite being perhaps the single most important asset of a codebase, but when it comes to auxiliary services like admin sites or CLIs when it comes to testing I tend to ask for forgiveness more than I ask for permission. Django's admin site is no different: and, because Django's admin DSL is very magic-string-y, there's a lot of stuff that never gets caught by CI or mypy until a lovely CS agent informs me that something is blowing up in their face. Take this example, which bites me more often than I care to admit: from django.contrib import admin from stripe.models import StripeCustomer class StripeCustomer(models.Model): id = models.CharField(max_length=100, unique=True) username = models.CharField(max_length=100, unique=True) email_address = models.EmailField() creation_date = models.DateTimeField(auto_now=True) @admin.register(StripeCustomer) class StripeCustomerAdmin(admin.ModelAdmin): list_display = ( "id", "username", "email", "creation_date", ) search_fields = ( "username", "email", ) One thing that has made my life slightly easier in this respect is a parametric test that just makes sure we can render the empty state for every single admin view. Code snippet first, explanation after: from django.urls import get_resolver, reverse def extract_routes(resolver: URLResolver) -> iter[str]: keys = [key for key in resolver.reverse_dict.keys() if isinstance(key, str)] key_to_route = {key: resolver.reverse_dict.getlist(key)[0][0][0] for key in keys} for key in keys: yield key for key, (prefix, subresolver) in resolver.namespace_dict.items(): for route in extract_routes(subresolver): yield f"{key}:{route.name}" def is_django_admin_route(route_name: str): # Matches, e.g., `admin:emails_event_changelist`. return route_name.split(":").endswith("changelist") ADMIN_URL_ROUTE = "buttondown.urls.admin" DJANGO_ADMIN_CHANGELIST_ROUTES = [ route.name for route in extract_routes(get_resolver(ADMIN_URL_ROUTE)) if is_django_admin_route(route.name) ] # The fixture is overkill for this example, but I'm copying this from the actual codebase. @pytest.fixture def superuser_client(superuser: User, client: Client) -> Client: client.force_login(superuser) return client @pytest.mark.parametrize( "url", DJANGO_ADMIN_CHANGELIST_ROUTES ) def test_can_render_route(superuser_client: Any, url: str) -> None: url = reverse(url, args=[]) response = superuser_client.get(url) assert response.status_code == 200 Okay, a bit of a mouthful, but the final test itself is very clean and tidy and catches a lot of stuff. That extract_routes implementation looks scary and magical, and it is — I use a more robust implementation in django-typescript-routes, which itself we gratefully purloined from django-js-reverse. Lots of scary indexing, but its held up well for a while. The fixture and parametrize assumes usage of pytest (you should use pytest!) but it's trivially rewritable to use subTest instead.

When we added support for complex filtering in Buttondown, I spent a long time trying to come up with a schema for filters that felt sufficiently ergonomic and future-proof. I had a few constraints, all of which were reasonable: It needed to be JSON-serializable, and trivially parsable by both the front-end and back-end. It needed to be arbitrarily extendible across a number of domains (you could filter subscribers, but also you might want to filter emails or other models.) It needed to be able to handle both and and or logic (folks tagged foo and bar as well as folded tagged foo or bar). It needed to handle nested logic (folks tagged foo and folks tagged bar or baz.) The solution I landed upon is not, I’m sure, a novel one, but googling “recursive filter schema” was unsuccessful and I am really happy with the result so here it is in case you need something like this: @dataclass class FilterGroup: filters: list[Filter] groups: list[FilterGroup] predicate: "and" | "or" @dataclass class Filter: field: str operator: "less_than" | "greater_than" | "equals" | "not_equals" | "contains" | "not_contains" value: str And there you have it. Simple, easily serializable/type-safe, can handle everything you throw at it. For example, a filter for all folks younger than 18 or older than 60 and retired: FilterGroup( predicate="or", filters=[ Field( field="age", operator="less_than", value="18" ) ], groups=[ FilterGroup( predicate="and", filters=[ Field( field="age", operator="greater_than", value="60" ), Field( field="status", operator="equals", value="retired" ) ] groups=[], ) ] )

If there's been one through line in changes to Buttondown's architecture over the past six months or so, it's been the removal and consolidation of dependencies: on the front-end, back-end, and in paid services. I built our own very spartan version of Metabase, Notion, and Storybook; we vended a half-dozen or so Django packages that were not worth the overhead of pulling from PyPI (and rewrote another half-dozen or so, which we will open-source in due time); we ripped out c3, our visualization library, and built our own; we ripped out vuedraggable and a headlessui and a slew more of otherwise-underwhelming frontend packages in favor of purpose-built (faster, smaller, less-flexible) versions. [1] There are a few reasons for this: Both Buttondown as an application and I as a developer have now been around long enough to be scarred by big ecosystem changes. Python has gone through both the 2.x to 3.x transition and, more recently, the untyped to typed transition; Vue has gone from 2.x to 3.x. The academic problem of "what happens if this language completely changes?" is no longer academic, and packages that we installed back in 2018 slowly succumbed to bitrot. It's more obvious to me now than a few years ago that pulling in dependencies incurs a non-trivial learning cost for folks paratrooping into the codebase. A wrapper library around fetch might be marginally easier to invoke once you get used to it, but it's a meaningful bump in the learning curve to adapt to it for the first time. It is easier than ever to build 60% of a tool, which is problematic in many respects but useful if you know exactly which 60% you care about. (Internal tools like Storybook or Metabase are great examples of this. It was a fun and trivial exercise to get Claude to build a tool that did everything I wanted Metabase to do, and save me $120/mo in the process.) We still use a lot of very heavy, very complex stuff that we're very happy with. Our editor sits on top of tiptap (and therefore ProseMirror); we use marked and turndown liberally, because they're fast and robust. On the Python side, our number of non-infrastructural packages is smaller but still meaningful (beautifulsoup, for instance, and django-allauth / django-anymail which are both worth their weight in gold). But the bar for pulling in a small dependency is much higher than it was, say, twelve months ago. My current white whale is to finally get rid of axios. 39 call sites to go! ↩︎