Earlier this week, I was having a lot of trouble understanding how to get Dev Containers working. Dev Containers are the required way to enable Github Codespaces, which was my actual goal. I finally got to something that works, so I want to document what I learned. Dev Containers is a standard developed my Microsoft for using docker containers to run dev environments for a project. I use Docker for most of my projects these days. I think it’s a great tool both for simplifying local development and for packaging and deploying my projects. In the current project, I already have a Dockerfile and a docker-compose.yml file. I deploy to fly.io which will also run my docker build to deploy the project. This works fine for me, and I had never used Dev Containers. It seemed overly complicated (and I was right about that). I want to give more of the context about why I’ve been pursuing github codespaces, because I think it’s relevant. But if you wanna skip to the tips for getting Dev Containers and Codespaces working, feel free. (I’m also going to talk positively about using LLMs to code, so if that makes you want to skip too, there’s no hard feelings). Background Here’s the idea. I have a solo project that I’m working on, and instead of hacking away on the main branch, I’m using a full feature branch and pull request workflow to keep my changes organized. This might seem like overkill, but it’s helpful for me because I tend to bounce around and leave things half done. I can run things locally and test different things by switching between branches. But I’m also doing a lot more AI-assisted development. Locally I use Claude Code. But I’ve been experimenting with allowing Github Copilot to create PRs. I create a github issue, assign it to copilot, and it spins up in the cloud and does work. Afterwards it opens a pull request for me to review. So even though this is a “solo” project, I end up with pull requests that I didn’t write myself and I need to validate. Giving tasks to copilot in the cloud is cool for a couple of reasons: It’s better than setting an LLM lose on my laptop unsupervised. I’m still working on how to use them safely. When I’m using them locally, I babysit diligently and approve every command it wants to run. But this gets tedious, and that can only lead to cutting corners. With github codespaces, I can make it github’s problem. I get to work on the project even if I can’t sit in front of my computer. When I get excited about a project, I tend to think about it constantly. It’s tough to compartmentalize when I’m having ideas but I’ve got other things to do. The workflow of creating issues and assigning them to copilot helps me move forward even in-between work sessions. I plan to give more of my thoughts on AI-assisted development in a future post. The result is I’ve got PRs from copilot that I need to review. I can look at the diffs, but we know that even if it “looks good to me”,LLMs can and do make subtle mistakes and require extra scrutiny. What I want to do is actually spin up the app with the changes and kick the tires. Github codespaces seemed perfect for this. But it took a lot of trial and error to get to something that works for me. Leaning on docker and docker compose We need to take a slight detour before diving into Dev Containers. We will be using docker and docker compose to help us bypass some of the complexity. So this post assumes that you have a working setup using docker-compose.yml. I’m not going to go into detail about that here. If you’re interested, maybe I can do a future blog post. Your docker compose setup should also include a database if you need one. Most apps like mine require a database to connect to. There are lots of ways you can go about this locally. In fact, if you’ve got a local setup using docker compose, you’ve probably already figured this out. But there will also need to be a database available once we upload to a github codespaces environment. So for that purpose, this post also assumes that you have a database container alongside your app container. It usually looks something like this for me (stripped down for brevity). services: app: image: my-app container_name: my-app build: context: . dockerfile: Dockerfile ports: - "3000:3000" env_file: - .env postgres: image: postgres:17-alpine container_name: my-app-db environment: POSTGRES_DB: my_app POSTGRES_USER: postgres POSTGRES_PASSWORD: password ports: - "5432:5432" volumes: - ./backend/data/db:/var/lib/postgresql/data Note: There are a lot of other details to configuring a non-trivial docker compose environment. That’s out of the scope of this post. Sorry. This creates an app that uses my project files to build a docker image. The database is pulled from an existing docker image that runs postgres. The resulting app can reference postgres as the hostname of the database. Docker automatically creates an internal network where the two containers can see each other. Because we’re going to have Dev Containers read our docker-compose.yml from the start, this should work as expected. Getting a Dev Container working This walkthrough assumes you have a few prerequisites already installed: vs code Dev Containers extension for vs code Docker Note: The Dev Containers extension uses your local docker system to build and run containers. If you run into early issues, check that docker is running and that vs code has permission to access it. Reading the docs for Dev Containers seemed straightforward enough. I used the vs code worflow to create a devcontainer.json file. But nothing worked out of the box. It came with it’s own docker-compose.yml file. It wanted me to use a standard Microsoft docker image for the environment rather than the one specified in my own Dockerfile. I’m pretty sure we’re mostly expected to run their pre-built templates for dev containers. You pick the one that seems to match your project most closely, and it’ll probably work by just injecting your code into it. But if you want to be more opinionated about your build, you’re gonna have a bad time. Eventually, I abandoned trying to figure this out and instead configured the dev container to just use the existing docker setup I aleady had. Open your command palette and select “Dev Containers: Add Dev Container Configuration Files…”. You might run through a couple of options like whether to add the dev container files to the workspace or the user config. I chose workspace. Then you’ll be prompted to choose what approach you want to take to create the dev container configuration. You’ll see the option to create it from a predefined template. But what you want to do is select the From 'Dockerfile' option or the From 'docker-compose.yml' option. The trick is that these options only appear if these files are already present in your project. When I tried to recreate what I did for this blog post, I got tripped up by that. If you wanna get started with this before you tackle the docker stuff, you can just touch Dockerfile docker-compose.yml in your project root and that’s enough to enable the menu items. However, the setup may error at some point if there’s nothing in the files. Creating a Dev Container config through vs code The second confusing thing is it asks you to “chose a service”. There is very little context here for what service to choose and why. You’ll see this prompt, because you have to pick one of the services to be the “primary” one for the dev container. Say your docker compose file specifies multple containers, like a web app, a database, and maybe a background worker. You probably want to choose whichever one is the main app. So for example, I selected app, which is the name in my docker-compose.yml that runs the node server. When you run through these setup prompts, you end up with a devcontainer.json that is pretty different from the one you see from the standard tutorials. Instead of having a bunch of setup for your project in the config file, it just references your docker-compose.yml file. // For format details, see https://aka.ms/devcontainer.json. For config options, see the // README at: https://github.com/devcontainers/templates/tree/main/src/docker-existing-docker-compose { "name": "Project name", // Update the 'dockerComposeFile' list if you have more compose files or use different names. // The .devcontainer/docker-compose.yml file contains any overrides you need/want to make. "dockerComposeFile": [ "../docker-compose.yml", "docker-compose.yml" ], // The 'service' property is the name of the service for the container that VS Code should // use. Update this value and .devcontainer/docker-compose.yml to the real service name. "service": "app", ... Notice that you’ll see two docker-compose.yml files. The first one is your existing file. Assuming it’s in the root of your project, it gets referenced from inside the .devcontainer folder. The second one is a custom override created by the dev container setup. It lives inside the .devcontainer folder. Leave that as-is. From here, you can add “features” to the dev container and any other things you might want from the setup wizard. For example, I added the PostgreSQL client so that I can use it to connect to my database from inside the dev container. Finally, you’ll choose to expose any ports that your app exposes. These might be added automatically if it exists in your Dockerfiles. Here is the full version of the devcontainer.json that currently works for my project: // For format details, see https://aka.ms/devcontainer.json. For config options, see the // README at: https://github.com/devcontainers/templates/tree/main/src/docker-existing-docker-compose { "name": "Project name", // Update the 'dockerComposeFile' list if you have more compose files or use different names. // The .devcontainer/docker-compose.yml file contains any overrides you need/want to make. "dockerComposeFile": [ "../docker-compose.yml", "docker-compose.yml" ], // The 'service' property is the name of the service for the container that VS Code should // use. Update this value and .devcontainer/docker-compose.yml to the real service name. "service": "app", // The optional 'workspaceFolder' property is the path VS Code should open by default when // connected. This is typically a file mount in .devcontainer/docker-compose.yml "workspaceFolder": "/workspaces/${localWorkspaceFolderBasename}", // Features to add to the dev container. More info: https://containers.dev/features. "features": { "ghcr.io/devcontainers/features/aws-cli:1": {}, "ghcr.io/devcontainers/features/github-cli:1": {}, "ghcr.io/devcontainers/features/node:1": {}, "ghcr.io/devcontainers-extra/features/ripgrep:1": {}, "ghcr.io/robbert229/devcontainer-features/postgresql-client:1": {} }, // Use 'forwardPorts' to make a list of ports inside the container available locally. "forwardPorts": [ 3000 ] // Uncomment the next line if you want start specific services in your Docker Compose config. // "runServices": [], // Uncomment the next line if you want to keep your containers running after VS Code shuts down. // "shutdownAction": "none", // Uncomment the next line to run commands after the container is created. // "postCreateCommand": "cat /etc/os-release", // Configure tool-specific properties. // "customizations": {}, // Uncomment to connect as an existing user other than the container default. More info: https://aka.ms/dev-containers-non-root. // "remoteUser": "devcontainer" } Note: As of this writing, it looks like you can only get Node version 22 as a feature in the official Microsoft images. If you need other versions of node, you can get them from third-party community builds. From here you should be able to tell vs code to Open Workspace in Container. You’ll see it spin up and you’ll see a bunch of logs. If it’s working properly, you’ll see it build an image from your dockerfile and then spin up the containers. Eventually the vs code terminal will open, and you’ll be dropped into a bash prompt inside your dev container. Success! But obviously it’s not gonna work the first time. I had to debug several issues first. Required files should be optional Because I was only using docker compose for local development up to this point, I had my local .env file referenced in the docker-compose.yml under the env_file property. services: app: image: my-app container_name: my-app build: context: . dockerfile: Dockerfile ports: - "3000:3000" env_file: - backend/.env The problem is that by default, this file is required. If it’s not present, docker compose will error. When you go to build the dev container, this file may not be present. Probably because you put it in .gitignore so you didn’t accidently check it into version control. This is the right thing. So the fix here is to use the extended format to specify that the env_file is optional. services: app: image: my-app container_name: my-app build: context: . dockerfile: Dockerfile ports: - "3000:3000" env_file: - path: backend/.env required: false Adding your env variables back to the build Because Dev Containers won’t have your .env file, you’ll need to add the environment variables directly to the docker-compose.yml. Yes this is redundant, but I don’t know a cleaner alternative. Make sure you’re only adding standard env variables and not secrets. Secrets will go elsewhere, but they’re fine in the .env for now. According to the docker compose docs, you can have both env_file and environment specified in your config, but entries in environment will always take precedence. So you can have your .env file for local development and the environment entries for when it gets pushed to github. services: app: image: my-app container_name: my-app build: context: . dockerfile: Dockerfile ports: - "3000:3000" environment: PORT: "3000" AWS_REGION: us-west-2 S3_BUCKET_NAME: bucket-dev SENTRY_DSN: "..." env_file: - path: backend/.env required: false Note: You can also add environment variables directly to the devcontainer.json. But I didn’t try this and YMMV. Build args are different from environment variables My project uses Hono for the backend. The frontend uses the Vue framework and gets built for production using Vite. If you’ve dealt with Vite, you know it has a specific workflow for how to get environment variables into your frontend. When you prefix your environment variables with VITE_, they get built into your frontend bundle at build time. For example if you use S3_BUCKET_NAME on the backend, you have to use VITE_S3_BUCKET_NAME in frontend code. You may be more familiar with NEXT_S3_BUCKET_NAME in Nextjs. Same deal. When you use vite locally, it knows how to find the build variables in your .env file, and it just works. But when you’re using docker, these variables need to be present when the Dockerfile is being built. This can be a pain to figure out even without the added complexity of Dev Containers. I’ll just give you the answer here. The short version is that you reference the the vite variables as docker “build args”, then you inject them into the environment during the docker build. ARG VITE_S3_BUCKET_NAME ENV VITE_S3_BUCKET_NAME=$VITE_S3_BUCKET_NAME RUN npm run build ... Then you need to add these build args to your docker-compose.yml in the build section so they get passed into the Dockerfile execution. Yes this is redundant, but I don’t know a cleaner alternative. And once again, make sure you’re only adding standard build variables and not secrets. services: app: image: my-app container_name: my-app build: context: . dockerfile: Dockerfile args: VITE_S3_BUCKET_NAME: "bucket-dev" VITE_SENTRY_DSN: "..." ports: - "3000:3000" environment: PORT: "3000" AWS_REGION: us-west-2 S3_BUCKET_NAME: bucket-dev SENTRY_DSN: "..." env_file: - path: backend/.env required: false Note: You can also add build args manually using the cli. docker build --build-arg "VITE_S3_BUCKET_NAME=..." There may be other changes you need to make to your docker compose setup. But these are the major issues I ran into that related to Dev Containers specifically. If you’re lucky, you should be able to spin up a dev container for your project locally and try it out. Next we have to get this to work with github codespaces. Which of course requires even more steps. Note: If your dev container spins up, but your app doesn’t work, make sure the app can connect to the database, and make you’ve actually populated the database. It’s probably still empty! Getting Your dev Container working in Github Codespaces Once you’ve got a dev container working locally, you can check all of the config files into github and push it. This includes the .devcontainer folder and any changes you had to make to Dockerfile and docker-compose.yml. I did this in a branch at first. Mostly because I wasn’t confident enough yet to drop it into main. But also because I was explicitly looking for this to work on pull requests. Creating a codespace Github will recognize your .devcontainer folder in your project and allow you to create a codespace. But if you’re working in a private repo, you may need to make sure to enable codespaces first. Then you can go to your repo page and look for codespaces under the green “Code” button on the top right. Click to create a codespace, and make sure you select the appropriate branch that you want to use. The codespace will clone that git branch into the environment before running the build. The Codespaces docs do a decent job of introducing this stuff, so I won’t go into detail here. We’ve done most of the heavy lifting with the local Dev Container setup. Where to create new codespaces Adding secrets You can try to spin up a codespace in github, but it probably won’t quite work yet. Codespaces works almost like using Dev Containers locally. It can find your environment variables in your docker-compose.yml files. But it can’t find any secrets. Those have to be injected by github. Codespaces has it’s own separate space for variables and secrets. See this screenshot of the settings menu. Where to add secrets in github settings Once you’ve got the secrets in, try rebuilding your codespace to pick them up. Running the app When using docker containers, you usually specify a default command to run when the container starts up. This is usually where you start your web server. By default, Dev Containers overrides your command. Instead they add a simple command that just sleeps and keeps the codespace up and running so you can connect to it. What this means is that your app is not running by default. You’ll have to go into the codespace terminal and run it. For me it’s just an npm script. npm run start This is important because you want github to create a unique url for you to access your app from the web browser. That doesn’t happen until the codespace detects that a port is being used. You can leave your terminal running with your app, or you can put it in the background and manage it with something like systemd. Depends on how sophisticated you wanna get. Wrapping up Hooray! Hopefully you’ve successfully got a github codespace running your app. Github will give you a unique, generated url to access it. And you should be able to spin one up against any branch or pull request. I hope this saves someone some headaches when getting Github Codespaces working. Once you get it going, it’s very cool and useful. Unfortunately I don’t have any advice for getting this working if you don’t already have your own docker setup. But the docker approach isn’t well documented in the searches I did, so this is my contribution to fixing that.

This is the first in probably a series of posts as I dig into the technical aspects of mastodon. My goal is to get a better understanding of the design of ActivityPub and how mastodon itself is designed to use ActivityPub. Eventually I want to learn enough to maybe do some hacking and create some of the experiences I want that mastodon doesn’t support today. The first milestone is just getting a mastodon instance set up on my laptop. I’m gonna give some background and context. If you want to skip straight to the meat of things, here’s an anchor link. Some background Mastodon is a complex application with lots of moving parts. For now, all I want is to get something running so I can poke at it. Docker should be a great tool for this. Because a lot of that complexity can be packaged up in pre-built images. I tried several times using using the official docs and various other alternative projects to get a working mastodon instance in docker. But I kept running into problems that were hard to understand and harder to resolve. I have a lot to learn about all the various pieces of mastodon and how they fit together. But I understand docker pretty well. So after some experimenting, I was able to get an instance running on my own. The rest of this post will be dedicated to explaining what I did and what I learned along the way. One final note. I know many folks work hard to write docs and provide an out of the box dev experience that works. This isn’t meant to dismiss that hard work. It just didn’t work for me. I’m certainly going to share this experience with the mastodon team. Hopefully these lessons can make the experience better for others in the future. The approach Here’s the outline of what we’re doing. We’re going to use a modified version of the docker-compose.yml that comes in the official mastodon repo. It doesn’t work out of the box. So I had to make some heavy tweaks. As of this writing, the mastodon docs seem to want people to use an alternate setup based on Dev Containers. I found that very confusing, and it didn’t work for me at all. Once we have all of the docker images we need, all of the headaches are in configuring them to work together. Most of mastodon is a ruby on rails app with a database. But there is also a node app to handle streaming updates, redis for caching and background jobs, and we need to handle file storage. We will do the minimum configuration to get all of that set up and able to talk to each other. There is also support for sending emails and optional search capabilities. These are not required just to get something working, so we’ll ignore them for now. It’s also worth noting that if we want to develop code in mastodon, we need to put our rails app in development mode. That introduces another layer of headaches and errors that I haven’t figured out yet. So that will be a later milestone. For now, all of this will be in “production” mode by default. That’s how the docker image comes packaged. Keep it simple. There are still many assumptions here. I am running on Mac OS with Apple Silicon (M3). If you’re trying this out, you may run into different issues depending your environment. Pre-requisites We need docker. And a relatively new version. The first thing I did was ditch the version: 3 specifier in the docker-compose.yml. Using versions in these files is deprecated, and we can use some newer features of docker compose. I have v4.30.0 of Docker Desktop for Mac. We also need caddy. Mastodon instances require a domain in most cases. This is mostly about identity and security. It would be bad if an actor on mastodon could change their identity very easily just by pretending to be a different domain or account. There are ways around this, but I couldn’t get any of them to work for me. That complicates our setup. Because we can’t just use localhost in the browser. We need a domain, which means we also need HTTPS support. Modern browsers require it by default unless you jump through a bunch of hoops. Caddy gives us all of that out of the box really easily. It will be the only thing running outside of docker. There’s only one caveat with caddy. The way that it is able to do ssl termination so easily is that it creates its own certificates on the fly. The way it does this is by installing it’s own root cert on your machine. You’ll have to give it permission by putting in your laptop password the first time you run caddy. If that makes you nervous, feel free to skip this and use whatever solution you’re comfortable with for SSL termination. But as far as I know, you need this part. Choose a domain for your local instance. For me it was polotek-social.local. Something that mkes it obvious that this is not a real online instance. Add an entry to your /etc/hosts and point this to localhost. Or whatever people have to do on Windows these days. Let’s run a mastodon I put all of my changes in my fork of the official mastodon repo. You can clone this branch and follow along. All of the commands assume you are in the root directory of the cloned repo. https://github.com/polotek/mastodon/tree/polotek-docker-build > git clone [email protected]:polotek/mastodon.git > cd mastodon > git co -b polotek-docker-build I rewrote the docker section of the README.md to outline the new instructions. I’m going to walk through my explanation of the changes. Pull docker images This is the easiest part. All of the docker images are prepackaged. Even the rails app. You can use the docker compose command to pull them all. It’ll take a minute or 2. > docker compose pull Setup config files We’re using a couple of config files. The repo comes with .env.production.sample. This is a nice way to outline the minimum configuration that is required. You can copy that to .env.production and everything is already set up to look for that file. The only thing you have to do here is update the LOCAL_DOMAIN field. This should be the same as the domain you chose and put in your /etc/hosts. You can put all of your configuration in this file. But I found it more convenient to separate out the various secrets. These often need to be changed or regenerated. I wrote a script to make that repeatable. Any secrets go in .env.secrets. We’ll come back to how you get those values in a bit. I had to make some other fixes here. Because we’re using docker, we need to update how the rails app finds the other dependencies. The default values seem to assume that redis and postgres are reachable locally on the same machine. I had to change those values to match the docker setup. The REDIS_HOST is redis, and the DB_HOST is db. Because that’s what they are named in the docker-compose file. Diff of config file on github The rest of the changes are just disabling non-essential services like elastic search and s3 storage. Generate secrets We need just a handful of config fields that are randomly generated and considered sensitive. Rails makes it easy to generate secrets. But run the required commands through docker and getting them in the right place is left as an exercise for the reader. I added a small script that runs these commands and outputs the right fields. Rather than try to edit the .env.production file in the right places everytime secrets get regenerated, I think it’s much easier to have them in a separate file. Fortunately, docker-compose allows us to specify multiple files to fill out the environment variables. Diff of config file on github This was a nice quality of life change. And now regenerated secrets and making them available is just one command. > bin/gen_secrets > .env.secrets Any additional secrets can be added by just updating this script. For example, I use 1password to store lots of things, even for development. And I can pull things out using their cli named op. Here’s how I configured the email secrets with the credentials from my mailgun account. # Email echo SMTP_LOGIN=$(op read "op://Dev/Mailgun SMTP/username") echo SMTP_PASSWORD=$(op read "op://Dev/Mailgun SMTP/password") Run the database Running the database is easy. > docker compose up db -d You’ll need to have your database running while you run these next steps. The -d flag will run it in the background so you can get your terminal back. I often prefer to skip the -d and run multiple terminal windows. That way I can know at a glance if something is running or not. But do whatever feels good. The only note here is to explain another small change to docker-compose to get this running. We’re using a docker image that comes ready to run postgres. This is great because it removes a lot of the fuss of running a database. The image also provides some convenient ways to configure the name of the database and the primary user account. This becomes important because maston preconfigures these values for rails. We can see this in the default .env.production values. DB_USER=mastodon DB_NAME=mastodon_production The database name is not a big issue. Rails will create a database with that name if it doesn’t exist. But it will not create the user (maybe there’s a non-standard flag you can set?). We have to make sure postgres already recognizes a user with the name mastodon. That’s easy enough to do by passing these as environment variables to the database container only. Diff of config file on github Load the database schema One thing that’s always a pain when running rails in docker. Rails won’t start successfully until you load the schema into the database and seed it with the minimal data. This is easy to do if you can run the rake tasks locally. You can’t run the rake tasks until you have a properly configured rails. And it’s hard to figure out if your rails is configured properly because it won’t run without the database. I don’t know what this is supposed to look like to a seasoned rails expert. But for me it’s always a matter of getting the db:setup rake task to run successfully at least once. After that, everything else starts making sense. However, how do you get this to work in our docker setup? We can’t just do docker compose up, because the rails container will fail. We can’t use docker compose exec because that expects to attach to an existing instance. So the best thing to do is run a one-off container that only runs the rake task. The way to achieve that with docker compose is docker compose run --rm. The rm flags just makes sure the container gets trashed afterwards. Because we’re running our own command instead of the default one, we don’t want it hanging around and potentially muddying the waters. Once we know the magic incantation, we can setup the database. > docker compose run --rm web bundle exec rails db:setup Note: Usually you don’t put quotes around the whole command. For some reason, this can cause problems in certain cases. You can put quotes around any individual arguments if you need to. Run rails and sidekiq If you’ve gotten through all of the steps above, you’re ready to run the whole shebang. > docker compose up This will start all of the other necessary containers, including rails and sidekiq. Everything should be able to recognize and connect to postgres and redis. We’re in the home stretch. But if you try to reach rails directly in your browser by going to https://localhost:3000, you’ll get this cryptic error. ERROR -- : [ActionDispatch::HostAuthorization::DefaultResponseApp] Blocked hosts: localhost:3000 It took me a while to track this down. It’s a nice security feature built into rails. When running in production, you need to configure a whitelist of domains that rails will run under. If it receives request headers that don’t match those domains, it produces this error. This prevents certain attacks like dns rebinding. (Which I also learned about at the same time) If you set RAILS_ENV=development, then localhost is added to the whitelist by default. That’s convenient, and what we would expect from dev mode. But remember we’re not running in development mode quite yet. So this is a problem for us. The nice thing is that mastodon has added a domain to the whitelist already. Whatever value you put in the LOCAL_DOMAIN field is recognized by rails. (In fact, if you just set this to localhost you might be good to go. Shoutout to Ben.) However, when you use an actual domain, then most modern web browsers force you to use HTTPS. This is another generally nice security feature that is getting in our way right now. So we need a way to use our LOCAL_DOMAIN, terminate SSL, and then proxy the request to the rails server running inside docker. That brings us to the last piece of the puzzle. Running caddy outside of docker. Run a reverse proxy The configuration for caddy is very basic. We put in our domain, we put in two reverse proxy entries. One for rails and one for the streaming server provided by node.js. Assuming you don’t need anything fancy, caddy provides SSL termination out of the box with no additional configuration. # Caddyfile polotek-social.local reverse_proxy :3000 reverse_proxy /api/v1/streaming/* :4000 We put this in a file named Caddyfile in the root of our mastodon project, then in a new terminal window, start caddy. > caddy run Success? If everything has gone as planned, you should be able to put your local mastodon domain in your browser and see the frontpage of mastodon! Mastodon frontpage running under local domain! In the future, I’ll be looking at how to get actual accounts set up and how to see what we can see under the hood of mastodon. I’m sure I’ll work to make all of this more developement friendly to work with. But I learned a lot about mastodon just by getting this to run. I hope some of these changes can be contributed back to the main project in the future. Or at least serve as lessons that can be incorporated. I’d like to see it be easier for more people to get mastodon set up and start poking around.

I don’t know who needs to hear this. But your frontend and backend systems don’t need to be completely separate. I started anew side project recently. You know, one of things that allows me to tinker with new technology but will probably never be finished. I’m using Angular for the frontend and Nestjs for the backend. All good. But then I go to do something that I thought was very normal and common and run into a wall. I want to integrate the two frameworks. I want to serve my initial html with nestjs and add script tags so that Angular takes over the frontend. This will allow me to do dynamic things on the backend and frontend however I want. But also deploy the system all as one cohesive product. Apparently this is no longer How Things Are Done. I literally could not find documentation on how to do this. When you read the docs and blog posts, everybody expects you to just have two systems that run entirely independently. Here’s the server for your backend and here’s the entirely different server for your frontend. Hashtag winning! When I google for “integrate angular and nestjs”, nobody knows what I’m talking about. On the surface, this seems like is a great technical blog post from LogRocket. It says “I will teach you how. First, set up two separate servers…” I think I know why the community has ended up in this place. But that’s a rant for another blog post. Let me try to explain what I’m talking about. Angular is designed as an a frontend framework (let’s set aside SSR for now). The primary output of an Angular build is javascript and css files that are meant to run in the browser. When you run ng build, you’ll get a set of files put into your output folder. Usually the folder is dist/<your_project_name>. Let’s look at what’s in there. polotek $> ls -la dist/my-angular-project -rw-r--r-- 1 polotek staff 12K Sep 13 14:15 3rdpartylicenses.txt -rw-r--r-- 1 polotek staff 948B Sep 13 14:15 favicon.ico -rw-r--r-- 1 polotek staff 573B Sep 13 14:15 index.html -rw-r--r-- 1 polotek staff 181K Sep 13 14:15 main.c01cba7b28b56cb8.js -rw-r--r-- 1 polotek staff 33K Sep 13 14:15 polyfills.2f491a303e062d57.js -rw-r--r-- 1 polotek staff 902B Sep 13 14:15 runtime.0b9744f158e85515.js -rw-r--r-- 1 polotek staff 0B Sep 13 14:15 styles.ef46db3751d8e999.css Some javascript and css files. Just as expected. A favicon. Sure, why not. Something about 3rd party licenses. I have no idea what that is, so let’s ignore it. But there’s also an index.html file. This is where the magic is. This file sets up your html so it can serve Angular files. It’s very simple and looks like this. <!doctype html> <html lang="en" data-critters-container> <head> <meta charset="utf-8"> <title>MyAngularProject</title> <base href="/"> <meta name="viewport" content="width=device-width, initial-scale=1"> <link rel="icon" type="image/x-icon" href="favicon.ico"> <link rel="stylesheet" href="styles.ef46db3751d8e999.css"> </head> <body> <app-root></app-root> <script src="runtime.b3cecf81bdcc5839.js" type="module"></script> <script src="polyfills.41808b7aa9da5ebc.js" type="module"></script> <script src="main.cf1267740c62d53b.js" type="module"></script> </body> </html> It turns out the web browser still works the way it always did. You use <script> tags and <link> tags to load your javascript and css into the page. But we want to let the backend do this rather than using this static html file. I’m using NestJS for the backend. It’s modeled after Angular, so a lot of the structures are very similar. Just without all of the browser-specific stuff. Nest is not so important here though. This problem is the same with whatever backend you’re using. The important thing is how static files are served. If you copy the above html into a backend template, it probably won’t work. This is what you get in the browser when you try this with NestJS. Angular fails to load. This is part of my gripe. By default, these are two separate systems right now. So NestJS doesn’t know that these files exist. And they’re in two separate folders. So it’s unclear what the best way is to integrate them. In the future, I might talk about more sustainable ways to do this for a real project. But for now, I’m going to do the simple thing just to illustrate how this is supposed to work. In NestJS, or whatever backend you’re using, you should be able to configure where your static files go. In Nest, it looks something like this. async function bootstrap() { const app = await NestFactory.create<NestExpressApplication>(AppModule); app.useStaticAssets(path.resolve("./public")); await app.listen(3000); } bootstrap(); So there should be a folder called public in your backend project, and that’s where it expect to find javascript and css files. So here’s the magic. Copy the Angular files into that folder. Let’s say you have the two projects side by side. It might look like this. polotek $> cp my-angular-project/dist/my-angular-project-ui/* my-nest-project/public/ This will also copy the original index.html file and the other junk. We don’t care about that for now. This is just for illustration. So now we’ve made NestJS aware of our Angular files. Reload your NestJS page and you should see this. Assets loading properly. Angular Welcome screen loading. We did it! This is how to integrate a cohesive system with frontend and backend. The frontend ecosystem has wandered away from this path. But this is how the web is supposed to work in my opinion. And more importantly, it is actually how a lot of real products companies want to manage their system. I want to acknowledge that there are still a lot of unanswered questions here. You can’t deploy this to production. The purpose of this blog post is to help the next person like me who was trying to google how to actually integrate Angular and a backend like NestJS because I assumed there was a common and documented path to doing so. If this was useful for you, and you’re interested in having me write about the rest of what we’re missing in modern frontend, let me know.