Menu

Subscribe

The Pragmatic Engineer in 2024

The last 12 months, The Pragmatic Engineer covered a variety of deepdives, revealing previously unshared details like:

We also did deepdives with hands-on experts on security engineering, reliability engineering and how to thrive as a founding engineer, just to name a few.

This list is just 8 of the ~100 industry deepdives that full subscribers received. This article is a collection of all The Pragmatic Engineer issues published in the past 12 months.

In every issue, The Pragmatic Engineer covers challenges at Big Tech and startups through the lens of senior engineers and engineering managers. If you’re not yet a member, consider subscribing. It’s the #1 technology newsletter on Substack. See what readers say about it.

The below list collects all articles from November 2023 to November 2024. Browse all deepdives, and also the most popular ones.

Real-World Engineering Challenges

How Anthropic built Artifacts. The team behind Artifacts - an innovative new way to interact with Claude - shares how they built the feature in three months with a distributed team. Exclusive details.

The biggest-ever global outage: lessons for software engineers. Cybersecurity vendor CrowdStrike shipped a routine rule definition change to all customers, and chaos followed as 8.5M machines crashed, worldwide. There are plenty of learnings for developers.

Building Bluesky: a Distributed Social Network (Real-World Engineering Challenges). Bluesky is built by around 10 engineers, and has amassed 5 million users since publicly launching in February this year. A deep dive into novel design decisions, moving off AWS, and more.

Scaling ChatGPT: Five Real-World Engineering Challenges Just one year after its launch, ChatGPT had more than 100M weekly users. In order to meet this explosive demand, the team at OpenAI had to overcome several scaling challenges. An exclusive deepdive.

Engineering culture deepdives

How to debug large, distributed systems: Antithesis. A brief history of debugging, why debugging large systems is different, and how the “multiverse debugger” built by Antithesis attempts to take on this challenging problem space

Inside Bluesky’s Engineering Culture. A deep dive into how a fully remote, open source-first, tech company is building a challenger social media platform. What can small teams learn from Bluesky about punching above your weight?

Startups on hard mode: Oxide. Part 1: Hardware. What is tougher than building a software-only or hardware-only startup? Building a combined hardware and software startup. This is what Oxide is doing, as they build a “cloud computer.” 

A startup on hard mode: Oxide, Part 2. Software & Culture. Oxide is a hardware and a software startup, assembling hardware for their Cloud Computer, and building the software stack from the ground up. A deep dive into the company’s tech stack & culture.

Inside Stripe’s Engineering Culture - Part 1. Stripe is one of the largest online payment companies and operates one of the world’s largest Ruby codebases. But there’s more to Stripe than payments and Ruby. A deep dive with CTO David Singleton.

Inside Stripe’s Engineering Culture: Part 2. A deep dive into its engineering culture: operational excellence, API review, internal tools, and more.

What is Old is New Again. The past 18 months have seen major change reshape the tech industry. What does this mean for businesses, dev teams, and what will pragmatic software engineering approaches look like, in the future?

The end of 0% interest rates. What it means for:

  • Tech startups and the industry. The past 15 years saw the lowest interest rates in modern history, and this “zero interest-rate period” (ZIRP) meant growing tech companies was easy. With this period over, what changes for startups?
  • Software engineers. The end of 10+ years of 0% interest rates is set to change the tech industry. What do higher rates mean for software engineering jobs, developers, and careers; and how can you prepare for this shift?
  • Engineering managers and tech leads. We’re likely to see a preference for flatter organizations, fewer managers, and a preference for the “player coach” leadership model. Some changes present new opportunities to shine as leaders.
  • For software engineering practices Could we see monoliths favored over microservices, full-stack tools over platform specific ones and pragmatic, simpler architecture as a response to 10+ years of 0% interest rates ending?

Career

How to become a more effective engineer. The importance of soft skills, implicit hierarchies, getting to “small wins”, understanding promotion processes and more. A guest post from software engineer Cindy Sridharan.

Hiring software engineers and engineering leaders from Big Tech (Part 2). Tactics and approaches for startups to hire software engineers with Big Tech experience, and why Amazon is a leading source of talent for early-stage businesse

Hiring software engineers and engineering leaders from Big Tech (Part 1) A dive into why hiring Big Tech talent can be a bad idea for startups, a look at cases when it works, what makes it hard to recruit from Big Tech, and how to do it

State of the software engineering job market in 2024. A deep dive into job market trends, the companies and cities hiring the most software engineers, growth areas, and more. Exclusive data and charts

Why techies leave Big Tech. A job in Big Tech is a career goal for many software engineers and engineering managers. So what leads people to quit, after working so hard to land these roles?

Software engineers training software engineers. What is it like to teach software engineers, full time? Reuven M. Lerner has done this for 15 years, and shares his hands-on learnings – including how to teach efficiently

Leading Effective Engineering Teams: a Deepdive. What makes software teams effective, and how do the TL, EM and TLM roles differ? An excerpt from Addy Osmani’s new book: Leading Effective Engineering Teams

Surprise uptick in software engineering recruitment. June and July are usually the quietest months for tech recruitment. This year there’s been a spike in interest from recruiters in software engineers and EMs. We dig into this unexpected, welcome trend

The Trimodal Nature of Tech Compensation Revisited. Why can a similar position offer 2-4x more in compensation, in the same market? A closer look at the trimodal model I published in 2021. More data, and new observations.

Getting an Engineering Executive Job. An overview of successful, tried-and-true routes into CTO, VPE, and Head of Engineering jobs, from the new book, ‘The Engineering Executive’s Primer’ by Will Larson.

Thriving as a Founding Engineer: Lessons from the Trenches. Being a founding engineer at an early-stage startup is a vastly different, broader role than many people think. Lessons from “serial” early-stage and founding engineer Apurva Chitnis.

Senior-and-Above Compensation in Tech. How well does tech pay, really? A deep look into almost 1,000 data points sent in by engineering managers, senior+ engineers, VP, and C-level folks in tech, mostly working in software engineering

Holiday Season Gift Ideas for Techies. From books, gadgets, and office accessories, to decor, wellness, and non-tech gifts. If you’re unsure what to buy friends and relations who work in tech, this article offers some inspiration

Software engineering approaches

What is Reliability Engineering? A history of SRE practice and where it stands today, plus advice on working with reliability engineers, as a software engineer. A guest post by SRE expert and former Googler, Dave O’Connor

Bug management that works (Part 1). Finding and triaging bugs, fixing bugs on the spot instead of ‘managing’ them, and how to make time for bug fixing

Paying down tech debt A guide for reducing tech debt effectively, and how to develop a mindset that welcomes the short-term benefits of eliminating it. A guest post by principal engineer Lou Franco

Adopting Software Engineering Practices Across the Team. Common software engineering practices, adopting them within a team, and why blindly adopting practices is a bad idea

How do AI software engineering agents work? Coding agents are the latest promising Artificial Intelligence (AI) tool, and an impressive step up from LLMs. This article is a deep dive into them, with the creators of SWE-bench and SWE-agent.

Applied AI Software Engineering: RAG. Retrieval-Augmented Generation (RAG) is a common building block of AI software engineering. A deep dive into what it is, its limitations, and some alternative use cases. By Ross McNairn.

What is Security Engineering? Part 1. A deep dive into the ever-changing field of security engineering; a domain that can feel intimidating to some software engineers. With Nielet D'Mello, security engineer at Datadog.

What is Security Engineering? Part 2. A broad overview of the ever-developing security engineering field; a domain that can feel intimidating to some software engineers. With Nielet D'Mello, security engineer at Datadog.

What’s Changed in 50 Years of Computing: Part 3. How has the industry changed 50 years after the ‘The Mythical Man-Month’ was published? A look into estimations, developer productivity and prototyping approaches evolving.

What Changed in 50 Years of Computing: Part 1. How has the classic book on software engineering, ‘The Mythical Man Month,’ aged with time, and is it still relevant half a century on – or does it belong in a museum, alongside floppy discs?

What Changed in 50 Years of Computing: Part 2. How has the classic book on software engineering, ‘The Mythical Man-Month,’ aged with time, and how have architecture approaches and tech org structures changed in half a century?

Organizing and Running Successful Hackathons Hackathons are fun for engineers, beneficial for businesses, and a good way to shake things up. This article suggests approaches for running successful hackathons – and whether you should hold one.

Quality Assurance Across the Tech Industry An overview of Quality Assurance (QA) approaches at various companies, and a look at tech segments where QA is on the decline, and where it is holding strong.

Measuring Developer Productivity: Real-World Examples. A deepdive into developer productivity metrics used by Google, LinkedIn, Peloton, Amplitude, Intercom, Notion, Postman, and 10 other tech companies.

Dead Code, Getting Untangled, and Coupling versus Decoupling. Three full chapters from the book Tidy First? by Kent Beck. The book offers book 33 practical - and increasingly sophisticated - approaches to make your code and systems more tidy.

Code Freezes: Part 3. It’s December, the month when many mid-size and large companies put code freeze policies in place. This article provides a wide ranging overview of this practice, based on responses from 185 readers

Practical industry research

AI Tooling for Software Engineers in 2024: Reality Check (Part 1). How do software engineers utilize GenAI tools in their software development workflow? We sidestep the hype, and look to the reality of tech professionals using LLMs for coding and other tasks.

AI Tooling for Software Engineers: Reality Check (Part 2). How do software engineers using AI tools view their impact at work? We sidestep the hype to find out how these cutting-edge tools really perform, from the people using them daily.

AI Tooling for Software Engineers: Rolling Out Company-Wide (Part 3) How large tech companies are using internal AI tools. Also: guidelines and practical approaches for embracing LLM tools for software development on the individual dev, and organizational level

What do GenZ software engineers really think? Young software engineers discuss values, what frustrates them about working in tech, and what they really think of older colleagues. Responses to our exclusive survey.

GenZ software engineers, according to older colleagues. Responses to a survey about GenZ suggest this new generation possesses standout differences. We explore what makes GenZ distinctive, and check out ideas for ways to work fruitfully together.

Podcast

Linear: move fast with little process (with first engineering manager Sabin Roman). The project management and issue tracking tool is wildly popular within startups and scaleups. Their 25-person eng team ships rapidly, with high-quality, while working full-remote. How do they do it?

AI tools for software engineers, but without the hype – with Simon Willison (co-creator of Django). Ways to use LLMs efficiently, as a software engineer, common misconceptions about them, and tips/hacks to better interact with GenAI tools. The first episode of The Pragmatic Engineer Podcast

Efficient scaleups in 2024 vs 2021: Sourcegraph (with CEO & Co-founder Quinn Slack). Sourcegraph is one of many scaleups that have significantly changed how they opreate versus just a few years ago. The observations and lessons here can be widely applicable across the industry.

Twisting the rules of building software: Bending Spoons (the team behind Evernote) Its controversial acquisitions approach, why Bending Spoons aims to have no oncall, the Evernote migration in order to retire the monolith, and more.

Promotions and tooling at Google (with Irina Stanescu, Ex-Google). An inside look at Google’s unique working processes, tactical advice for getting promoted at companies like Google and Uber, and how to build influence as a software engineer.

The Pulse

Q4 2024

#116: Netflix sets live streaming world record with boxing match. Also: why some late-stage companies don’t want to go public, possible crackdown on low-output remote engineers, and more

#115: LLM improvements slowing down? Several signs indicate that improving LLMs with more training/compute is no longer efficient. Also: dev charged $1,100 after following a tutorial, a reminder to be vigilant with open source, and more

#114: What does Trump’s win mean for Tech? More influence for Musk and VCs, potentially reversing Section 174, pro-crypto, uncertainty for Big Tech, and more. Also: a third embarrassing security issue for Okta in two years

#113: Engineering culture at Google vs Meta. Also: AI now generates 25% of code at Google; Deep cuts at profitable companies like Dropbox and Miro; Business booming at the likes of Google, Meta and Microsoft, and more.

#112: Similarities between AI bots using a computer and end-to-end testing. Also: Automated reasoning proves system correctness at AWS, Winamp code shows why software licenses are important, and more

#111: Did Automattic commit open source theft? The maker of WordPress took 2M customers from its biggest rival: has a red line been crossed? Also: OpenAI’s impossible business projections, top AI researchers making more than engineers, and more.

#110: VC-funded companies acting more like bootstrapped ones? Also: first-ever double Nobel Prize wins for AI research, and an interesting cloud platform price comparison startup built on a budget

#109: Open source business model struggles at Wordpress. Also: OpenAI’s biggest-ever fundraise even as key people keep quitting; why executive recruiters ignore tech professionals, and more

Q3 2024

#108: Elasticsearch unexpectedly goes open source again. Forced by AWS to change the license, Elasticsearch reverts to a permissive one three years later. Also: Amazon cuts the number of managers, engineers critiquing YouTube’s biggest critic.

#107: What does Amazon’s 5-day RTO mean for tech? Amazon is the first Big Tech to mandate a strict 5-day return to office. What are the real reasons, will Amazon see a ‘brain drain’ as a result, and could other Big Tech companies follow?

#106: Why does the EU tech sector lag behind the US? Also: non-AI companies like Klarna want to look like AI ones, unusual details about Google, Amazon, and Microsoft levels, and more

#105: More trad tech companies to move off the public cloud? Also: CockroachDB joins the trend of going from open source to proprietary license, a software glitch nearly floods Amsterdam, and more.

#104: The startup purge event is, unfortunately, here. Also: Sonos’ app rewrite was a disastrous YOLO release, similarities between AI companies and telco companies, what it’s like to test compilers, and more

#103: Why did AWS just sunset 8 products? AWS rarely discontinues products, but now it’s sunsetting eight in one go. Also: GenAI investments are money pits, and the “wicked loop” of problems at tech companies.

#102: Intel’s rough business outlook and full reorg. Also: AI startup founders keep defecting to Big Tech, buggy app takes Sonos 6 months to fix, CrowdStrike faces huge bills for historic global outage, and more

#101: Did AWS forget it’s a cloud infra company? Also: why GitLab is seeking a buyer, how Alexa got left behind in conversational AI, and Cloudflare offering customers less AI – because those customers want this. 

#100: Large AI Players Snap Up Smaller Ones. Also: why dev tools ask for work email, the “Big Stay” phenomenon, ChatGPT usage stalls then recovers, and more.

#99: Relational databases here to stay as good fits for AI? Also: $415M not enough for founders to stay at startup; France targeting NVIDIA with antitrust; a standout dev tools success story at GitLab, and more.

Q2 2024

#98: Is there a GenAI startup cooldown or not? Plenty of signs point to a cooldown happening, but there’s also GenAI mega-funding rounds. Also: Polyfill.js supply-chain attack, the importance of internships, and more.

#97: Lone hacker takes down North Korea’s internet. Also: what NVIDIA becoming the world’s most valuable company says about AI, controversy at Slack and Adobe about terms and conditions in the GenAI era, and more

#96: Apple demonstrates AI is best as many small features. Apple showcased how generative AI will spread across its operating systems, and how users can expect it to be free. Also: a new standard in confidential computing, and an outage “caused” by ChatGPT.

#95: Microsoft's security fiasco with Recall. A new Windows feature takes screenshots of users screens, but Microsoft has added no encryption or audits before shipping it. Also, shock serverless bills, Robotics + AI investments, and more.

#94: OpenAI’s ethics crisis. Claims of predatory stock clawback clause and unethical use of an actor’s voice plague leading AI startup. Also: Microsoft’s urgent focus on security.

#93: OpenAI makes Google dance. Google’s core search business has never been under as much threat as it is today – and the threat comes from OpenAI. Also: Google Cloud deletes a customer’s infra, and Tesla in trouble.

#92: GitHub’s AI-assisted developer workflow vision. Also: Google laying off engineers as their business is booming; a good period for startup fundraising and IPOs; and how WorkOS acquired devtools startup Warrant

#91: The end of US non-competes within sight? Also: the Humane AI pin flop and how it relates to the “AI goldrush,” and a look at whether developers will move from Redis to Valkey after a license change, or stay.

#90: Devin reversing ambitious claims. The “world’s first AI developer” tones down expectations and has been outperformed by an open source tool. Also: hiring upticks at Big Tech; a very realistic AI video generator by Microsoft, and more.

#89: The end of Hopin. In only 5 years, Hopin went from zero to a $7.7B valuation, and back to zero again. Also: Bending Spoons’ startup acquisition model, hiring upticks at Meta, Netflix and Amazon, and more

#88: are we at peak AI hype? Several signs are pointing that we’ve hit the peak of this AI hype cycle: that things could cool down soon enough. Also: the sudden license change at Redis; HashiCorp looking for a buyer, and more.

Q1 2024

#87: Stripe’s investment in reliability, by the numbers. The Fintech giant spends more on running test suites than Agoda does for all its infra. Plus, why taking out a loan for equity can backfire, and why did Donald Trump’s social media company use a SPAC?

#86: Is Shopify’s new “mastery” framework a response to higher interest rates? The e-commerce giant is taking a new and different approach to career growth and promotions. Also: more evidence the tech job market is tougher, and AI consolidation is already underway

#85: The Pulse #85: is the “AI developer”a threat to jobs – or a marketing stunt? One startup released “the first AI software engineer,” while another aims to build a “superhuman software engineer.” As intimidating as these sound: what if it’s more marketing than reality?

#84: Why is Apple bullying its own developers? In the 1990s, Microsoft was the company most developers hated with a passion. Today, Apple is working harder than any other organization to earn a similar reputation.

#83: Happy Leap Day! 29 February is causing problems in software systems across the globe. It’s a good reminder on how few assumptions we should make about dates – and why to use a date library when you can.

#82: Why did ChatGPT start to produce gibberish? Understanding the architecture of ChatGPT we can better pinpoint what exactly might have gone wrong. Also: Twilio’s cofounder CEO stepped down – could we see more cofounders follow suit?

#81: Could Vision Pro become a coding sidekick? Developers as power users should be an obvious target group for the Vision Pro. But the technology doesn’t seem to be there just yet. Also: NVIDIA became the world’s 3rd most valuable tech company.

#80: Meta’s Remarkable Turnaround 15 months ago, Meta was valued at a 7-year low, and embarked on laying off 25% of staff. Today, the company is valued more than it’s ever been before, and will hand out higher bonuses than ever.

#79: Is it fair for profitable companies to fire staff to make more money? Big Tech companies are enjoying record profits, but still doing mass layoffs. In the US, this is fair game, but in some EU countries, companies can’t let staff go without more justification.

#78: Is Google “the new IBM?” When Google went public, its founders stated “Google is not a conventional company. We do not intend to become one.” More signs point how it’s turning into the kind of organization it wanted to avoid.

#77: Will EMs and PMs take over TPM roles in a post-ZIRP world? Also: Instagram practically eliminated the TPM role. With so much overlap between EMs and PMs, will other companies follow? Also: a small win for app developers in the Epic vs Apple battle.

#76: Why are layoffs coming hard and fast, again? Also: why Salesforce seems to be hiring and firing based on their quarterly results; it’s a tough time to be a developer platform; and whether the Rabbit AI companion could be a smartphone replacement

#75: Will US companies hire fewer engineers due to Section 174? It’s rare that a tax change causes panic across the tech industry, but it’s happening in the US. If Section 174 tax changes stay, the US will be one of the least desirable countries to launch startups

Q4 2023

#74: Adobe Can’t buy Figma: the Impact of this on the Tech Industry. Regulators were always unlikely to allow Adobe’s $20B acquisition of Figma, and this intervention will have a ripple effect.Even fewer Big Tech companies buying startups. We analyze what it all means.

#73: Affirm Compensation Packages Made Public. A deepdive into software engineer compensation ranges at the buy-now-pay-later tech company. Also: senior engineering leadership roles are hard to get and Europe close to passing AI regulation.

#72: Spotify’s Shock Cuts. Despite its stock more than doubling in value in a year, the streaming giant is cutting 17% of its staff. But why? Also: Twitch shuts down in Korea and OpenSea’s revenue plummets 99% in 18 months.

#71: The Tech Behind Stripe’s Realtime Cyber Monday Dashboard. Also: a startup bragging how it “tricked” Google search found Google respond in record time; Sam Altman back at OpenAI; and the DevTernity scandal

What is OpenAI, Really? It’s been five incredibly turbulent days at the leading AI tech company, with the exit and then return of CEO Sam Altman. As we dig into what went wrong, an even bigger question looms: what is OpenAI?

See also:

Subscribe to my weekly newsletter to get articles like this in your inbox. It's a pretty good read - and the #1 tech newsletter on Substack.