More from Marco.org
Today, on the tenth anniversary of Overcast 1.0, I’m happy to launch a complete rewrite and redesign of most of the iOS app, built to carry Overcast into the next decade — and hopefully beyond. Like podcasts better than blog posts? Listen to ATP #596 for more! What’s new Much faster, more responsive, more reliable, and more accessible. Modern design, optimized for easily-reached controls on today’s phone sizes. Improvements throughout, such as undoing large seeks, new playlist-priority options, easier navigation, and more. What’s not Most features. Overcast is still Overcast! The audio engine. It’s the best part of Overcast, and still leads the industry in sound quality, silence skipping, and volume normalization. (More soon!) The business. I’m still a one-person operation, with no funding or external ownership, serving only my customers. My principles. I always want to make the best podcast app, and I’ll never disrespect your time, attention, or privacy. What’s gone Streaming. Most big podcasts now use dynamic ad insertion, which causes bugs and problems for streaming playback.1 Downloading episodes completely before they begin playback is much more reliable. Tapping a non-downloading episode will now open the playback screen, download it, then start playback. It works similarly to the way streaming did before, but playback begins after the download completes, not after a portion of it is buffered. On today’s fast networks, this usually only takes a few extra seconds. And in the near future, I’ll be adding smarter options and more control over selective downloading of episodes to further improve the experience for people who don’t automatically download every episode. What’s next The last few missing features from the old app, such as Shortcuts support, storage management, and OPML. These are absent now, but will return soon. More options for downloading and deleting episodes. Upgrading the Apple Watch app to the new, faster sync engine. (The Watch app is currently unchanged from the previous one.) And, of course, more features, including some of your most-requested features over the last decade. Getting this rewrite out the door was a monumental task. Thank you for your patience as I work through this list! Why? Most of Overcast’s core code was 10 years old, which made it cumbersome or impossible to easily move with the times, adopt new iOS functionality, or add new features, especially as one person. That’s why there haven’t been many new features or changes in years. You saw it, and I saw it. I wasn’t able to serve my customers as well as I wanted. For Overcast to have a future, it needed a modern foundation for its second decade. I’ve spent the past 18 months rebuilding most of the app with Swift, SwiftUI, Blackbird, and modern Swift concurrency. Now, development is rapidly accelerating. I’m more responsive, iterating more quickly, and ultimately making the app much better. Thank you all so much for the first decade of Overcast. Here’s to the next one. Dynamic ad insertion (DAI) splices ads into each download, and no two downloads are guaranteed to have the same number or duration of ads. So, for example, if the first half of an episode downloads, then the download fails, and it downloads the second half with another request, the combined audio may jump forward or back at the halfway mark, losing or repeating content. ↩︎
Overcast’s latest update (2022.2) brings the largest redesign in its nearly-eight-year history, plus many of the most frequently requested features and lots of under-the-hood improvements. I’m pretty proud of this one. For this first and largest phase of the redesign, I focused on the home screen, playlist screen, typography, and spacing. (I plan to revamp the now-playing and individual-podcast screens in a later update.) The home screen is radically different: Home screen, before (left) and after (right). Playlists now have strong visual identities for nicer and easier navigation. Each playlist has a customizable color, and a custom icon can be selected from over 3,000 SF Symbols to match modern iOS design and the other icons within Overcast. And playlists can be manually reordered with drag-and-drop. Recently played and newly published episodes can now be displayed on the home screen for quick access, much like the widget and CarPlay experience. Podcasts can now be pinned to the top of the home-screen list. Pinned podcasts can also be manually reordered with drag-and-drop. I’ve also rethought the old stacked “Podcasts” and “Played Podcasts” sections to better match people’s needs and expectations. Now, the toggle atop the podcast list switches between three modes: podcasts with current episodes, all followed podcasts, and inactive podcasts (those that you don’t follow and therefore won’t get any more episodes from, or haven’t posted a new episode in a long time). The playlist screen’s structure remains mostly the same, while refining the design for the modern era: Playlist screen, before (left) and after (right). Here, it’s more apparent that I’ve replaced the system San Francisco font with an alternate variant, San Francisco Rounded, to increase legibility and better match the personality of the app. I’ve also added highly demanded features: By far, Overcast’s most-requested feature is a Mark as Played feature. That’s now available as a checkmark button on episode rows, as well as a left-side swipe action. The second-most-requested feature is a way to view all starred episodes. Special playlists for Starred, Downloaded, and In Progress can now be created. The light and dark themes now each have a customizable tint color from the modern iOS UI-color palette, including these favorites from beta testers: And throughout the app, I’ve made tons of tweaks and bug fixes, including: Notifications and background downloads are now much more reliable. Episode downloads can now be individually deleted or re-downloaded. Links can now be opened in Safari. (under Nitpicky Details) Performance is now significantly better with very large playlists and collections. Fixed bugs with episode-duration detection, CarPlay lists, Mac-app sharing, and much more. So much is better in this update that I can’t even remember it all. Thank you so much to everyone who helped me beta-test this massive update. As always, Overcast is free in the App Store. Go get it!
Losing Steve affected me more than it probably should have, given that I never met him or had any correspondence with him. But losing him was devastating — not just to my world, but the world. He was a sort of virtual father figure: I was always hoping that maybe Steve would notice something I did. We all wanted his attention and approval, and that drove us to do better work — even those of us who never worked at Apple. Nobody replaced him in this role. Nobody can. But as an outsider who had no personal relationship with him to mourn, it has been most depressing to consider how much of his work the world missed out on. He wasn’t taken from us after a long, complete life — he was taken in his prime. He had so much more to offer the world.
After the dust settles from the developer class-action settlement, the South Korean law, the JFTC announcement, and the Apple v. Epic decision, I think the most likely long-term outcome isn’t very different from the status quo — and that’s a good thing. Allowing external purchases Here’s what I think we’ll end up with: Apple will still require apps to use their IAP system for any qualifying purchases that occur in the apps themselves. All app types will be allowed to link out to a browser for other purchase methods. Most apps will be required to also offer IAP side-by-side with any external methods.1 Only “Reader apps” will be exempt from this requirement.2 Apple will have many rules regarding the display, descriptions, and behavior of external purchases, many of which will be unpublished and ever-changing. App Review will be extremely harsh, inconsistent, capricious, petty, and punitive with their enforcement.3 Apple won’t require price-matching between IAP and external purchases. These few but important corrections reduce Apple’s worst behavior and should relieve most regulatory pressure. The result won’t look much different than the status quo: Most big media apps (qualifying as “reader” apps) won’t offer IAP, but will finally be allowed to link to their websites from their apps and offer purchases there. Many games will offer both IAP and external purchases, with the external choice offering a discount, bonus gems, extra loot boxes, or other manipulative tricks to optimize the profitability of casino games for children (commissions from which have been the largest portion of Apple’s “services revenue” to date). Most importantly, many products, services, and business models will become possible that previously weren’t, leading to more apps, more competition, and more money going to more places. External purchase methods will evolve to be almost as convenient as IAP (especially if Apple Pay is permitted in this context), and payment processors will reduce the burden of manual credit-card entry with shared credentials available across multiple apps. The payment-fraud doomsday scenarios argued by Apple and many fans mostly won’t happen, in part because App Review will prevent most obvious cases, but also because parents don’t typically offer their credit cards to untrustworthy children; and for buyers of all ages, most credit cards themselves provide stronger fraud prevention and easier recourse from unwanted charges than the App Store ever has. No side-loading I don’t expect side-loading or alternative app stores to become possible, and I’m relieved, because that is not a future I want for iOS. When evaluating such ideas, I merely ask myself: “What would Facebook do?” Facebook owns four of the top ten apps in the world. If side-loading became possible, Facebook could remove Instagram, WhatsApp, the Facebook app, and Messenger from Apple’s App Store, requiring customers to install these extremely popular apps directly from Facebook via side-loading. And everyone would. Most people use a Facebook-owned app not because it’s a good app, but because it’s a means to an important end in their life. Social pressure, family pressure, and network lock-in prevent most users from seeking meaningful alternatives. People would jump through a few hoops if they had to. Facebook would soon have apps that bypassed App Review installed on the majority of iPhones in the world. Technical limitations of the OS would prevent the most egregious abuses, but there’s a lot they could still do. We don’t need to do much imagining — they already have attempted multiple hacks, workarounds, privacy invasions, and other unscrupulous and technically invasive behavior with their apps over time to surveil user behavior outside of their app and stay running longer in the background than users intend or expect. The OS could evolve over time to reduce some of these vulnerabilities, but technical measures alone cannot address all of them. Without the threat of App Review to keep them in check, Facebook’s apps would become even more monstrous than they already are. As a user and a fan of iOS, I don’t want any part of that. No alternative app stores Alternative app stores would be even worse. Rather than offering individual apps via side-loading, Facebook could offer just one: The Facebook App Store. Instagram, WhatsApp, the Facebook app, and Messenger could all be available exclusively there. The majority of iOS users in the world would soon install it, and Facebook would start using leverage in other areas — apps’ social accounts, stats packages, app-install ads, ad-attribution requirements — to heavily incentivize (and likely strong-arm) a huge number of developers to offer their apps in the Facebook App Store, likely in addition to Apple’s. Maybe I’d be required to add the Facebook SDK to my app in order to be in their store, which they would then use to surveil my users. Maybe I’d need to buy app-install ads to show up in search there at all. Maybe I’d need to pay Facebook to “promote” each app update to reach more than a tiny percentage of my existing customers. And Facebook wouldn’t even be the only app store likely to become a large player on iOS. Amazon would almost certainly bring their garbage “Appstore” to iOS, but at least that one probably wouldn’t go anywhere. Maybe Google would bring the Play Store to iOS and offer a unified SDK to develop a single codebase for iOS and Android, effectively making every app feel like an Android app and further marginalizing native apps when they’re already hurting. Media conglomerates that own many big-name properties, like Disney, might each have their own app stores for their high-profile apps. Running your own store means you can promote all of your own apps as much as you want. What giant corporation would resist? Don’t forget games! Epic and Steam would come to iOS with their own game stores. Maybe Microsoft and Nintendo, too. Maybe you’d need to install seven different app stores on your iPhone just to get the apps and games you already use — and all without App Review to keep them in check. Most developers would probably need to start submitting our apps to multiple app stores, each with its own rules, metadata, technical requirements, capabilities, approval delays, payment processing, stats, crash reports, ads, promotion methods, and user reviews. As a user, a multiple-app-store world sounds like an annoying mess; as a developer, it terrifies me. Apple’s App Store is the devil we know. The most viable alternatives that would crop up would be far worse. Course correction The way Apple runs its business isn’t perfect, but it’s also not a democracy. I loved this part of Judge Yvonne Gonzalez Rogers’ decision in Apple v. Epic, as quoted by Ben Thompson’s excellent article that you should read: Apple has not offered any justification for the actions other than to argue entitlement. Where its actions harm competition and result in supracompetitive pricing and profits, Apple is wrong. I interpret “entitlement” without a negative connotation here — Apple is entitled to run their platform mostly as they wish, with governmental interference only warranted to fix market-scale issues that harm large segments of commerce or society. As a developer, I’d love to see more changes to Apple’s control over iOS. But it’s hard to make larger changes without potentially harming much of what makes iOS great for both users and developers. Judge Gonzalez Rogers got it right: we needed a minor course correction to address the most egregiously anticompetitive behavior, but most of the way Apple runs iOS is best left to Apple. If the South Korean law holds, IAP may not be required — but only in South Korea. With this exception, I expect the rest of these rules to be enforced the same way globally. ↩︎ Apple defines “reader” apps as “[allowing] a user to access previously purchased content or content subscriptions (specifically: magazines, newspapers, books, audio, music, and video).” This includes many apps that Apple’s services compete with, such as Netflix, Spotify, and Kindle, that raise anticompetitive concerns among regulators and legislators when forced to give Apple 30%. ↩︎ App Review has higher-level queues for managerial review of controversial rules or edge cases, typically identifiable from the outside by an app stuck with “In Review” status for days or weeks, and often ending in a phone call from “Bill”. I’d expect any app offering external purchases to have a very high chance of being escalated to a slower, more pain-in-the-ass review process, possibly causing it not to be worthwhile for most small developers to deal with. I have no plans to add external purchases to Overcast for multiple reasons, including this — but mostly because, for my purposes, I’m satisfied with Apple’s IAP system. ↩︎
Apple’s leaders continue to deny developers of two obvious truths: That our apps provide substantial value to iOS beyond the purchase commissions collected by Apple. That any portion of our customers came to our apps from our own marketing or reputation, rather than the App Store. For Apple to continue to deny these is dishonest, factually wrong, and extremely insulting — not only to our efforts, but to the intelligence of all Apple developers and customers. This isn’t about the 30%, or the 15%, or the prohibition of other payment systems, or the rules against telling our customers about our websites, or Apple’s many other restrictions. (Not today, at least.) It’s about what Apple’s leadership thinks of us and our work. * * * It isn’t the App Store’s responsibility to the rest of Apple to “pay its way” by leveraging hefty fees on certain types of transactions. Modern society has come to rely so heavily on mobile apps that any phone manufacturer must ensure that such a healthy ecosystem exists as table stakes for anyone to buy their phones. Without our apps, the iPhone has little value to most of its customers today. If Apple wishes to continue advancing bizarre corporate-accounting arguments, the massive profits from the hardware business are what therefore truly “pay the way” of the App Store, public APIs, developer tools, and other app-development resources, just as the hardware profits must fund the development of Apple’s own hardware, software, and services that make the iPhone appeal to customers. The forced App Store commissions, annual developer fees, and App Store Search Ads income are all just gravy. The “way” is already paid by the hardware — but Apple uses their position of power to double-dip. And that’s just business. Apple’s a lot of things, and “generous” isn’t one. But to bully and gaslight developers into thinking that we need to be kissing Apple’s feet for permitting us to add billions of dollars of value to their platform is not only greedy, stingy, and morally reprehensible, but deeply insulting. * * * Apple further extends the value argument, and defends their justification for forced commissions, by claiming responsibility for and ownership of the customer relationship between all iOS users and each app they choose to use. This argument only makes sense — and even then, only somewhat — when apps are installed by a customer browsing the App Store, finding an app they hadn’t previously heard of, and choosing to install it based on App Store influence alone. But in the common case — and for most app installations, the much more common case — of searching for a specific app by name or following a link or ad based on its developer’s own marketing or reputation, Apple has served no meaningful role in the customer acquisition and “deserves” nothing more from the transaction than what a CDN and commodity credit-card processor would charge. The idea that the App Store is responsible for most customers of any reasonably well-known app is a fantasy. It isn’t the App Store that has enabled all of the commerce on iOS — it’s the entire world of computing and modern society, created by a symbiotic ecosystem in which Apple played one part alongside many others. The world was already moving in this direction, and had Apple not played its part, someone else would’ve. The App Store is merely one platform’s forced distribution gateway, “facilitating” the commerce no more and no less than a web browser, an ISP or cellular carrier, a server-hosting company, or a credit-card processor. For Apple to continue to claim otherwise is beyond insulting, and borders on delusion. * * * At WWDC next week, these same people are going to try to tell us a different story. They’re going to tell us how amazing we are, how important our work is, and how much they value us. And for thousands of Apple employees who’ve made the great products and platforms that we love, including the hundreds of engineers presenting the sessions and working the labs, it’ll be genuine and true. But the leaders have already shown us who they really are, what they really think of us, and how much they value our work. Please forgive some sloppiness in my metaphors or phrasing — my writing skills are pretty rusty — and I’ll return the favor to anyone who responds.
More in programming
I like the job title “Design Engineer”. When required to label myself, I feel partial to that term (I should, I’ve written about it enough). Lately I’ve felt like the term is becoming more mainstream which, don’t get me wrong, is a good thing. I appreciate the diversification of job titles, especially ones that look to stand in the middle between two binaries. But — and I admit this is a me issue — once a title starts becoming mainstream, I want to use it less and less. I was never totally sure why I felt this way. Shouldn’t I be happy a title I prefer is gaining acceptance and understanding? Do I just want to rebel against being labeled? Why do I feel this way? These were the thoughts simmering in the back of my head when I came across an interview with the comedian Brian Regan where he talks about his own penchant for not wanting to be easily defined: I’ve tried over the years to write away from how people are starting to define me. As soon as I start feeling like people are saying “this is what you do” then I would be like “Alright, I don't want to be just that. I want to be more interesting. I want to have more perspectives.” [For example] I used to crouch around on stage all the time and people would go “Oh, he’s the guy who crouches around back and forth.” And I’m like, “I’ll show them, I will stand erect! Now what are you going to say?” And then they would go “You’re the guy who always feels stupid.” So I started [doing other things]. He continues, wondering aloud whether this aversion to not being easily defined has actually hurt his career in terms of commercial growth: I never wanted to be something you could easily define. I think, in some ways, that it’s held me back. I have a nice following, but I’m not huge. There are people who are huge, who are great, and deserve to be huge. I’ve never had that and sometimes I wonder, ”Well maybe it’s because I purposely don’t want to be a particular thing you can advertise or push.” That struck a chord with me. It puts into words my current feelings towards the job title “Design Engineer” — or any job title for that matter. Seven or so years ago, I would’ve enthusiastically said, “I’m a Design Engineer!” To which many folks would’ve said, “What’s that?” But today I hesitate. If I say “I’m a Design Engineer” there are less follow up questions. Now-a-days that title elicits less questions and more (presumed) certainty. I think I enjoy a title that elicits a “What’s that?” response, which allows me to explain myself in more than two or three words, without being put in a box. But once a title becomes mainstream, once people begin to assume they know what it means, I don’t like it anymore (speaking for myself, personally). As Brian says, I like to be difficult to define. I want to have more perspectives. I like a title that befuddles, that doesn’t provide a presumed sense of certainty about who I am and what I do. And I get it, that runs counter to the very purpose of a job title which is why I don’t think it’s good for your career to have the attitude I do, lol. I think my own career evolution has gone something like what Brian describes: Them: “Oh you’re a Designer? So you make mock-ups in Photoshop and somebody else implements them.” Me: “I’ll show them, I’ll implement them myself! Now what are you gonna do?” Them: “Oh, so you’re a Design Engineer? You design and build user interfaces on the front-end.” Me: “I’ll show them, I’ll write a Node server and setup a database that powers my designs and interactions on the front-end. Now what are they gonna do?” Them: “Oh, well, we I’m not sure we have a term for that yet, maybe Full-stack Design Engineer?” Me: “Oh yeah? I’ll frame up a user problem, interface with stakeholders, explore the solution space with static designs and prototypes, implement a high-fidelity solution, and then be involved in testing, measuring, and refining said solution. What are you gonna call that?” [As you can see, I have some personal issues I need to work through…] As Brian says, I want to be more interesting. I want to have more perspectives. I want to be something that’s not so easily definable, something you can’t sum up in two or three words. I’ve felt this tension my whole career making stuff for the web. I think it has led me to work on smaller teams where boundaries are much more permeable and crossing them is encouraged rather than discouraged. All that said, I get it. I get why titles are useful in certain contexts (corporate hierarchies, recruiting, etc.) where you’re trying to take something as complicated and nuanced as an individual human beings and reduce them to labels that can be categorized in a database. I find myself avoiding those contexts where so much emphasis is placed in the usefulness of those labels. “I’ve never wanted to be something you could easily define” stands at odds with the corporate attitude of, “Here’s the job req. for the role (i.e. cog) we’re looking for.” Email · Mastodon · Bluesky
We received over 2,200 applications for our just-closed junior programmer opening, and now we're going through all of them by hand and by human. No AI screening here. It's a lot of work, but we have a great team who take the work seriously, so in a few weeks, we'll be able to invite a group of finalists to the next phase. This highlights the folly of thinking that what it'll take to land a job like this is some specific list of criteria, though. Yes, you have to present a baseline of relevant markers to even get into consideration, like a great cover letter that doesn't smell like AI slop, promising projects or work experience or educational background, etc. But to actually get the job, you have to be the best of the ones who've applied! It sounds self-evident, maybe, but I see questions time and again about it, so it must not be. Almost every job opening is grading applicants on the curve of everyone who has applied. And the best candidate of the lot gets the job. You can't quantify what that looks like in advance. I'm excited to see who makes it to the final stage. I already hear early whispers that we got some exceptional applicants in this round. It would be great to help counter the narrative that this industry no longer needs juniors. That's simply retarded. However good AI gets, we're always going to need people who know the ins and outs of what the machine comes up with. Maybe not as many, maybe not in the same roles, but it's truly utopian thinking that mankind won't need people capable of vetting the work done by AI in five minutes.
Recently I got a question on formal methods1: how does it help to mathematically model systems when the system requirements are constantly changing? It doesn't make sense to spend a lot of time proving a design works, and then deliver the product and find out it's not at all what the client needs. As the saying goes, the hard part is "building the right thing", not "building the thing right". One possible response: "why write tests"? You shouldn't write tests, especially lots of unit tests ahead of time, if you might just throw them all away when the requirements change. This is a bad response because we all know the difference between writing tests and formal methods: testing is easy and FM is hard. Testing requires low cost for moderate correctness, FM requires high(ish) cost for high correctness. And when requirements are constantly changing, "high(ish) cost" isn't affordable and "high correctness" isn't worthwhile, because a kinda-okay solution that solves a customer's problem is infinitely better than a solid solution that doesn't. But eventually you get something that solves the problem, and what then? Most of us don't work for Google, we can't axe features and products on a whim. If the client is happy with your solution, you are expected to support it. It should work when your customers run into new edge cases, or migrate all their computers to the next OS version, or expand into a market with shoddy internet. It should work when 10x as many customers are using 10x as many features. It should work when you add new features that come into conflict. And just as importantly, it should never stop solving their problem. Canonical example: your feature involves processing requested tasks synchronously. At scale, this doesn't work, so to improve latency you make it asynchronous. Now it's eventually consistent, but your customers were depending on it being always consistent. Now it no longer does what they need, and has stopped solving their problems. Every successful requirement met spawns a new requirement: "keep this working". That requirement is permanent, or close enough to decide our long-term strategy. It takes active investment to keep a feature behaving the same as the world around it changes. (Is this all a pretentious of way of saying "software maintenance is hard?" Maybe!) Phase changes In physics there's a concept of a phase transition. To raise the temperature of a gram of liquid water by 1° C, you have to add 4.184 joules of energy.2 This continues until you raise it to 100°C, then it stops. After you've added two thousand joules to that gram, it suddenly turns into steam. The energy of the system changes continuously but the form, or phase, changes discretely. Software isn't physics but the idea works as a metaphor. A certain architecture handles a certain level of load, and past that you need a new architecture. Or a bunch of similar features are independently hardcoded until the system becomes too messy to understand, you remodel the internals into something unified and extendable. etc etc etc. It's doesn't have to be totally discrete phase transition, but there's definitely a "before" and "after" in the system form. Phase changes tend to lead to more intricacy/complexity in the system, meaning it's likely that a phase change will introduce new bugs into existing behaviors. Take the synchronous vs asynchronous case. A very simple toy model of synchronous updates would be Set(key, val), which updates data[key] to val.3 A model of asynchronous updates would be AsyncSet(key, val, priority) adds a (key, val, priority, server_time()) tuple to a tasks set, and then another process asynchronously pulls a tuple (ordered by highest priority, then earliest time) and calls Set(key, val). Here are some properties the client may need preserved as a requirement: If AsyncSet(key, val, _, _) is called, then eventually db[key] = val (possibly violated if higher-priority tasks keep coming in) If someone calls AsyncSet(key1, val1, low) and then AsyncSet(key2, val2, low), they should see the first update and then the second (linearizability, possibly violated if the requests go to different servers with different clock times) If someone calls AsyncSet(key, val, _) and immediately reads db[key] they should get val (obviously violated, though the client may accept a slightly weaker property) If the new system doesn't satisfy an existing customer requirement, it's prudent to fix the bug before releasing the new system. The customer doesn't notice or care that your system underwent a phase change. They'll just see that one day your product solves their problems, and the next day it suddenly doesn't. This is one of the most common applications of formal methods. Both of those systems, and every one of those properties, is formally specifiable in a specification language. We can then automatically check that the new system satisfies the existing properties, and from there do things like automatically generate test suites. This does take a lot of work, so if your requirements are constantly changing, FM may not be worth the investment. But eventually requirements stop changing, and then you're stuck with them forever. That's where models shine. As always, I'm using formal methods to mean the subdiscipline of formal specification of designs, leaving out the formal verification of code. Mostly because "formal specification" is really awkward to say. ↩ Also called a "calorie". The US "dietary Calorie" is actually a kilocalorie. ↩ This is all directly translatable to a TLA+ specification, I'm just describing it in English to avoid paying the syntax tax ↩
While Stripe is a widely admired company for things like its creation of the Sorbet typer project, I personally think that Stripe’s most interesting strategy work is also among its most subtle: its willingness to significantly prioritize API stability. This strategy is almost invisible externally. Internally, discussions around it were frequent and detailed, but mostly confined to dedicated API design conversations. API stability isn’t just a technical design quirk, it’s a foundational decision in an API-driven business, and I believe it is one of the unsung heroes of Stripe’s business success. This is an exploratory, draft chapter for a book on engineering strategy that I’m brainstorming in #eng-strategy-book. As such, some of the links go to other draft chapters, both published drafts and very early, unpublished drafts. Reading this document To apply this strategy, start at the top with Policy. To understand the thinking behind this strategy, read sections in reverse order, starting with Explore. More detail on this structure in Making a readable Engineering Strategy document. Policy & Operation Our policies for managing API changes are: Design for long API lifetime. APIs are not inherently durable. Instead we have to design thoughtfully to ensure they can support change. When designing a new API, build a test application that doesn’t use this API, then migrate to the new API. Consider how integrations might evolve as applications change. Perform these migrations yourself to understand potential friction with your API. Then think about the future changes that we might want to implement on our end. How would those changes impact the API, and how would they impact the application you’ve developed. At this point, take your API to API Review for initial approval as described below. Following that approval, identify a handful of early adopter companies who can place additional pressure on your API design, and test with them before releasing the final, stable API. All new and modified APIs must be approved by API Review. API changes may not be enabled for customers prior to API Review approval. Change requests should be sent to api-review email group. For examples of prior art, review the api-review archive for prior requests and the feedback they received. All requests must include a written proposal. Most requests will be approved asynchronously by a member of API Review. Complex or controversial proposals will require live discussions to ensure API Review members have sufficient context before making a decision. We never deprecate APIs without an unavoidable requirement to do so. Even if it’s technically expensive to maintain support, we incur that support cost. To be explicit, we define API deprecation as any change that would require customers to modify an existing integration. If such a change were to be approved as an exception to this policy, it must first be approved by the API Review, followed by our CEO. One example where we granted an exception was the deprecation of TLS 1.2 support due to PCI compliance obligations. When significant new functionality is required, we add a new API. For example, we created /v1/subscriptions to support those workflows rather than extending /v1/charges to add subscriptions support. With the benefit of hindsight, a good example of this policy in action was the introduction of the Payment Intents APIs to maintain compliance with Europe’s Strong Customer Authentication requirements. Even in that case the charge API continued to work as it did previously, albeit only for non-European Union payments. We manage this policy’s implied technical debt via an API translation layer. We release changed APIs into versions, tracked in our API version changelog. However, we only maintain one implementation internally, which is the implementation of the latest version of the API. On top of that implementation, a series of version transformations are maintained, which allow us to support prior versions without maintaining them directly. While this approach doesn’t eliminate the overhead of supporting multiple API versions, it significantly reduces complexity by enabling us to maintain just a single, modern implementation internally. All API modifications must also update the version transformation layers to allow the new version to coexist peacefully with prior versions. In the future, SDKs may allow us to soften this policy. While a significant number of our customers have direct integrations with our APIs, that number has dropped significantly over time. Instead, most new integrations are performed via one of our official API SDKs. We believe that in the future, it may be possible for us to make more backwards incompatible changes because we can absorb the complexity of migrations into the SDKs we provide. That is certainly not the case yet today. Diagnosis Our diagnosis of the impact on API changes and deprecation on our business is: If you are a small startup composed of mostly engineers, integrating a new payments API seems easy. However, for a small business without dedicated engineers—or a larger enterprise involving numerous stakeholders—handling external API changes can be particularly challenging. Even if this is only marginally true, we’ve modeled the impact of minimizing API changes on long-term revenue growth, and it has a significant impact, unlocking our ability to benefit from other churn reduction work. While we believe API instability directly creates churn, we also believe that API stability directly retains customers by increasing the migration overhead even if they wanted to change providers. Without an API change forcing them to change their integration, we believe that hypergrowth customers are particularly unlikely to change payments API providers absent a concrete motivation like an API change or a payment plan change. We are aware of relatively few companies that provide long-term API stability in general, and particularly few for complex, dynamic areas like payments APIs. We can’t assume that companies that make API changes are ill-informed. Rather it appears that they experience a meaningful technical debt tradeoff between the API provider and API consumers, and aren’t willing to consistently absorb that technical debt internally. Future compliance or security requirements—along the lines of our upgrade from TLS 1.2 to TLS 1.3 for PCI—may necessitate API changes. There may also be new tradeoffs exposed as we enter new markets with their own compliance regimes. However, we have limited ability to predict these changes at this point.
I've been biking in Brooklyn for a few years now! It's hard for me to believe it, but I'm now one of the people other bicyclists ask questions to now. I decided to make a zine that answers the most common of those questions: Bike Brooklyn! is a zine that touches on everything I wish I knew when I started biking in Brooklyn. A lot of this information can be found in other resources, but I wanted to collect it in one place. I hope to update this zine when we get significantly more safe bike infrastructure in Brooklyn and laws change to make streets safer for bicyclists (and everyone) over time, but it's still important to note that each release will reflect a specific snapshot in time of bicycling in Brooklyn. All text and illustrations in the zine are my own. Thank you to Matt Denys, Geoffrey Thomas, Alex Morano, Saskia Haegens, Vishnu Reddy, Ben Turndorf, Thomas Nayem-Huzij, and Ryan Christman for suggestions for content and help with proofreading. This zine is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, so you can copy and distribute this zine for noncommercial purposes in unadapted form as long as you give credit to me. Check out the Bike Brooklyn! zine on the web or download pdfs to read digitally or print here!