More from elementary Blog
It’s only been a little over 2 weeks since we released elementary OS 8, but we’re already back with updates just in time for the holidays! Terminal The headliner this month is Terminal which comes with a bunch of fixes and new features thanks to Jeremy. It now uses the more modern tab bar widget you’re used to from Web, Files, and Code. There’s an overlay bar that shows the current zoom level when it changes. We do a better job of handling URIs which contain spaces. And we now show unsafe paste warnings for Drag n Drop operations. Plus, we now show the unsafe paste warning for more commands like doas thanks to Elsie and there’s a new option in the gear menu to toggle unsafe paste alerts thanks to Stella and Charlie. Michal upped the contrast for gray in our default style and Igor made sure we focus the relevant tab when notifications are clicked. Plus, we now replace notifications from the same tab and withdraw notifications when a tab is focused, so your notification center should be a lot less noisy. This release was really a group effort with several new contributors, so major shoutouts to everyone who worked on it! AppCenter AppCenter will use Dark Mode screenshots when available Thanks to Italo, AppCenter will now use provided dark mode screenshots and brand colors when developers provide them. Plus, he addressed a visual bug with release notes. And Juan added support for the latest Appstream Developer tag, so we’re staying up on standards. Window Manager & Dock In the Window Manager, Leo fixed an issue where the dock could sometimes still be clicked when hidden in the Classic session, while Leonhard contributed some performance improvements. In the Dock, Leonhard made sure launcher bounces don’t run too long for apps that don’t notify on startup. Leo fixed an issue where launchers with large icons could become clipped while they bounce and made sure running indicators have a bit more room to breath. Plus the dock now also respects the “Panel Translucency” setting, making it completely solid when requested for added contrast. System Settings Alain added some visual polish to the System view as well as a new progress bar that represents how close we are to meeting our monthly sponsorship goal. Plus Leonhard made sure automatic updates won’t download on metered networks, and we avoid checking for system updates altogether in Demo Mode. We now show monthly funding goal progress right in System Settings You can now prevent Apps from sending notifications from Applications → Permissions, even for apps that don’t report their notification usage in Notification settings. and the check mark next to the current language in Language & Region settings will now follow your accent color thanks to Leo. Installation & Onboarding David fixed a crash with certain partitioning schemes in the Installer’s custom install view, and the encrypt view was simplified. Onboarding will now always stay centered on the screen, even when resized. Icon Browser A new version of the Icon Browser for app developers is available in AppCenter that includes the latest icons for Platform 8 as well as a quick button for copying code snippets thanks to Ryo. And we now focus the search automatically when you start typing, thanks to Alain. And More You can now close the captive network assistant with the keyboard shortcut Ctrl + Q, thanks to Stanisław. Alain fixed copying screenshots to the clipboard. And there a ton of translation updates, especially including traditional Chinese thanks to Kisaragi. Sponsors At the moment we’re at 22% of our monthly funding goal and 430 Sponsors on GitHub! Shoutouts to everyone helping us reach our goals here. Your monthly sponsorship funds development and makes sure we have the resources we need to give you the best version of elementary OS we can! Monthly release candidate builds and daily Early Access builds are available to GitHub Sponsors from any tier! Beware that Early Access builds are not considered stable and you will encounter fresh issues when you run them. We’d really appreciate reporting any problems you encounter with the Feedback app or directly on GitHub.
We’re proud to announce that elementary OS 8 is available to download later today and shipping on several high-quality computers! With OS 8, we’ve focused in on: Creating a new Secure Session that ensures applications respect your privacy and require your consent A brand new Dock with productive multitasking and window management features Empowering our diverse community through Inclusive Design To get elementary OS 8, head to elementary.io later today for the download—or read on for an overview of what’s new. Privacy, Security & Consent Over the past several years we’ve been building features to improve the trust relationship with your computer by requiring your explicit informed consent and disallowing untrustworthy behavior on a technical level. We’ve done that by embracing Flatpak as the way to install apps on elementary OS and Portals for confining them to a safer sandbox. Now we’re extending that story with both new settings to put you in control of the system features apps can access and a new Secure Session powered by Wayland. In the Secure Session apps need your explicit permission for more things On the lock screen, you’ll now see a gear menu next to the password field that gives you the option of Classic or Secure sessions. If you select the Secure Session, elementary OS will use Wayland, a modern and secure method for apps to draw themselves and accept your input. In the Secure Session, apps will be more restricted and will require your consent for access to system features. When an app wants to listen in the background for your keystrokes, take a screenshot, record the screen, or even pick up the color from a single pixel, you will be asked first to make sure that it’s okay. The Secure Session also comes with other modern features like support for Mixed DPI modes—A hotly requested feature for folks using a HiDPI notebook or tablet with a LoDPI external display—and improved support for multi-touch gestures on touch screens and tablets. You might also experience improved performance and smoothness, especially on low-powered hardware. OS 8 will use the Classic Session by default and apps will work and behave as they always have Portals are the standardized system interfaces that apps use to access features in a way that respects your privacy and requires your explicit consent. Four new Portals are now supported in OS 8: Color Picker, Screenshot, Screencast, and Wallpaper. These Portals are essential for enabling modern apps to work in the Secure Session when they don’t have direct access to the pixels on your display. Since some apps haven’t yet made use of the Portals required to operate under the Secure Session, OS 8 will continue to use the Classic Session by default. Apps will work and behave as they always have there, with the same level of system access you’re used to from OS 7 and before. If you rely on certain accessibility features, you may find that those are not yet available under the new Secure Session as well. However, we highly encourage you to give the Secure Session a try and you might be surprised to find that the apps and features you use are already compatible. System Settings → Applications has expanded options Application settings has an all-new design that expands your control over permissions. We now support adjusting the run-time permissions in Flatpak’s Permissions Store—these are set when an app explicitly asks for your permission to access a feature while it’s running. So if you’ve previously denied an app access to run in the background or granted an app permission to set the wallpaper, you can change your mind at any time and adjust permissions here. We’ve also adjusted the language of install-time permissions—aka sandbox holes—to be more clear that these represent advanced system access and the implications of adjusting them. Plus the descriptions of several individual items were changed based on your feedback to use less technical language. And app permission pages now show the app’s icon and description. Getting Apps You Need & Staying Up to Date In 2017 we shipped AppCenter, the Open Source pay-what-you-can app store and in 2021 we revamped that store to use Flatpak, an app distribution technology that is decentralized by design and makes cross-platform app distribution on Linux-based operating systems a breeze. Since the move to Flatpak, you’ve always had the option to easily sideload apps directly from developers or use entire alternative app stores. In OS 8 we’re expanding your access to apps even further by including the most popular app store for Linux out of the box: Flathub. We’re expanding your access to apps even further by including Flathub out of the box This means you’ll be able to access apps made specifically for elementary OS, apps made for Linux, and popular cross-platform apps like Discord and Spotify all directly from AppCenter without having to manually sideload or configure an alt store. To support this change, we’ve made a few changes to App info pages in AppCenter. We’ve removed the “non-curated” badge based on your feedback and instead show a “Made for elementary OS” badge when appropriate. The links section has also been redesigned, featuring colorful iconography. We now show a Sponsor link for app developers that fund the development of their app using third-party platforms like GitHub or Patreon and we show a link directly to the app’s source code for apps that provide it. With the introduction of the Secure Session and new Portals to support it, expanded permissions settings, and sandbox warnings in AppCenter we feel much more confident in providing this expanded app access out of the box while upholding the expectation that the apps you get from AppCenter are reasonably safe, will ask for your consent, and respect your privacy. In elementary OS there are two different kinds of updates. Updates to the operating system itself are installed offline, when your computer restarts, to make sure services are restarted correctly and to prevent issues. Updates to apps, on the other hand, are quickly installed while your computer is running. In OS 7, both of these types of updates appear side-by-side in AppCenter, but in OS 8 operating system updates will now appear in System Settings. Operating system updates now appear in System Settings Splitting apart these two update systems makes it faster to check for updates, more reliable to install them, and clearer which updates will require a restart: updates in AppCenter will never require a restart, while updates in System Settings will always require a restart. Updates in AppCenter will never require a restart, while updates in System Settings will always require a restart. The new system updates mechanism is super fast and includes the option to download updates automatically—which you can now opt-in to during Onboarding. It will also let you know if the updates package contains security updates and has improved error handling if things go wrong. Plus there are new options in the system shutdown dialog so you can install updates before shutting down or choose to skip a pending update, even when automatic updates are enabled. Multitasking & Window Management When planning for the Secure Session we realized that our Dock would need to be completely rewritten. So we took the opportunity a few years ago to run a survey and get better insights into the way you multitask on elementary OS and other operating systems. We then combined those new insights with the feedback we’ve received in GitHub over the years and carefully reconsidered the role of the Dock in our desktop alongside other desktop features which have appeared over the years. This has resulted in a Dock that retains the features you love from OS 7 and before and introduces whole new features to improve your multitasking workflow. Cassidy James Blaede Former Co-founder & CXO Thu, Jan 27, 2022 15 min read In particular, we’ve revisited the way we handle multi-window apps and made the behavior of clicking app icons more predictable. When an app isn’t open yet, a single-click of its icon will still launch it. When an app has a single window open, a single-click will always focus that window, even switching workspaces if necessary. And, when an app has multiple windows open, a single-click will show a window spread so you can quickly select the right window, even outside of the Multitasking View. In this way, a single-click always takes you to an app window instead of sometimes opening a new window or even hiding windows. When an app has multiple windows, clicking shows a window spread For apps that support multiple windows, we’ve implemented a new system that is aware of the FreeDesktop.org standard for hinting this feature, so we can now reliably open new windows when middle-clicking an app’s icon. Plus you can still scroll over an app icon to cycle through open windows. And, you can now launch pinned apps with ⌘ + 1—9, a hotly requested feature. We’ve also added several new optional multitasking features including the ability to switch between windows with a horizontal swipe gesture, the ability to disable hotcorners when on a workspace that contains a fullscreen app, and the ability to switch between workspaces by scrolling over the panel Designing for Inclusivity We sat down this summer with self-described fully-blind cybersecurity enthusiast Florian Beijers to evaluate our experience for blind folks and identify areas of improvement. A particular showstopper we noticed was keyboard navigation and screen reader support during Onboarding, which has now been completely rewritten. We also took a second look at keyboard navigation and screen reader support during Installation and Initial Setup and the entire first run experience has been much improved for blind folks in OS 8. We also now have screen reader support in the Alt + Tab window switcher and we’ve made sure that there’s audio—or visual depending on your settings—feedback when we’re unable to complete window management tasks like cycling workspaces in response to the keyboard shortcut. Navigation has been rewritten in Onboarding System Settings has been refreshed with a modern space-saving dual-pane design that is more responsive for small and large displays. We’ve also vastly improved support for text scaling, screen readers, keyboard navigation, right-to-left language layouts, and improved contrast in illustrations. Plus search now returns more relevant results and the titles of those results now reflect both the exact setting name they’re matching and the path to that setting. Instead of removing features during this redesign, we’ve added new ones. For example, if you’re not a fan of overlaid scrollbars or have a motor disability that makes them difficult to use, there’s a new setting to always show scrollbars in Desktop → Appearance. Language & Region settings has a new option to automatically select the temperature unit based on locale. And there are new keyboard shortcut options for switching between keyboard layouts or using features like emoji or unicode typing. Instead of removing features during this redesign, we’ve added new ones Settings that use dropdowns are now frequently searchable. We’ve also improved setting descriptions, added new ones based on your feedback, and made sure help text is less frequently hidden behind a mouse hover. Plus, System got a redesign of external links similar to the one in AppCenter, with clearer help and documentation links as well as a better call for contributions. Quick Settings improves access to features while reducing clutter OS 8 also brings a new Quick Settings menu that improves access to features while reducing clutter in the panel. We’ve started by combining the accessibility and session menus which contain useful controls, but don’t indicate a change in status. We’ve also added hotly requested controls like Dark Mode and Rotation Lock. Features like the Screen Reader and Onscreen Keyboard are now available from the Quick Settings menu by default, but you can still choose to hide them in System Settings → Desktop → Dock & Panel. By popular demand, we’re making a major change to our default keyboard shortcuts: pressing ⌘ will now open the Applications menu instead of the Shortcuts overlay and ⌘ + Space will now switch keyboard layouts by default. This brings us more in line with the defaults from other desktops and operating systems and will hopefully be more comfortable for folks who rely on these shortcuts to get around. Of course you can always change the ⌘ key behavior and keyboard shortcuts in general in System Settings → Keyboard. Visual design plays a huge role in the appeal of our operating system and elementary has always had a strong identity in using colorful and playful design to convey a sense of friendliness and fun. In OS 8 we’ve maintained our careful balance of learning and evolving while avoiding chasing design trends to retain our unique personality. Pointers are more consistent and make better use of color A perfect example of this is our new pointers. Pointers were completely redrawn to be more consistent, make better use of color, and be more precise. The new design is more fun and playful with softer edges and rounder corners while maintaining high contrast and legibility. The new design feels extremely familiar but also more modern. We have two new wallpapers to share, “A Large Body of Water Surrounded By Mountains” by Peter Thomas and “A Trail of Footprints In The Sand” by David Emrich. Both of these images have been slightly edited for use as wallpapers in elementary OS and are distributed under the permissive Unsplash license. Instead of a plain dark gray background, Multitasking View now features a blurred version of your wallpaper that is adjusted for light and dark modes. Workspace cards now have rounded corners and the switcher at the bottom of the screen has been updated for light and dark modes as well. The Login & Lock Screen also features a blurred background similar to the Multitasking View as well as a larger and bolder clock Several applications have a noticeably more modern design as well. Notably, Videos has a completely redesigned player page and now follows the system light and dark style preference. The new Fonts looks fantastic and has much better performance. And Web 46 brings its own set of performance improvements along with a more minimal appearance. Hardware Support OS 8 includes the latest long-term support Hardware Enablement stack from Ubuntu, including Linux 6.8. We’re also shipping with Pipewire which improves latency and bluetooth audio quality while being architected for the world of sandboxed Flatpak apps running in the Secure Session. This is an especially big deal for folks doing audio production tasks on elementary OS. Drivers moved to System Settings → System Driver management has moved from AppCenter to System Settings → System. The new design for drivers is more in line with how drivers are managed on other operating systems and is easier to work with, especially for hardware that has multiple driver options like NVIDIA® graphics. Power Settings now shows battery charging levels Power settings now shows the charging level and status for both internal batteries and connected battery devices like mice and keyboards. You can also choose to automatically set different power profiles based on whether your device is plugged in or on battery power, and power modes can be quickly changed from the power menu in the panel. Plus the battery icon in the panel will now show much more accurate battery levels for mobile computers. Power modes can be changed from the power menu Get elementary OS 8 elementary OS 8 is available as a pay-what-you-can purchase at elementary.io later today. Localized direct downloads and a torrent magnet link are provided. OS 8 FAQ Download elementary OS 8 OS 8 will receive additional feature and bug fix updates on a monthly schedule that will be reported on here on our blog, so stay tuned for even more updates in the future! Get A New Computer Our hardware retailers Laptop with Linux, Star Labs, and Slimbook are offering elementary OS 8 out of the box starting today! Visit retailers’ individual sites for more information. Shop Devices Special Thanks I want to give special thanks to all of our volunteer contributors for working hard over the last 13 months to make this an incredible release. We set some really ambitious goals and have made major architectural changes to accomplish them that required a lot of planning and coordination. Some of the features landed in this cycle have been years in the making. Our monthly blog posts highlight more of our individual contributors and it’s worth reading through them to admire their passion and dedication. I’m also eternally grateful to our individual Early Access sponsors for providing consistent funding to keep producing our operating system and distributing it under our pay-what-you-can model. We’re funded almost entirely by the good will of individuals without any VC funding or major corporate backing. The only partnerships we have is with our indie hardware vendors. Choosing to support an operating system made by a community like ours is an act of protest in the world we currently find ourselves in and your solidarity means everything.
This month’s update is fairly brief since a lot of what we did last month was minor bug fixes, regression testing, updating metadata, taking screenshots, and releasing packages. We’re getting down to the last few items before we can release elementary OS 8. Read ahead to find out more! OS 7 Updates Just a couple of small OS 7 updates this month! Ryan backported a fix for an issue in AppCenter where the updates page would continue to show the loading screen after it was finished loading. And the latest Network Indicator was released and now shows cellular modems as toggle buttons like it does with other devices. Cellular modems now show as toggle buttons OS 8 Updates Continuing on with our work to vastly improve screen reader support this cycle, Leo implemented the accessibility interface in the Alt + Tab window switcher! Leonhard added a new option to the system shutdown dialog so you can choose to skip a pending update, even when automatic updates are enabled. You can choose to skip updates when shutting down or restarting If you have a mixed-dpi setup—like a HiDPI laptop or tablet and a LoDPI external monitor—You can now set per-display scaling in the Secure Session thanks to Leonhard. And power modes can also now be quickly changed from the power indicator thanks to Subhadeep. Power modes now appear in the power indicator Release Planning Last month we finished releasing nearly every component that makes up elementary OS—over 80 packages. The only thing left is the Login & Lock Screen which is blocked by two small issues. We also have just two more OS patches to complete. Once these issues are resolved and the Login & Lock Screen has a package release, we can build release-candidate images of elementary OS 8 from the stable updates channel—and these builds will be available to Sponsors in Early Access right away. There’s still a couple more issues we want to try to solve before the final public OS 8 release, but we’re very close! As always you can follow along with our progress towards the release of OS 8 in this GitHub project. When this project board is empty, it’s public release time! Sponsors At the moment we’re at 20% of our monthly funding goal and 385 Sponsors on GitHub! Shoutouts to everyone helping us reach our goals here. Your monthly sponsorship funds development and makes sure we have the resources we need to give you the best version of elementary OS we can! Monthly release candidate builds and daily Early Access builds are available to GitHub Sponsors from any tier! Beware that Early Access builds are not considered stable and you will encounter fresh issues when you run them. We’d really appreciate reporting any problems you encounter with the Feedback app or directly on GitHub.
This month we have a bunch of surprise updates for OS 7 and as always a progress update on OS 8. We’re getting very close to releasing the latest version of our operating system and that means releasing new versions of all of the projects we maintain! That means big new versions of apps and new platform features, some of which we’re also able to release as an update for OS 7. Community Just a little follow up on our Discord community: we’re now just over 550 members! It’s quickly becoming a great place to ask questions and get help, share ideas, and generally hang out and chat with other people who use elementary OS or run Pantheon on other Linux distributions. It’s been really fun to watch this server grow and I’m really excited to participate more in a much less formal way with our community. Join us on Discord OS 7 Updates While most of the releases going out at the moment are exclusive to OS 8, there were still a number of significant updates that we were able to release for OS 7! This is in large part due to shipping many of our apps as Flatpak packages, but it also includes a sneaky platform update that you might not even know you needed. Videos Videos has a new modern and minimalist design Videos now sports a more modern and minimal design which is especially apparent on the player page. On the library page it uses larger landscape format thumbnails and the app now follows the system light and dark style preference. The upgrade to GTK4 also brings performance improvements. Major shoutouts to Leonhard for his work modernizing this code base. Videos—like all of our Flatpak apps—is of course also available to download as a Flatpak for folks running Linux distros other than elementary OS: Download Videos as Flatpak Developer Tools Code now uses the LibHandy tab bar widget which brings improved animations and drag-n-drop behavior. The Terminal pane can now open to your preference of project subdirectory by default. The preferences dialog was slightly redesigned to fit more modern platform conventions and improve screen reader compatibility. And we fixed a dozen reported issues! There’s a new setting in Terminal for event alerts on invalid input and we now do a better job saving tab state. Plus man page documentation has been improved. Restoring tabs from last time is now optional in Files and it now supports hiding files and folders via a .hidden file, a feature you may be familiar with from other file manager apps. Special thanks go to Colin, Gustavo, and Jeremy for working on our developer tools. Portals Portals are special API that apps can use to access system features in a way that respects your privacy and requires your explicit consent. Three new Portals are now supported in OS 7: Color Picker, Screenshot, and Screencast. These portals are essential for maintaining compatibility with modern apps which are written to work in a Wayland world and don’t have direct access to the pixels on your display. If you’ve previously experienced trouble using modern color picker or screen recording apps from alt stores like Flathub on elementary OS 7, this update should fix that for you! Thanks to Davidand Leonhard for their work here! And More Music can now open individual audio files from within the app instead of requiring you to open them from within Files and it gains the now-familiar sticky toolbar style when scrolling in the queue. Camera has been updated to use GTK4 which for now simply means improved performance. And a new Tasks release fixes an issue where it would crash when the system style was changed from light to dark. OS 8 Updates We’ve landed a rename of the session options on the Lock Screen to hopefully improve clarity for folks that aren’t sure if they should be using a Wayland or X11 session. The X11 session is now called the “Compatibility Session” since it offers improved compatibility with legacy apps and some accessibility tools. The Wayland session is now called the “Secure Session” since it requires apps to use modern APIs that improve your security and respect your privacy. There was a lot of back-and-forth discussion about the best way to concisely describe these sessions in a non-technical way—we’re aware these descriptions are not perfect—and we think that for now this the best way to sum up the trade offs when selecting a session. In the future this may change as new features may rely on Wayland and the performance benefits of Wayland become more distinct. But for now we want to make sure that folks who rely on the X11 session for certain workflows aren’t being discouraged by a word choice that doesn’t reflect their reality. And speaking of the compatibility session—pending approval of a new window manager protocol—we will be able to ship the new Dock in both the Secure and Compatibility sessions in OS 8. This is particularly great news since the new Dock offers a much better multitasking workflow based on the feedback we gathered in our survey; For those times you may need to switch back to the Compatibility session for certain apps you won’t need to manage disparate dock settings. Navigation in Onboarding has been rewritten for improved accessibility and with a neat progress bar On the heels of some of our recent accessibility work, I’m proud to say that navigation in Onboarding has been rewritten for much improved keyboard navigation and screen reader compatibility. This was a show stopper when Florian took a look at OS 8 in June. Onboarding is such an important part of introducing a new operating system and making sure people new to elementary OS have a great time, so I’m particularly glad to improve this first impression for folks with vision-related disabilities. The Bluetooth Daemon was previously shipped in the same package as the Bluetooth Indicator and it now lives in its own separate package and has its own project. This should both make it clearer which components are responsible for which parts of Bluetooth features on elementary OS and make things a bit easier to maintain. For now features remain completely unchanged and this is purely organizational. We’re also now shipping Font Viewer as a Flatpak app. Previously we had packaged it and released it to AppCenter, but it is now pulled into OS 8 daily by default. This means we can continually ship the latest GNOME Font Viewer in elementary OS built against our Flatpak runtime so that it fits in stylistically. Release Planning Nearly everything is now released in the OS 8 stable repository, which means we’re very close to building stable release candidate quality builds for OS 8 in Early Access. At the moment we’re mainly waiting on the new Window Manager protocol for the Dock in the compatibility session which will unblock releases for the Dock and Panel. As always you can follow along with our progress towards the release of OS 8 in this GitHub project. At this rate we may be looking at a September release of OS 8 if everything goes smoothly; keep your fingers crossed! Sponsors At the moment we’re just above 21% of our monthly funding goal and we’re at 382 Sponsors on GitHub! Shoutouts to everyone helping us reach our goals here. Your monthly sponsorship funds development and makes sure we have the resources we need to give you the best version of elementary OS we can! Monthly release candidate builds and daily Early Access builds are available to GitHub Sponsors from any tier! Beware that Early Access builds are not considered stable and you will encounter fresh issues when you run them. We’d really appreciate reporting any problems you encounter with the Feedback app or directly on GitHub.
More in programming
I took an amazing trip to SE Asia last month, including Angkor Wat. I had a hard time finding good reading or other resources to learn from before I went, in part because Amazon is awash in AI garbage. Here’s some books and podcasts I found useful about the Khmer empire in general and Angkor in particular: Ancient Angkor by Michael Freeman and Claude Jacques. The closest thing to a coffee-table book to preview what you will see. The practical information is outdated but the pictures and descriptions are good. Empire Podcast #185: The God Kings of Angkor Wat by William Dalrymple and Anita Anand. An entertaining and fully detailed account of the Khmer empire. It’s basically an excerpt from Dalrymple’s new book The Golden Road: How Ancient India Transformed the World. Fall of Civilizations Podcast #5: The Khmer Empire by Paul Cooper. Another history, not quite as magically well told as Dalrymple but full of good information. Angkor and the Khmer Civilization by Michael D. Coe. A highly recommended history of the Khmer region. Honestly I found this very dry and too detailed, but I did learn from it. Lonely Planet Pocket Guide: Siem Reap & the Temples of Angkor. We didn’t use this much but it seemed like a useful practical guide. OTOH it dates to 2018 so things have changed. My other advice for visiting Siem Reap and Angkor is: go. It is amazing. Plan for at least two full days of touristing there. Hire a private guide and driver if you can, it is absolutely worth it. (Email me for a recommendation.)
Often you’ll see a disorganized collection of ideas labeled as a “strategy.” Even when they’re dense with ideas, these can be hard to parse, and are a major reason why most engineers will claim their company doesn’t have a clear strategy even though my experience is that all companies follow some strategy, even if it’s undocumented. This chapter lays out a repeatable, structured approach to drafting strategy. It introduces each step of that approach, which are then detailed further in their respective chapters. Here we’ll cover: How these five steps fit together to facilitate creating strategy, especially by preventing practicioners from skipping steps that feel awkward or challenging. Step 1: Exploring the wider industry’s ideas and practices around the strategy you’re working on. Exploration is understanding what recent research might change your approach, and how the state of art has changed since you last tackled a similar problem. Step 2: Diagnosing the details of your problem. It’s hard to slow down to understand your problem clearly before attempting to solve it, but it’s even more difficult to solve anything well without a clear diagnosis. Step 3: Refinement is taking a raw, unproven set of ideas and testing them against reality. Three techniques are introduced to support this validation process: strategy testing, systems modeling, and Wardley mapping. Step 4: Policy makes the tradeoffs and decisions to solve your diagnosis. These can range from specifying how software is architected, to how pull requests are reviewed, to how headcount is allocated within an organization. Step 5: Operations are the concrete mechanisms that translate policy into an active force within your organization. These can be nudges that remind you about code changes without associated tests, or weekly meetings where you study progress on a migration. Whether these steps are sacred or are open to adaptation and experimentation, including when you personally should persevere in attempting steps that don’t feel effective. From this chapter’s launching point, you’ll have the high-level summaries of each step in strategy creation, and can decide where you want to read further. This is an exploratory, draft chapter for a book on engineering strategy that I’m brainstorming in #eng-strategy-book. As such, some of the links go to other draft chapters, both published drafts and very early, unpublished drafts. How the steps become strategy Creating effective strategy is not rote incantation of a formula. You can’t merely follow these steps to guarantee that you’ll create a great strategy. However, I’ve found over and over is that strategies fail more due to avoidable errors than from fundamentally unsound thinking. Busy people skip steps. Especially steps they dislike or have failed at before. These steps are the scaffolding to avoid those errors. By practicing routinely, you’ll build powerful habits and intuition around which approach is most appropriate for the current strategy you’re working on. They also help turn strategy into a community practice that you, your colleagues, and the wider engineering ecosystem can participate in together. Each step is an input that flows into the next step. Your exploration is the foundation of a solid diagnosis. Your diagnosis helps you search the infinite space of policy for what you need now. Operational mechanisms help you turn policy into an active force supporting your strategy rather than an abstract treatise. If you’re skeptical of the steps, you should certainly maintain your skepticism, but do give them a few tries before discarding them entirely. You may also appreciate the discussion in the chapter on bridging between theory and practice when doing strategy. Explore Exploration is the deliberate practice of searching through a strategy’s problem and solution spaces before allowing yourself to commit to a given approach. It’s understanding how other companies and teams have approached similar questions, and whether their approaches might also work well for you. It’s also learning why what brought you so much success at your former employer isn’t necessarily the best solution for your current organization. The Uber service migration strategy used exploration to understand the service ecosystem by reading industry literature: As a starting point, we find it valuable to read Large-scale cluster management at Google with Borg which informed some elements of the approach to Kubernetes, and Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center which describes the Mesos/Aurora approach. It also used a Wardley map to explore the cloud compute ecosystem. For more detail, read the Exploration chapter. Diagnose Diagnosis is your attempt to correctly recognize the context that the strategy needs to solve before deciding on the policies to address that context. Starting from your exploration’s learnings, and your understanding of your current circumstances, building a diagnosis forces you to delay thinking about solutions until you fully understand your problem’s nuances. A diagnosis can be largely data driven, such as the navigating a Private Equity ownership transition strategy: Our Engineering headcount costs have grown by 15% YoY this year, and 18% YoY the prior year. Headcount grew 7% and 9% respectively, with the difference between headcount and headcount costs explained by salary band adjustments (4%), a focus on hiring senior roles (3%), and increased hiring in higher cost geographic regions (1%). It can also be less data driven, instead aiming to summarize a problem, such as the Index acquisition strategy’s summary of the known and unknown elements of the technical integration prior to the acquisition closing: We will need to rapidly integrate the acquired startup to meet this timeline. We only know a small number of details about what this will entail. We do know that point-of-sale devices directly operate on payment details (e.g. the point-of-sale device knows the credit card details of the card it reads). Our compliance obligations restrict such activity to our “tokenization environment”, a highly secured and isolated environment with direct access to payment details. This environment converts payment details into a unique token that other environments can utilize to operate against payment details without the compliance overhead of having direct access to the underlying payment details. The approach, and challenges, of developing a diagnosis are detailed in the Diagnosis chapter. Refine (Test, Map & Model) Strategy refinement is a toolkit of methods to identify which parts of your diagnosis are most important, and verify that your approach to solving the diagnosis actually works. This chapter delves into the details of using three methods in particular: strategy testing, systems modeling, and Wardley mapping. An example of a systems modeling diagram. These techniques are also demonstrated in the strategy case studies, such as the Wardley map of the LLM ecosystem, or the systems model of backfilling roles without downleveling them. For more detail, read the Refinement chapter. Why isn’t refinement earlier (or later)? A frequent point of disagreement is that refinement should occur before the diagnosis. Another is that mapping and modeling are two distinct steps, and mapping should occur before diagnosis, and modeling should occur after policy. A third is that refinement ought to be the final step of strategy, turning the steps into a looping cycle. These are all reasonable observations, so let me unpack my rationale for this structure. By far the biggest risk for most strategies is not that you model too early or map too late, but instead that you simply skip both steps entirely. My foremost concern is minimizing the required investment into mapping and modeling such that more folks do these steps at all. Refining after exploring and diagnosing allows you to concentrate your efforts on a smaller number of load-bearing areas. That said, it’s common to refine many places in your strategy creation. You’re just as likely to have three small refinement steps as one bigger one. Policy Policy is interpreting your diagnosis into a concrete plan. This plan also needs to work, which requires careful study of what’s worked within your company, and what new ideas you’ve discovered while exploring the current problem. Policies can range from providing directional guidance, such as the user data controls strategy’s guidance: Good security discussions don’t frame decisions as a compromise between security and usability. We will pursue multi-dimensional tradeoffs to simultaneously improve security and efficiency. Whenever we frame a discussion on trading off between security and utility, it’s a sign that we are having the wrong discussion, and that we should rethink our approach. We will prioritize mechanisms that can both automatically authorize and automatically document the rationale for accesses to customer data. The most obvious example of this is automatically granting access to a customer support agent for users who have an open support ticket assigned to that agent. (And removing that access when that ticket is reassigned or resolved.) To committing not to make a decision until later, as practiced in the Index acquisition strategy: Defer making a decision regarding the introduction of Java to a later date: the introduction of Java is incompatible with our existing engineering strategy, but at this point we’ve also been unable to align stakeholders on how to address this decision. Further, we see attempting to address this issue as a distraction from our timely goal of launching a joint product within six months. We will take up this discussion after launching the initial release. This chapter further goes into evaluating policies, overcoming ambiguous circumstances that make it difficult to decide on an approach, and developing novel policies. For full detail, read the Policy chapter. Operations Even the best policies have to be interpreted. There will be new circumstances their authors never imagined, and the policies may be in effect long after their authors have left the organization. Operational mechanisms are the concrete implementation of your policy. The simplest mechanisms are an explicit escalation path, as shown in Calm’s product engineering strategy: Exceptions are granted by the CTO, and must be in writing. The above policies are deliberately restrictive. Sometimes they may be wrong, and we will make exceptions to them. However, each exception should be deliberate and grounded in concrete problems we are aligned both on solving and how we solve them. If we all scatter towards our preferred solution, then we’ll create negative leverage for Calm rather than serving as the engine that advances our product. From that starting point, the mechanisms can get far more complex. This chapter works through evaluating mechanisms, composing an operational plan, and the most common sorts of operational mechanisms that I’ve seen across strategies. For more detail, read the Operations chapter. Is the structure sacrosanct? When someone’s struggling to write a strategy document, one of the first tools someone will often recommend is a strategy template. Templates are great: they reduce the ambiguity of an already broad project into something more tractable. If you’re wondering if you should use a template to craft strategy: sure, go ahead! However, I find that well-meaning, thoughtful templates often turn into lumbering, callous documents that serve no one well. The secret to good templates is that someone has to own it, and that person has to care about the template writer first and foremost, rather than the various constituencies that want to insert requirements into the strategy creation process. The security, compliance and cost of your plans matter a lot, but many organizations start to layer in more and more requirements into these sorts of documents until the idea of writing them becomes prohibitively painful. The best advice I can give someone attempting to write strategy, is that you should discard every element of strategy that gets in your way as long as you can explain what that element was intended to accomplish. For example, if you’re drafting a strategy and you don’t find any operational mechanisms that fit. That’s fine, discard that section. Ultimately, the structure is not sacrosanct, it’s the thinking behind the sections that really matter. This topic is explored in more detail in the chapter on Making engineering strategies more readable. Summary Now, you know the foundational steps to conducting strategy. From here, you can dive into the details with the strategy case studies like How should you adopt LLMs? or you can maintain a high altitude starting with how exploration creates the foundation for an effective strategy. Whichever you start with, I encourage yout o eventually work through both to get the full perspective.
Adam Silver has an article titled “Do you trust design advice from ChatGPT?” wherein he prompted the LLM: How do you add hint text to radio buttons? It gave various suggestions, each of which Adam breaks down. Here’s an an example response from ChatGPT: If you want the hint to appear when the user hovers on the radio button, use a tooltip for a cleaner design Adam’s response: ‘If you want’ Design is not about what you want. It’s about what users need. ‘use a tooltip’ If a hint is useful, why hide it behind a difficult-to-use and inaccessible interaction? ‘for a cleaner design’ Design is about clarity, not cleanliness. Adam’s point-by-point breakdowns are excellent. The entire article is a great example of how plausible-sounding ideas can quickly fall apart under scrutiny from an expert who reframes the issue. It’s funny how prevalent this feels in our age of fast-paced information overload. You read an argument and it seems rational — that is, if you don’t think about it too long, which who has the time? But an expert with deep experience can quickly refute these mediocre rationales and offer a more informed perspective that leaves you wondering how you ever nodded along to the original argument in the first place. Humorously, it reminds me of the culture of conspiracy theories where the burden of proof is on you to disprove the bare assertions being made (a time-consuming job). Hence the value of experience (and what’s experience but an investment of time?) to pierce through these kinds of middle-of-the-road rationales. Experience helps clarify and articulate what lesser experience cannot see, let alone articulate. That all leads me back to Adam: ChatGPT pulls unreliable, uninformed and untrustworthy design advice from the internet and delivers it with confidence. I mean you can certainly listen to its advice. But I think it’s better to develop the instinct to ask the right questions and be able to recognise bad advice when you see it. There’s no shortcut to gaining experience. You can’t consume enough content to get it. You have to do. Email · Mastodon · Bluesky
Logic for Programmers v0.8 now out! The new release has minor changes: new formatting for notes and a better introduction to predicates. I would have rolled it all into v0.9 next month but I like the monthly cadence. Get it here! Betteridge's Law of Software Engineering Specialness In There is No Automatic Reset in Engineering, Tim Ottinger asks: Do the other people have to live with January 2013 for the rest of their lives? Or is it only engineering that has to deal with every dirty hack since the beginning of the organization? Betteridge's Law of Headlines says that if a journalism headline ends with a question mark, the answer is probably "no". I propose a similar law relating to software engineering specialness:1 If someone asks if some aspect of software development is truly unique to just software development, the answer is probably "no". Take the idea that "in software, hacks are forever." My favorite example of this comes from a different profession. The Dewey Decimal System hierarchically categorizes books by discipline. For example, Covered Bridges of Pennsylvania has Dewey number 624.37. 6-- is the technology discipline, 62- is engineering, 624 is civil engineering, and 624.3 is "special types of bridges". I have no idea what the last 0.07 means, but you get the picture. Now if you look at the 6-- "technology" breakdown, you'll see that there's no "software" subdiscipline. This is because when Dewey preallocated the whole technology block in 1876. New topics were instead to be added to the 00- "general-knowledge" catch-all. Eventually 005 was assigned to "software development", meaning The C Programming Language lives at 005.133. Incidentally, another late addition to the general knowledge block is 001.9: "controversial knowledge". And that's why my hometown library shelved the C++ books right next to The Mothman Prophecies. How's that for technical debt? If anything, fixing hacks in software is significantly easier than in other fields. This came up when I was interviewing classic engineers. Kludges happened all the time, but "refactoring" them out is expensive. Need to house a machine that's just two inches taller than the room? Guess what, you're cutting a hole in the ceiling. (Even if we restrict the question to other departments in a software company, we can find kludges that are horrible to undo. I once worked for a company which landed an early contract by adding a bespoke support agreement for that one customer. That plagued them for years afterward.) That's not to say that there aren't things that are different about software vs other fields!2 But I think that most of the time, when we say "software development is the only profession that deals with XYZ", it's only because we're ignorant of how those other professions work. Short newsletter because I'm way behind on writing my April Cools. If you're interested in April Cools, you should try it out! I make it way harder on myself than it actually needs to be— everybody else who participates finds it pretty chill. Ottinger caveats it with "engineering, software or otherwise", so I think he knows that other branches of engineering, at least, have kludges. ↩ The "software is different" idea that I'm most sympathetic to is that in software, the tools we use and the products we create are made from the same material. That's unusual at least in classic engineering. Then again, plenty of machinists have made their own lathes and mills! ↩
We're spending just shy of $1.5 million/year on AWS S3 at the moment to host files for Basecamp, HEY, and everything else. The only way we were able to get the pricing that low was by signing a four-year contract. That contract expires this summer, June 30, so that's our departure date for the final leg of our cloud exit. We've already racked the replacement from Pure Storage in our two primary data centers. A combined 18 petabytes, securely replicated a thousand miles apart. It's a gorgeous rack full of blazing-fast NVMe storage modules. Each card in the chassis capable of storing 150TB now. Pure Storage comes with an S3-compatible API, so no need for CEPH, Minio, or any of the other object storage software solutions you might need, if you were trying to do this exercise on commodity hardware. This makes it pretty easy from the app side to do the swap. But there's still work to do. We have to transfer almost six petabytes out of S3. In an earlier age, that egress alone would have cost hundreds of thousands of dollars in fees alone. But now AWS offers a free 60-day egress window for anyone who wants to leave, so that drops the cost to $0. Nice! It takes a while to transfer that much data, though. Even on the fat 40-Gbit pipe we have set aside for the purpose, it'll probably take at least three weeks, once you factor in overhead and some babysitting of the process. That's when it's good to remind ourselves why June 30th matters. And the reminder math pens out in nice, round numbers for easy recollection: If we don't get this done in time, we'll be paying a cool five thousand dollars a day to continue to use S3 (if all the files are still there). Yikes! That's $35,000/week! That's $150,000/month! Pretty serious money for a company of our size. But so are the savings. Over five years, it'll now be almost five million! Maybe even more, depending on the growth in files we need to store for customers. About $1.5 million for the Pure Storage hardware, and a bit less than a million over five years for warranty and support. But those big numbers always seem a bit abstract to me. The idea of paying $5,000/day, if we miss our departure date, is awfully concrete in comparison.