Full Width [alt+shift+f] Shortcuts [alt+shift+k]
Sign Up [alt+shift+s] Log In [alt+shift+l]
16
elementary OS 7 has been out for just over a month now and I’m excited to announce the first round of updates to our latest operating system includes both great feature updates and a long list of fixes for reported issues. As you may know, we prioritize our work based on your feedback so a big thanks to everyone sending us reports on GitHub and through the Feedback app. Files The headlining feature of this Files release is the new app menu in the headerbar. Jeremy put together this menu to better improve discoverability for features like zoom and undo/redo as well as to clean up folder context menus. The Undo and Redo buttons include tooltips showing what operation will be performed before you click them and we also updated the description for the double-click setting to make it more clear. Files now has an app menu with app-wide settings and features We also fixed a number of reported issues including some off click behavior above and below text in the list view, case sensitivity in...
over a year ago

Improve your reading experience

Logged in users get linked directly to articles resulting in a better reading experience. Please login for free, it takes less than 1 minute.

More from elementary Blog

Happy Holidays! We Come Bringing Gifts!

It’s only been a little over 2 weeks since we released elementary OS 8, but we’re already back with updates just in time for the holidays! Terminal The headliner this month is Terminal which comes with a bunch of fixes and new features thanks to Jeremy. It now uses the more modern tab bar widget you’re used to from Web, Files, and Code. There’s an overlay bar that shows the current zoom level when it changes. We do a better job of handling URIs which contain spaces. And we now show unsafe paste warnings for Drag n Drop operations. Plus, we now show the unsafe paste warning for more commands like doas thanks to Elsie and there’s a new option in the gear menu to toggle unsafe paste alerts thanks to Stella and Charlie. Michal upped the contrast for gray in our default style and Igor made sure we focus the relevant tab when notifications are clicked. Plus, we now replace notifications from the same tab and withdraw notifications when a tab is focused, so your notification center should be a lot less noisy. This release was really a group effort with several new contributors, so major shoutouts to everyone who worked on it! AppCenter AppCenter will use Dark Mode screenshots when available Thanks to Italo, AppCenter will now use provided dark mode screenshots and brand colors when developers provide them. Plus, he addressed a visual bug with release notes. And Juan added support for the latest Appstream Developer tag, so we’re staying up on standards. Window Manager & Dock In the Window Manager, Leo fixed an issue where the dock could sometimes still be clicked when hidden in the Classic session, while Leonhard contributed some performance improvements. In the Dock, Leonhard made sure launcher bounces don’t run too long for apps that don’t notify on startup. Leo fixed an issue where launchers with large icons could become clipped while they bounce and made sure running indicators have a bit more room to breath. Plus the dock now also respects the “Panel Translucency” setting, making it completely solid when requested for added contrast. System Settings Alain added some visual polish to the System view as well as a new progress bar that represents how close we are to meeting our monthly sponsorship goal. Plus Leonhard made sure automatic updates won’t download on metered networks, and we avoid checking for system updates altogether in Demo Mode. We now show monthly funding goal progress right in System Settings You can now prevent Apps from sending notifications from Applications → Permissions, even for apps that don’t report their notification usage in Notification settings. and the check mark next to the current language in Language & Region settings will now follow your accent color thanks to Leo. Installation & Onboarding David fixed a crash with certain partitioning schemes in the Installer’s custom install view, and the encrypt view was simplified. Onboarding will now always stay centered on the screen, even when resized. Icon Browser A new version of the Icon Browser for app developers is available in AppCenter that includes the latest icons for Platform 8 as well as a quick button for copying code snippets thanks to Ryo. And we now focus the search automatically when you start typing, thanks to Alain. And More You can now close the captive network assistant with the keyboard shortcut Ctrl + Q, thanks to Stanisław. Alain fixed copying screenshots to the clipboard. And there a ton of translation updates, especially including traditional Chinese thanks to Kisaragi. Sponsors At the moment we’re at 22% of our monthly funding goal and 430 Sponsors on GitHub! Shoutouts to everyone helping us reach our goals here. Your monthly sponsorship funds development and makes sure we have the resources we need to give you the best version of elementary OS we can! Monthly release candidate builds and daily Early Access builds are available to GitHub Sponsors from any tier! Beware that Early Access builds are not considered stable and you will encounter fresh issues when you run them. We’d really appreciate reporting any problems you encounter with the Feedback app or directly on GitHub.

3 months ago 57 votes
elementary OS 8 Available Today

We’re proud to announce that elementary OS 8 is available to download later today and shipping on several high-quality computers! With OS 8, we’ve focused in on: Creating a new Secure Session that ensures applications respect your privacy and require your consent A brand new Dock with productive multitasking and window management features Empowering our diverse community through Inclusive Design To get elementary OS 8, head to elementary.io later today for the download—or read on for an overview of what’s new. Privacy, Security & Consent Over the past several years we’ve been building features to improve the trust relationship with your computer by requiring your explicit informed consent and disallowing untrustworthy behavior on a technical level. We’ve done that by embracing Flatpak as the way to install apps on elementary OS and Portals for confining them to a safer sandbox. Now we’re extending that story with both new settings to put you in control of the system features apps can access and a new Secure Session powered by Wayland. In the Secure Session apps need your explicit permission for more things On the lock screen, you’ll now see a gear menu next to the password field that gives you the option of Classic or Secure sessions. If you select the Secure Session, elementary OS will use Wayland, a modern and secure method for apps to draw themselves and accept your input. In the Secure Session, apps will be more restricted and will require your consent for access to system features. When an app wants to listen in the background for your keystrokes, take a screenshot, record the screen, or even pick up the color from a single pixel, you will be asked first to make sure that it’s okay. The Secure Session also comes with other modern features like support for Mixed DPI modes—A hotly requested feature for folks using a HiDPI notebook or tablet with a LoDPI external display—and improved support for multi-touch gestures on touch screens and tablets. You might also experience improved performance and smoothness, especially on low-powered hardware. OS 8 will use the Classic Session by default and apps will work and behave as they always have Portals are the standardized system interfaces that apps use to access features in a way that respects your privacy and requires your explicit consent. Four new Portals are now supported in OS 8: Color Picker, Screenshot, Screencast, and Wallpaper. These Portals are essential for enabling modern apps to work in the Secure Session when they don’t have direct access to the pixels on your display. Since some apps haven’t yet made use of the Portals required to operate under the Secure Session, OS 8 will continue to use the Classic Session by default. Apps will work and behave as they always have there, with the same level of system access you’re used to from OS 7 and before. If you rely on certain accessibility features, you may find that those are not yet available under the new Secure Session as well. However, we highly encourage you to give the Secure Session a try and you might be surprised to find that the apps and features you use are already compatible. System Settings → Applications has expanded options Application settings has an all-new design that expands your control over permissions. We now support adjusting the run-time permissions in Flatpak’s Permissions Store—these are set when an app explicitly asks for your permission to access a feature while it’s running. So if you’ve previously denied an app access to run in the background or granted an app permission to set the wallpaper, you can change your mind at any time and adjust permissions here. We’ve also adjusted the language of install-time permissions—aka sandbox holes—to be more clear that these represent advanced system access and the implications of adjusting them. Plus the descriptions of several individual items were changed based on your feedback to use less technical language. And app permission pages now show the app’s icon and description. Getting Apps You Need & Staying Up to Date In 2017 we shipped AppCenter, the Open Source pay-what-you-can app store and in 2021 we revamped that store to use Flatpak, an app distribution technology that is decentralized by design and makes cross-platform app distribution on Linux-based operating systems a breeze. Since the move to Flatpak, you’ve always had the option to easily sideload apps directly from developers or use entire alternative app stores. In OS 8 we’re expanding your access to apps even further by including the most popular app store for Linux out of the box: Flathub. We’re expanding your access to apps even further by including Flathub out of the box This means you’ll be able to access apps made specifically for elementary OS, apps made for Linux, and popular cross-platform apps like Discord and Spotify all directly from AppCenter without having to manually sideload or configure an alt store. To support this change, we’ve made a few changes to App info pages in AppCenter. We’ve removed the “non-curated” badge based on your feedback and instead show a “Made for elementary OS” badge when appropriate. The links section has also been redesigned, featuring colorful iconography. We now show a Sponsor link for app developers that fund the development of their app using third-party platforms like GitHub or Patreon and we show a link directly to the app’s source code for apps that provide it. With the introduction of the Secure Session and new Portals to support it, expanded permissions settings, and sandbox warnings in AppCenter we feel much more confident in providing this expanded app access out of the box while upholding the expectation that the apps you get from AppCenter are reasonably safe, will ask for your consent, and respect your privacy. In elementary OS there are two different kinds of updates. Updates to the operating system itself are installed offline, when your computer restarts, to make sure services are restarted correctly and to prevent issues. Updates to apps, on the other hand, are quickly installed while your computer is running. In OS 7, both of these types of updates appear side-by-side in AppCenter, but in OS 8 operating system updates will now appear in System Settings. Operating system updates now appear in System Settings Splitting apart these two update systems makes it faster to check for updates, more reliable to install them, and clearer which updates will require a restart: updates in AppCenter will never require a restart, while updates in System Settings will always require a restart. Updates in AppCenter will never require a restart, while updates in System Settings will always require a restart. The new system updates mechanism is super fast and includes the option to download updates automatically—which you can now opt-in to during Onboarding. It will also let you know if the updates package contains security updates and has improved error handling if things go wrong. Plus there are new options in the system shutdown dialog so you can install updates before shutting down or choose to skip a pending update, even when automatic updates are enabled. Multitasking & Window Management When planning for the Secure Session we realized that our Dock would need to be completely rewritten. So we took the opportunity a few years ago to run a survey and get better insights into the way you multitask on elementary OS and other operating systems. We then combined those new insights with the feedback we’ve received in GitHub over the years and carefully reconsidered the role of the Dock in our desktop alongside other desktop features which have appeared over the years. This has resulted in a Dock that retains the features you love from OS 7 and before and introduces whole new features to improve your multitasking workflow. Cassidy James Blaede Former Co-founder & CXO Thu, Jan 27, 2022 15 min read In particular, we’ve revisited the way we handle multi-window apps and made the behavior of clicking app icons more predictable. When an app isn’t open yet, a single-click of its icon will still launch it. When an app has a single window open, a single-click will always focus that window, even switching workspaces if necessary. And, when an app has multiple windows open, a single-click will show a window spread so you can quickly select the right window, even outside of the Multitasking View. In this way, a single-click always takes you to an app window instead of sometimes opening a new window or even hiding windows. When an app has multiple windows, clicking shows a window spread For apps that support multiple windows, we’ve implemented a new system that is aware of the FreeDesktop.org standard for hinting this feature, so we can now reliably open new windows when middle-clicking an app’s icon. Plus you can still scroll over an app icon to cycle through open windows. And, you can now launch pinned apps with ⌘ + 1­—9, a hotly requested feature. We’ve also added several new optional multitasking features including the ability to switch between windows with a horizontal swipe gesture, the ability to disable hotcorners when on a workspace that contains a fullscreen app, and the ability to switch between workspaces by scrolling over the panel Designing for Inclusivity We sat down this summer with self-described fully-blind cybersecurity enthusiast Florian Beijers to evaluate our experience for blind folks and identify areas of improvement. A particular showstopper we noticed was keyboard navigation and screen reader support during Onboarding, which has now been completely rewritten. We also took a second look at keyboard navigation and screen reader support during Installation and Initial Setup and the entire first run experience has been much improved for blind folks in OS 8. We also now have screen reader support in the Alt + Tab window switcher and we’ve made sure that there’s audio—or visual depending on your settings—feedback when we’re unable to complete window management tasks like cycling workspaces in response to the keyboard shortcut. Navigation has been rewritten in Onboarding System Settings has been refreshed with a modern space-saving dual-pane design that is more responsive for small and large displays. We’ve also vastly improved support for text scaling, screen readers, keyboard navigation, right-to-left language layouts, and improved contrast in illustrations. Plus search now returns more relevant results and the titles of those results now reflect both the exact setting name they’re matching and the path to that setting. Instead of removing features during this redesign, we’ve added new ones. For example, if you’re not a fan of overlaid scrollbars or have a motor disability that makes them difficult to use, there’s a new setting to always show scrollbars in Desktop → Appearance. Language & Region settings has a new option to automatically select the temperature unit based on locale. And there are new keyboard shortcut options for switching between keyboard layouts or using features like emoji or unicode typing. Instead of removing features during this redesign, we’ve added new ones Settings that use dropdowns are now frequently searchable. We’ve also improved setting descriptions, added new ones based on your feedback, and made sure help text is less frequently hidden behind a mouse hover. Plus, System got a redesign of external links similar to the one in AppCenter, with clearer help and documentation links as well as a better call for contributions. Quick Settings improves access to features while reducing clutter OS 8 also brings a new Quick Settings menu that improves access to features while reducing clutter in the panel. We’ve started by combining the accessibility and session menus which contain useful controls, but don’t indicate a change in status. We’ve also added hotly requested controls like Dark Mode and Rotation Lock. Features like the Screen Reader and Onscreen Keyboard are now available from the Quick Settings menu by default, but you can still choose to hide them in System Settings → Desktop → Dock & Panel. By popular demand, we’re making a major change to our default keyboard shortcuts: pressing ⌘ will now open the Applications menu instead of the Shortcuts overlay and ⌘ + Space will now switch keyboard layouts by default. This brings us more in line with the defaults from other desktops and operating systems and will hopefully be more comfortable for folks who rely on these shortcuts to get around. Of course you can always change the ⌘ key behavior and keyboard shortcuts in general in System Settings → Keyboard. Visual design plays a huge role in the appeal of our operating system and elementary has always had a strong identity in using colorful and playful design to convey a sense of friendliness and fun. In OS 8 we’ve maintained our careful balance of learning and evolving while avoiding chasing design trends to retain our unique personality. Pointers are more consistent and make better use of color A perfect example of this is our new pointers. Pointers were completely redrawn to be more consistent, make better use of color, and be more precise. The new design is more fun and playful with softer edges and rounder corners while maintaining high contrast and legibility. The new design feels extremely familiar but also more modern. We have two new wallpapers to share, “A Large Body of Water Surrounded By Mountains” by Peter Thomas and “A Trail of Footprints In The Sand” by David Emrich. Both of these images have been slightly edited for use as wallpapers in elementary OS and are distributed under the permissive Unsplash license. Instead of a plain dark gray background, Multitasking View now features a blurred version of your wallpaper that is adjusted for light and dark modes. Workspace cards now have rounded corners and the switcher at the bottom of the screen has been updated for light and dark modes as well. The Login & Lock Screen also features a blurred background similar to the Multitasking View as well as a larger and bolder clock Several applications have a noticeably more modern design as well. Notably, Videos has a completely redesigned player page and now follows the system light and dark style preference. The new Fonts looks fantastic and has much better performance. And Web 46 brings its own set of performance improvements along with a more minimal appearance. Hardware Support OS 8 includes the latest long-term support Hardware Enablement stack from Ubuntu, including Linux 6.8. We’re also shipping with Pipewire which improves latency and bluetooth audio quality while being architected for the world of sandboxed Flatpak apps running in the Secure Session. This is an especially big deal for folks doing audio production tasks on elementary OS. Drivers moved to System Settings → System Driver management has moved from AppCenter to System Settings → System. The new design for drivers is more in line with how drivers are managed on other operating systems and is easier to work with, especially for hardware that has multiple driver options like NVIDIA® graphics. Power Settings now shows battery charging levels Power settings now shows the charging level and status for both internal batteries and connected battery devices like mice and keyboards. You can also choose to automatically set different power profiles based on whether your device is plugged in or on battery power, and power modes can be quickly changed from the power menu in the panel. Plus the battery icon in the panel will now show much more accurate battery levels for mobile computers. Power modes can be changed from the power menu Get elementary OS 8 elementary OS 8 is available as a pay-what-you-can purchase at elementary.io later today. Localized direct downloads and a torrent magnet link are provided. OS 8 FAQ Download elementary OS 8 OS 8 will receive additional feature and bug fix updates on a monthly schedule that will be reported on here on our blog, so stay tuned for even more updates in the future! Get A New Computer Our hardware retailers Laptop with Linux, Star Labs, and Slimbook are offering elementary OS 8 out of the box starting today! Visit retailers’ individual sites for more information. Shop Devices Special Thanks I want to give special thanks to all of our volunteer contributors for working hard over the last 13 months to make this an incredible release. We set some really ambitious goals and have made major architectural changes to accomplish them that required a lot of planning and coordination. Some of the features landed in this cycle have been years in the making. Our monthly blog posts highlight more of our individual contributors and it’s worth reading through them to admire their passion and dedication. I’m also eternally grateful to our individual Early Access sponsors for providing consistent funding to keep producing our operating system and distributing it under our pay-what-you-can model. We’re funded almost entirely by the good will of individuals without any VC funding or major corporate backing. The only partnerships we have is with our indie hardware vendors. Choosing to support an operating system made by a community like ours is an act of protest in the world we currently find ourselves in and your solidarity means everything.

3 months ago 73 votes
Package Releases Are Almost Done, You Won't Believe What Happens Next!

This month’s update is fairly brief since a lot of what we did last month was minor bug fixes, regression testing, updating metadata, taking screenshots, and releasing packages. We’re getting down to the last few items before we can release elementary OS 8. Read ahead to find out more! OS 7 Updates Just a couple of small OS 7 updates this month! Ryan backported a fix for an issue in AppCenter where the updates page would continue to show the loading screen after it was finished loading. And the latest Network Indicator was released and now shows cellular modems as toggle buttons like it does with other devices. Cellular modems now show as toggle buttons OS 8 Updates Continuing on with our work to vastly improve screen reader support this cycle, Leo implemented the accessibility interface in the Alt + Tab window switcher! Leonhard added a new option to the system shutdown dialog so you can choose to skip a pending update, even when automatic updates are enabled. You can choose to skip updates when shutting down or restarting If you have a mixed-dpi setup—like a HiDPI laptop or tablet and a LoDPI external monitor—You can now set per-display scaling in the Secure Session thanks to Leonhard. And power modes can also now be quickly changed from the power indicator thanks to Subhadeep. Power modes now appear in the power indicator Release Planning Last month we finished releasing nearly every component that makes up elementary OS—over 80 packages. The only thing left is the Login & Lock Screen which is blocked by two small issues. We also have just two more OS patches to complete. Once these issues are resolved and the Login & Lock Screen has a package release, we can build release-candidate images of elementary OS 8 from the stable updates channel—and these builds will be available to Sponsors in Early Access right away. There’s still a couple more issues we want to try to solve before the final public OS 8 release, but we’re very close! As always you can follow along with our progress towards the release of OS 8 in this GitHub project. When this project board is empty, it’s public release time! Sponsors At the moment we’re at 20% of our monthly funding goal and 385 Sponsors on GitHub! Shoutouts to everyone helping us reach our goals here. Your monthly sponsorship funds development and makes sure we have the resources we need to give you the best version of elementary OS we can! Monthly release candidate builds and daily Early Access builds are available to GitHub Sponsors from any tier! Beware that Early Access builds are not considered stable and you will encounter fresh issues when you run them. We’d really appreciate reporting any problems you encounter with the Feedback app or directly on GitHub.

6 months ago 61 votes
Surprise! Big Updates for OS 7 Are Here!

This month we have a bunch of surprise updates for OS 7 and as always a progress update on OS 8. We’re getting very close to releasing the latest version of our operating system and that means releasing new versions of all of the projects we maintain! That means big new versions of apps and new platform features, some of which we’re also able to release as an update for OS 7. Community Just a little follow up on our Discord community: we’re now just over 550 members! It’s quickly becoming a great place to ask questions and get help, share ideas, and generally hang out and chat with other people who use elementary OS or run Pantheon on other Linux distributions. It’s been really fun to watch this server grow and I’m really excited to participate more in a much less formal way with our community. Join us on Discord OS 7 Updates While most of the releases going out at the moment are exclusive to OS 8, there were still a number of significant updates that we were able to release for OS 7! This is in large part due to shipping many of our apps as Flatpak packages, but it also includes a sneaky platform update that you might not even know you needed. Videos Videos has a new modern and minimalist design Videos now sports a more modern and minimal design which is especially apparent on the player page. On the library page it uses larger landscape format thumbnails and the app now follows the system light and dark style preference. The upgrade to GTK4 also brings performance improvements. Major shoutouts to Leonhard for his work modernizing this code base. Videos—like all of our Flatpak apps—is of course also available to download as a Flatpak for folks running Linux distros other than elementary OS: Download Videos as Flatpak Developer Tools Code now uses the LibHandy tab bar widget which brings improved animations and drag-n-drop behavior. The Terminal pane can now open to your preference of project subdirectory by default. The preferences dialog was slightly redesigned to fit more modern platform conventions and improve screen reader compatibility. And we fixed a dozen reported issues! There’s a new setting in Terminal for event alerts on invalid input and we now do a better job saving tab state. Plus man page documentation has been improved. Restoring tabs from last time is now optional in Files and it now supports hiding files and folders via a .hidden file, a feature you may be familiar with from other file manager apps. Special thanks go to Colin, Gustavo, and Jeremy for working on our developer tools. Portals Portals are special API that apps can use to access system features in a way that respects your privacy and requires your explicit consent. Three new Portals are now supported in OS 7: Color Picker, Screenshot, and Screencast. These portals are essential for maintaining compatibility with modern apps which are written to work in a Wayland world and don’t have direct access to the pixels on your display. If you’ve previously experienced trouble using modern color picker or screen recording apps from alt stores like Flathub on elementary OS 7, this update should fix that for you! Thanks to Davidand Leonhard for their work here! And More Music can now open individual audio files from within the app instead of requiring you to open them from within Files and it gains the now-familiar sticky toolbar style when scrolling in the queue. Camera has been updated to use GTK4 which for now simply means improved performance. And a new Tasks release fixes an issue where it would crash when the system style was changed from light to dark. OS 8 Updates We’ve landed a rename of the session options on the Lock Screen to hopefully improve clarity for folks that aren’t sure if they should be using a Wayland or X11 session. The X11 session is now called the “Compatibility Session” since it offers improved compatibility with legacy apps and some accessibility tools. The Wayland session is now called the “Secure Session” since it requires apps to use modern APIs that improve your security and respect your privacy. There was a lot of back-and-forth discussion about the best way to concisely describe these sessions in a non-technical way—we’re aware these descriptions are not perfect—and we think that for now this the best way to sum up the trade offs when selecting a session. In the future this may change as new features may rely on Wayland and the performance benefits of Wayland become more distinct. But for now we want to make sure that folks who rely on the X11 session for certain workflows aren’t being discouraged by a word choice that doesn’t reflect their reality. And speaking of the compatibility session—pending approval of a new window manager protocol—we will be able to ship the new Dock in both the Secure and Compatibility sessions in OS 8. This is particularly great news since the new Dock offers a much better multitasking workflow based on the feedback we gathered in our survey; For those times you may need to switch back to the Compatibility session for certain apps you won’t need to manage disparate dock settings. Navigation in Onboarding has been rewritten for improved accessibility and with a neat progress bar On the heels of some of our recent accessibility work, I’m proud to say that navigation in Onboarding has been rewritten for much improved keyboard navigation and screen reader compatibility. This was a show stopper when Florian took a look at OS 8 in June. Onboarding is such an important part of introducing a new operating system and making sure people new to elementary OS have a great time, so I’m particularly glad to improve this first impression for folks with vision-related disabilities. The Bluetooth Daemon was previously shipped in the same package as the Bluetooth Indicator and it now lives in its own separate package and has its own project. This should both make it clearer which components are responsible for which parts of Bluetooth features on elementary OS and make things a bit easier to maintain. For now features remain completely unchanged and this is purely organizational. We’re also now shipping Font Viewer as a Flatpak app. Previously we had packaged it and released it to AppCenter, but it is now pulled into OS 8 daily by default. This means we can continually ship the latest GNOME Font Viewer in elementary OS built against our Flatpak runtime so that it fits in stylistically. Release Planning Nearly everything is now released in the OS 8 stable repository, which means we’re very close to building stable release candidate quality builds for OS 8 in Early Access. At the moment we’re mainly waiting on the new Window Manager protocol for the Dock in the compatibility session which will unblock releases for the Dock and Panel. As always you can follow along with our progress towards the release of OS 8 in this GitHub project. At this rate we may be looking at a September release of OS 8 if everything goes smoothly; keep your fingers crossed! Sponsors At the moment we’re just above 21% of our monthly funding goal and we’re at 382 Sponsors on GitHub! Shoutouts to everyone helping us reach our goals here. Your monthly sponsorship funds development and makes sure we have the resources we need to give you the best version of elementary OS we can! Monthly release candidate builds and daily Early Access builds are available to GitHub Sponsors from any tier! Beware that Early Access builds are not considered stable and you will encounter fresh issues when you run them. We’d really appreciate reporting any problems you encounter with the Feedback app or directly on GitHub.

7 months ago 61 votes
It's Disability Pride Month! Let's Get Accessible

This month we have several community updates, a couple of Flatpak releases available on OS 7, and plenty of OS 8 news. Disability Pride Month It’s disability pride month, which means making space to talk about how we can build communities and systems that better accommodate people with disabilities. Last year for disability pride month we announced color deficiency assistance filters and a new feature that reads out the keyboard shortcut for the screen reader during Installation & Initial Setup. You might also be familiar with our Curb Cuts initiative that was started a few years ago and how we completed it in OS 7.1. We also expanded the number of places affected by the Reduce Motion setting, increased the upper intensity limit for Night Light, and made sure accommodations were present on the Lock Screen. This year we had the pleasure of being introduced to Florian Biejers who describes himself as a fully blind cybersecurity enthusiast. You may have heard him recently on Linux After Dark talking about accessibility on Linux desktops generally. A couple weeks ago, he took some time to take a look at the state of OS 8 with regards to accessibility for someone who has no vision at all, and I’ll be honest we were definitely humbled by his experience. We took notes, filed issue reports, and I’m proud to say we’ve already taken meaningful steps towards addressing his feedback with fixes across the desktop and in our platform libraries, but there’s clearly a long way for us to go! If you want to follow along or help us address accessibility issues in elementary OS, we’d love your help! We’re tracking issues in this GitHub project. If you discover a new issue—accessibility related or otherwise—we’d love to get your feedback and we have a handy contributor guide to help you file a report here. Community We’ve moved our recommended community chat from Slack to Discord! You may know that we’ve had quite a bit of trouble keeping the invite link for Slack active which has really prevented that community from growing and staying vibrant. There’s also been issues with the policies for Slack’s free tier around keeping/deleting messages and generally we recognize that at its core Slack is a product for workplaces, not for building communities. So after a bit of poking around and discussion, we’re now recommending folks join the community Discord server instead. So far we’re already at over 330 members and it’s been a much more active and lively place than the Slack was. Plus, we have dedicated moderators besides the core team of developers. Thanks to Tiana and Tony for creating this community and being our mods! Join us on Discord {: .button.suggested } Fedora is asking for contributors to join the Pantheon special interest group. If you’re interested in running our desktop environment on Fedora or contributing to maintaining it on Fedora, they’d love to hear from you! OS 7 Updates Photos 8 has been released to AppCenter as a Flatpak app. This means you can continue to receive updates to Photos by installing the Flatpak version from AppCenter even on older versions of elementary OS, and Photos is now easily available to folks running Linux distributions other than elementary OS. This new version of Photos is largely a maintenance release so don’t expect any major design changes or new features, but it now uses the Wallpaper and Open Directory portals improving cross-platform compatibility. The Captive Network Assistant was also updated to version 8. This release contains the update to GTK4 and the latest WebKit which improves its performance and web compatibility as well as a couple of bug fixes. AppCenter Another couple of big design updates landed in AppCenter in June! Pages now draw their own individual headers, which means we can show more contextual controls and have more design freedom. You’ll notice that options related to updates have now moved to the Updates & Installed Apps page, for example. On App info pages, main action buttons like Install and Open are now always available from the headerbar, and when you scroll past an app’s banner a smaller icon and app title will appear. AppCenter has a flatter design where each page has unique headerbar contents The links section of App Info pages has also been redesigned, featuring colorful iconography and an expanded set of supported links. We now show a Sponsor link for apps who monetize outside of AppCenter and we show a link directly to the app’s source code for apps that provide it. Plus we’ve made a ton of cleanups, bug fixes, and performance improvements, especially around updates. And AppCenter now starts much faster thanks to Leonhard. System Settings Locale settings saw the biggest improvements with a new setting for automatically selecting the temperature unit based on locale, fixed freezing while getting advanced permissions, and it will no longer prompt system administrators for a password unnecessarily for setting the system language. Plus we made some improvements to error handling and other feedback. Operating System settings has a redesigned links section System got a redesign of external links similar to the one in AppCenter, with clearer help and documentation links as well as a better call for contributions. Plus, network settings now shows the name of connected wireless networks in the sidebar and we fixed a missing icon for some wireless headphones in Bluetooth settings. And More The Screencast portal landed this past month, meaning screen recording applications are now able to capture the screen in the Wayland session, thanks again to Leonhard. Code now uses the TabBar widget from LibHandy instead of the deprecated widget from Granite, an important step in porting to GTK 4. There’s also been a lot of progress towards using GLib.Action and modernizing menus thanks to Colin. And we have a new GitHub project for tracking issues with hardware that ships with elementary OS, including the StarLite Mk V. Release Planning At this point OS 8 is almost finished! We’re currently making releases for every single one of the nearly 150 repositories that make up Pantheon and elementary OS. This involves things like writing release notes, taking screenshots, checking for regressions and merging last minute fixes etc. Some of these releases will trickle back to OS 7, but many of them will be OS 8 exclusive as they rely on big under-the-hood changes that can’t easily be backported. We’re also putting the finishing touches on our Wayland session and fixing the last few major bugs and regressions there. We’re nearly at parity with the X11 session and I’ve personally been using it every day with only minor issues! But don’t worry the X11 session will be sticking around if you need it. One small concession we’ve needed to make is that we’ll almost certainly be shipping our fork of Plank for use in the X11 session. So the dock experience between Wayland and X11 will be slightly different in this release and you’ll only get the features of the new dock when running under Wayland. As always you can follow along with our progress towards the release of OS 8 in this GitHub project. Hang tight, we’re almost there! Sponsors At the moment we’re just above 21% of our monthly funding goal and we’re at 365 Sponsors on GitHub! Shoutouts to everyone helping us reach our goals here. Your monthly sponsorship funds development and makes sure we have the resources we need to give you the best version of elementary OS we can! We’re also now automatically building monthly release candidate quality stable builds! These builds are created on the 1st of every month and include all stable updates for the current stable OS series. They have not been reviewed by a human, but should usually be of high quality. Monthly release candidate builds and daily Early Access builds are available to GitHub Sponsors from any tier! Beware that Early Access builds are not considered stable and you will encounter fresh issues when you run them. We’d really appreciate reporting any problems you encounter with the Feedback app or directly on GitHub.

8 months ago 58 votes

More in programming

ChatGPT Would be a Decent Policy Advisor

Revealed: How the UK tech secretary uses ChatGPT for policy advice by Chris Stokel-Walker for the New Scientist

11 hours ago 3 votes
Setting policy for strategy.

This book’s introduction started by defining strategy as “making decisions.” Then we dug into exploration, diagnosis, and refinement: three chapters where you could argue that we didn’t decide anything at all. Clarifying the problem to be solved is the prerequisite of effective decision making, but eventually decisions do have to be made. Here in this chapter on policy, and the following chapter on operations, we finally start to actually make some decisions. In this chapter, we’ll dig into: How we define policy, and how setting policy differs from operating policy as discussed in the next chapter The structured steps for setting policy How many policies should you set? Is it preferable to have one policy, many policies, or does it not matter much either way? Recurring kinds of policies that appear frequently in strategies Why it’s valuable to be intentional about your strategy’s altitude, and how engineers and executives generally maintain different altitudes in their strategies Criteria to use for evaluating whether your policies are likely to be impactful How to develop novel policies, and why it’s rare Why having multiple bundles of alternative policies is generally a phase in strategy development that indicates a gap in your diagnosis How policies that ignore constraints sound inspirational, but accomplish little Dealing with ambiguity and uncertainty created by missing strategies from cross-functional stakeholders By the end, you’ll be ready to evaluate why an existing strategy’s policies are struggling to make an impact, and to start iterating on policies for strategy of your own. This is an exploratory, draft chapter for a book on engineering strategy that I’m brainstorming in #eng-strategy-book. As such, some of the links go to other draft chapters, both published drafts and very early, unpublished drafts. What is policy? Policy is interpreting your diagnosis into a concrete plan. That plan will be a collection of decisions, tradeoffs, and approaches. They’ll range from coding practices, to hiring mandates, to architectural decisions, to guidance about how choices are made within your organization. An effective policy solves the entirety of the strategy’s diagnosis, although the diagnosis itself is encouraged to specify which aspects can be ignored. For example, the strategy for working with private equity ownership acknowledges in its diagnosis that they don’t have clear guidance on what kind of reduction to expect: Based on general practice, it seems likely that our new Private Equity ownership will expect us to reduce R&D headcount costs through a reduction. However, we don’t have any concrete details to make a structured decision on this, and our approach would vary significantly depending on the size of the reduction. Faced with that uncertainty, the policy simply acknowledges the ambiguity and commits to reconsider when more information becomes available: We believe our new ownership will provide a specific target for Research and Development (R&D) operating expenses during the upcoming financial year planning. We will revise these policies again once we have explicit targets, and will delay planning around reductions until we have those numbers to avoid running two overlapping processes. There are two frequent points of confusion when creating policies that are worth addressing directly: Policy is a subset of strategy, rather than the entirety of strategy, because policy is only meaningful in the context of the strategy’s diagnosis. For example, the “N-1 backfill policy” makes sense in the context of new, private equity ownership. The policy wouldn’t work well in a rapidly expanding organization. Any strategy without a policy is useless, but you’ll also find policies without context aren’t worth much either. This is particularly unfortunate, because so often strategies are communicated without those critical sections. Policy describes how tradeoffs should be made, but it doesn’t verify how the tradeoffs are actually being made in practice. The next chapter on operations covers how to inspect an organization’s behavior to ensure policies are followed. When reworking a strategy to be more readable, it often makes sense to merge policy and operation sections together. However, when drafting strategy it’s valuable to keep them separate. Yes, you might use a weekly meeting to review whether the policy is being followed, but whether it’s an effective policy is independent of having such a meeting, and what operational mechanisms you use will vary depending on the number of policies you intend to implement. With this definition in mind, now we can move onto the more interesting discussion of how to set policy. How to set policy Every part of writing a strategy feels hard when you’re doing it, but I personally find that writing policy either feels uncomfortably easy or painfully challenging. It’s never a happy medium. Fortunately, the exploration and diagnosis usually come together to make writing your policy simple: although sometimes that simple conclusion may be a difficult one to swallow. The steps I follow to write a strategy’s policy are: Review diagnosis to ensure it captures the most important themes. It doesn’t need to be perfect, but it shouldn’t have omissions so obvious that you can immediately identify them. Select policies that address the diagnosis. Explicitly match each policy to one or more diagnoses that it addresses. Continue adding policies until every diagnosis is covered. This is a broad instruction, but it’s simpler than it sounds because you’ll typically select from policies identified during your exploration phase. However, there certainly is space to tweak those policies, and to reapply familiar policies to new circumstances. If you do find yourself developing a novel policy, there’s a later section in this chapter, Developing novel policies, that addresses that topic in more detail. Consolidate policies in cases where they overlap or adjoin. For example, two policies about specific teams might be generalized into a policy about all teams in the engineering organization. Backtest policy against recent decisions you’ve made. This is particularly effective if you maintain a decision log in your organization. Mine for conflict once again, much as you did in developing your diagnosis. Emphasize feedback from teams and individuals with a different perspective than your own, but don’t wholly eliminate those that you agree with. Just as it’s easy to crowd out opposing views in diagnosis if you don’t solicit their input, it’s possible to accidentally crowd out your own perspective if you anchor too much on others’ perspectives. Consider refinement if you finish writing, and you just aren’t sure your approach works – that’s fine! Return to the refinement phase by deploying one of the refinement techniques to increase your conviction. Remember that we talk about strategy like it’s done in one pass, but almost all real strategy takes many refinement passes. The steps of writing policy are relatively pedestrian, largely because you’ve done so much of the work already in the exploration, diagnosis, and refinement steps. If you skip those phases, you’d likely follow the above steps for writing policy, but the expected quality of the policy itself would be far lower. How many policies? Addressing the entirety of the diagnosis is often complex, which is why most strategies feature a set of policies rather than just one. The strategy for decomposing a monolithic application is not one policy deciding not to decompose, but a series of four policies: Business units should always operate in their own code repository and monolith. New integrations across business unit monoliths should be done using gRPC. Except for new business unit monoliths, we don’t allow new services. Merge existing services into business-unit monoliths where you can. Four isn’t universally the right number either. It’s simply the number that was required to solve that strategy’s diagnosis. With an excellent diagnosis, your policies will often feel inevitable, and perhaps even boring. That’s great: what makes a policy good is that it’s effective, not that it’s novel or inspiring. Kinds of policies While there are so many policies you can write, I’ve found they generally fall into one of four major categories: approvals, allocations, direction, and guidance. This section introduces those categories. Approvals define the process for making a recurring decision. This might require invoking an architecture advice process, or it might require involving an authority figure like an executive. In the Index post-acquisition integration strategy, there were a number of complex decisions to be made, and the approval mechanism was: Escalations come to paired leads: given our limited shared context across teams, all escalations must come to both Stripe’s Head of Traffic Engineering and Index’s Head of Engineering. This allowed the acquired and acquiring teams to start building trust between each other by ensuring both were consulted before any decision was finalized. On the other hand, the user data access strategy’s approval strategy was more focused on managing corporate risk: Exceptions must be granted in writing by CISO. While our overarching Engineering Strategy states that we follow an advisory architecture process as described in Facilitating Software Architecture, the customer data access policy is an exception and must be explicitly approved, with documentation, by the CISO. Start that process in the #ciso channel. These two different approval processes had different goals, so they made tradeoffs differently. There are so many ways to tweak approval, allowing for many different tradeoffs between safety, productivity, and trust. Allocations describe how resources are split across multiple potential investments. Allocations are the most concrete statement of organizational priority, and also articulate the organization’s belief about how productivity happens in teams. Some companies believe you go fast by swarming more people onto critical problems. Other companies believe you go fast by forcing teams to solve problems without additional headcount. Both can work, and teach you something important about the company’s beliefs. The strategy on Uber’s service migration has two concrete examples of allocation policies. The first describes the Infrastructure engineering team’s allocation between manual provision tasks and investing into creating a self-service provisioning platform: Constrain manual provisioning allocation to maximize investment in self-service provisioning. The service provisioning team will maintain a fixed allocation of one full time engineer on manual service provisioning tasks. We will move the remaining engineers to work on automation to speed up future service provisioning. This will degrade manual provisioning in the short term, but the alternative is permanently degrading provisioning by the influx of new service requests from newly hired product engineers. The second allocation policy is implicitly noted in this strategy’s diagnosis, where it describes the allocation policy in the Engineering organization’s higher altitude strategy: Within infrastructure engineering, there is a team of four engineers responsible for service provisioning today. While our organization is growing at a similar rate as product engineering, none of that additional headcount is being allocated directly to the team working on service provisioning. We do not anticipate this changing. Allocation policies often create a surprising amount of clarity for the team, and I include them in almost every policy I write either explicitly, or implicitly in a higher altitude strategy. Direction provides explicit instruction on how a decision must be made. This is the right tool when you know where you want to go, and exactly the way that you want to get there. Direction is appropriate for problems you understand clearly, and you value consistency more than empowering individual judgment. Direction works well when you need an unambiguous policy that doesn’t leave room for interpretation. For example, Calm’s policy for working in the monolith: We write all code in the monolith. It has been ambiguous if new code (especially new application code) should be written in our JavaScript monolith, or if all new code must be written in a new service outside of the monolith. This is no longer ambiguous: all new code must be written in the monolith. In the rare case that there is a functional requirement that makes writing in the monolith implausible, then you should seek an exception as described below. In that case, the team couldn’t agree on what should go into the monolith. Individuals would often make incompatible decisions, so creating consistency required removing personal judgment from the equation. Sometimes judgment is the issue, and sometimes consistency is difficult due to misaligned incentives. A good example of this comes in strategy on working with new Private Equity ownership: We will move to an “N-1” backfill policy, where departures are backfilled with a less senior level. We will also institute a strict maximum of one Principal Engineer per business unit. It’s likely that hiring managers would simply ignore this backfill policy if it was stated more softly, although sometimes less forceful policies are useful. Guidance provides a recommendation about how a decision should be made. Guidance is useful when there’s enough nuance, ambiguity, or complexity that you can explain the desired destination, but you can’t mandate the path to reaching it. One example of guidance comes from the Index acquisition integration strategy: Minimize changes to tokenization environment: because point-of-sale devices directly work with customer payment details, the API that directly supports the point-of-sale device must live within our secured environment where payment details are stored. However, any other functionality must not be added to our tokenization environment. This might read like direction, but it’s clarifying the desired outcome of avoiding unnecessary complexity in the tokenization environment. However, it’s not able to articulate what complexity is necessary, so ultimately it’s guidance because it requires significant judgment to interpret. A second example of guidance comes in the strategy on decomposing a monolithic codebase: Merge existing services into business-unit monoliths where you can. We believe that each choice to move existing services back into a monolith should be made “in the details” rather than from a top-down strategy perspective. Consequently, we generally encourage teams to wind down their existing services outside of their business unit’s monolith, but defer to teams to make the right decision for their local context. This is another case of knowing the desired outcome, but encountering too much uncertainty to direct the team on how to get there. If you ask five engineers about whether it’s possible to merge a given service back into a monolithic codebase, they’ll probably disagree. That’s fine, and highlights the value of guidance: it makes it possible to make incremental progress in areas where more concrete direction would cause confusion. When you’re working on a strategy’s policy section, it’s important to consider all of these categories. Which feel most natural to use will vary depending on your team and role, but they’re all usable: If you’re a developer productivity team, you might have to lean heavily on guidance in your policies and increased support for that guidance within the details of your platform. If you’re an executive, you might lean heavily on direction. Indeed, you might lean too heavily on direction, where guidance often works better for areas where you understand the direction but not the path. If you’re a product engineering organization, you might have to narrow the scope of your direction to the engineers within that organization to deal with the realities of complex cross-organization dynamics. Finally, if you have a clear approach you want to take that doesn’t fit cleanly into any of these categories, then don’t let this framework dissuade you. Give it a try, and adapt if it doesn’t initially work out. Maintaining strategy altitude The chapter on when to write engineering strategy introduced the concept of strategy altitude, which is being deliberate about where certain kinds of policies are created within your organization. Without repeating that section in its entirety, it’s particularly relevant when you set policy to consider how your new policies eliminate flexibility within your organization. Consider these two somewhat opposing strategies: Stripe’s Sorbet strategy only worked in an organization that enforced the use of a single programming language across (essentially) all teams Uber’s service migration strategy worked well in an organization that was unwilling to enforce consistent programming language adoption across teams Stripe’s organization-altitude policy took away the freedom of individual teams to select their preferred technology stack. In return, they unlocked the ability to centralize investment in a powerful way. Uber went the opposite way, unlocking the ability of teams to pick their preferred technology stack, while significantly reducing their centralized teams’ leverage. Both altitudes make sense. Both have consequences. Criteria for effective policies In The Engineering Executive’s Primer’s chapter on engineering strategy, I introduced three criteria for evaluating policies. They ought to be applicable, enforced, and create leverage. Defining those a bit: Applicable: it can be used to navigate complex, real scenarios, particularly when making tradeoffs. Enforced: teams will be held accountable for following the guiding policy. Create Leverage: create compounding or multiplicative impact. The last of these three, create leverage, made sense in the context of a book about engineering executives, but probably doesn’t make as much sense here. Some policies certainly should create leverage (e.g. empower developer experience team by restricting new services), but others might not (e.g. moving to an N-1 backfill policy). Outside the executive context, what’s important isn’t necessarily creating leverage, but that a policy solves for part of the diagnosis. That leaves the other two–being applicable and enforced–both of which are necessary for a policy to actually address the diagnosis. Any policy which you can’t determine how to apply, or aren’t willing to enforce, simply won’t be useful. Let’s apply these criteria to a handful of potential policies. First let’s think about policies we might write to improve the talent density of our engineering team: “We only hire world-class engineers.” This isn’t applicable, because it’s unclear what a world-class engineer means. Because there’s no mutually agreeable definition in this policy, it’s also not consistently enforceable. “We only hire engineers that get at least one ‘strong yes’ in scorecards.” This is applicable, because there’s a clear definition. This is enforceable, depending on the willingness of the organization to reject seemingly good candidates who don’t happen to get a strong yes. Next, let’s think about a policy regarding code reuse within a codebase: “We follow a strict Don’t Repeat Yourself policy in our codebase.” There’s room for debate within a team about whether two pieces of code are truly duplicative, but this is generally applicable. Because there’s room for debate, it’s a very context specific determination to decide how to enforce a decision. “Code authors are responsible for determining if their contributions violate Don’t Repeat Yourself, and rewriting them if they do.” This is much more applicable, because now there’s only a single person’s judgment to assess the potential repetition. In some ways, this policy is also more enforceable, because there’s no longer any ambiguity around who is deciding whether a piece of code is a repetition. The challenge is that enforceability now depends on one individual, and making this policy effective will require holding individuals accountable for the quality of their judgement. An organization that’s unwilling to distinguish between good and bad judgment won’t get any value out of the policy. This is a good example of how a good policy in one organization might become a poor policy in another. If you ever find yourself wanting to include a policy that for some reason either can’t be applied or can’t be enforced, stop to ask yourself what you’re trying to accomplish and ponder if there’s a different policy that might be better suited to that goal. Developing novel policies My experience is that there are vanishingly few truly novel policies to write. There’s almost always someone else has already done something similar to your intended approach. Calm’s engineering strategy is such a case: the details are particular to the company, but the general approach is common across the industry. The most likely place to find truly novel policies is during the adoption phase of a new widespread technology, such as the rise of ubiquitous mobile phones, cloud computing, or large language models. Even then, as explored in the strategy for adopting large-language models, the new technology can be engaged with as a generic technology: Develop an LLM-backed process for reactivating departed and suspended drivers in mature markets. Through modeling our driver lifecycle, we determined that improving onboarding time will have little impact on the total number of active drivers. Instead, we are focusing on mechanisms to reactivate departed and suspended drivers, which is the only opportunity to meaningfully impact active drivers. You could simply replace “LLM” with “data-driven” and it would be equally readable. In this way, policy can generally sidestep areas of uncertainty by being a bit abstract. This avoids being overly specific about topics you simply don’t know much about. However, even if your policy isn’t novel to the industry, it might still be novel to you or your organization. The steps that I’ve found useful to debug novel policies are the same steps as running a condensed version of the strategy process, with a focus on exploration and refinement: Collect a number of similar policies, with a focus on how those policies differ from the policy you are creating Create a systems model to articulate how this policy will work, and also how it will differ from the similar policies you’re considering Run a strategy testing cycle for your proto-policy to discover any unknown-unknowns about how it works in practice Whether you run into this scenario is largely a function of the extent of your, and your organization’s, experience. Early in my career, I found myself doing novel (for me) strategy work very frequently, and these days I rarely find myself doing novel work, instead focusing on adaptation of well-known policies to new circumstances. Are competing policy proposals an anti-pattern? When creating policy, you’ll often have to engage with the question of whether you should develop one preferred policy or a series of potential strategies to pick from. Developing these is a useful stage of setting policy, but rather than helping you refine your policy, I’d encourage you to think of this as exposing gaps in your diagnosis. For example, when Stripe developed the Sorbet ruby-typing tooling, there was debate between two policies: Should we build a ruby-typing tool to allow a centralized team to gradually migrate the company to a typed codebase? Should we migrate the codebase to a preexisting strongly typed language like Golang or Java? These were, initially, equally valid hypotheses. It was only by clarifying our diagnosis around resourcing that it became clear that incurring the bulk of costs in a centralized team was clearly preferable to spreading the costs across many teams. Specifically, recognizing that we wanted to prioritize short-term product engineering velocity, even if it led to a longer migration overall. If you do develop multiple policy options, I encourage you to move the alternatives into an appendix rather than including them in the core of your strategy document. This will make it easier for readers of your final version to understand how to follow your policies, and they are the most important long-term user of your written strategy. Recognizing constraints A similar problem to competing solutions is developing a policy that you cannot possibly fund. It’s easy to get enamored with policies that you can’t meaningfully enforce, but that’s bad policy, even if it would work in an alternate universe where it was possible to enforce or resource it. To consider a few examples: The strategy for controlling access to user data might have proposed requiring manual approval by a second party of every access to customer data. However, that would have gone nowhere. Our approach to Uber’s service migration might have required more staffing for the infrastructure engineering team, but we knew that wasn’t going to happen, so it was a meaningless policy proposal to make. The strategy for navigating private equity ownership might have argued that new ownership should not hold engineering accountable to a new standard on spending. But they would have just invalidated that strategy in the next financial planning period. If you find a policy that contemplates an impractical approach, it doesn’t only indicate that the policy is a poor one, it also suggests your policy is missing an important pillar. Rather than debating the policy options, the fastest path to resolution is to align on the diagnosis that would invalidate potential paths forward. In cases where aligning on the diagnosis isn’t possible, for example because you simply don’t understand the possibilities of a new technology as encountered in the strategy for adopting LLMs, then you’ve typically found a valuable opportunity to use strategy refinement to build alignment. Dealing with missing strategies At a recent company offsite, we were debating which policies we might adopt to deal with annual plans that kept getting derailed after less than a month. Someone remarked that this would be much easier if we could get the executive team to commit to a clearer, written strategy about which business units we were prioritizing. They were, of course, right. It would be much easier. Unfortunately, it goes back to the problem we discussed in the diagnosis chapter about reframing blockers into diagnosis. If a strategy from the company or a peer function is missing, the empowering thing to do is to include the absence in your diagnosis and move forward. Sometimes, even when you do this, it’s easy to fall back into the belief that you cannot set a policy because a peer function might set a conflicting policy in the future. Whether you’re an executive or an engineer, you’ll never have the details you want to make the ideal policy. Meaningful leadership requires taking meaningful risks, which is never something that gets comfortable. Summary After working through this chapter, you know how to develop policy, how to assemble policies to solve your diagnosis, and how to avoid a number of the frequent challenges that policy writers encounter. At this point, there’s only one phase of strategy left to dig into, operating the policies you’ve created.

16 hours ago 3 votes
Fast and random sampling in SQLite

I was building a small feature for the Flickr Commons Explorer today: show a random selection of photos from the entire collection. I wanted a fast and varied set of photos. This meant getting a random sample of rows from a SQLite table (because the Explorer stores all its data in SQLite). I’m happy with the code I settled on, but it took several attempts to get right. Approach #1: ORDER BY RANDOM() My first attempt was pretty naïve – I used an ORDER BY RANDOM() clause to sort the table, then limit the results: SELECT * FROM photos ORDER BY random() LIMIT 10 This query works, but it was slow – about half a second to sample a table with 2 million photos (which is very small by SQLite standards). This query would run on every request for the homepage, so that latency is unacceptable. It’s slow because it forces SQLite to generate a value for every row, then sort all the rows, and only then does it apply the limit. SQLite is fast, but there’s only so fast you can sort millions of values. I found a suggestion from Stack Overflow user Ali to do a random sort on the id column first, pick my IDs from that, and only fetch the whole row for the photos I’m selecting: SELECT * FROM photos WHERE id IN ( SELECT id FROM photos ORDER BY RANDOM() LIMIT 10 ) This means SQLite only has to load the rows it’s returning, not every row in the database. This query was over three times faster – about 0.15s – but that’s still slower than I wanted. Approach #2: WHERE rowid > (…) Scrolling down the Stack Overflow page, I found an answer by Max Shenfield with a different approach: SELECT * FROM photos WHERE rowid > ( ABS(RANDOM()) % (SELECT max(rowid) FROM photos) ) LIMIT 10 The rowid is a unique identifier that’s used as a primary key in most SQLite tables, and it can be looked up very quickly. SQLite automatically assigns a unique rowid unless you explicitly tell it not to, or create your own integer primary key. This query works by picking a point between the biggest and smallest rowid values used in the table, then getting the rows with rowids which are higher than that point. If you want to know more, Max’s answer has a more detailed explanation. This query is much faster – around 0.0008s – but I didn’t go this route. The result is more like a random slice than a random sample. In my testing, it always returned contiguous rows – 101, 102, 103, … – which isn’t what I want. The photos in the Commons Explorer database were inserted in upload order, so photos with adjacent row IDs were uploaded at around the same time and are probably quite similar. I’d get one photo of an old plane, then nine more photos of other planes. I want more variety! (This behaviour isn’t guaranteed – if you don’t add an ORDER BY clause to a SELECT query, then the order of results is undefined. SQLite is returning rows in rowid order in my table, and a quick Google suggests that’s pretty common, but that may not be true in all cases. It doesn’t affect whether I want to use this approach, but I mention it here because I was confused about the ordering when I read this code.) Approach #3: Select random rowid values outside SQLite Max’s answer was the first time I’d heard of rowid, and it gave me an idea – what if I chose random rowid values outside SQLite? This is a less “pure” approach because I’m not doing everything in the database, but I’m happy with that if it gets the result I want. Here’s the procedure I came up with: Create an empty list to store our sample. Find the highest rowid that’s currently in use: sqlite> SELECT MAX(rowid) FROM photos; 1913389 Use a random number generator to pick a rowid between 1 and the highest rowid: >>> import random >>> random.randint(1, max_rowid) 196476 If we’ve already got this rowid, discard it and generate a new one. (The rowid is a signed, 64-bit integer, so the minimum possible value is always 1.) Look for a row with that rowid: SELECT * FROM photos WHERE rowid = 196476 If such a row exists, add it to our sample. If we have enough items in our sample, we’re done. Otherwise, return to step 3 and generate another rowid. If such a row doesn’t exist, return to step 3 and generate another rowid. This requires a bit more code, but it returns a diverse sample of photos, which is what I really care about. It’s a bit slower, but still plenty fast enough (about 0.001s). This approach is best for tables where the rowid values are mostly contiguous – it would be slower if there are lots of rowids between 1 and the max that don’t exist. If there are large gaps in rowid values, you might try multiple missing entries before finding a valid row, slowing down the query. You might want to try something different, like tracking valid rowid values separately. This is a good fit for my use case, because photos don’t get removed from Flickr Commons very often. Once a row is written, it sticks around, and over 97% of the possible rowid values do exist. Summary Here are the four approaches I tried: Approach Performance (for 2M rows) Notes ORDER BY RANDOM() ~0.5s Slowest, easiest to read WHERE id IN (SELECT id …) ~0.15s Faster, still fairly easy to understand WHERE rowid > ... ~0.0008s Returns clustered results Random rowid in Python ~0.001s Fast and returns varied results, requires code outside SQL I’m using the random rowid in Python in the Commons Explorer, trading code complexity for speed. I’m using this random sample to render a web page, so it’s important that it returns quickly – when I was testing ORDER BY RANDOM(), I could feel myself waiting for the page to load. But I’ve used ORDER BY RANDOM() in the past, especially for asynchronous data pipelines where I don’t care about absolute performance. It’s simpler to read and easier to see what’s going on. Now it’s your turn – visit the Commons Explorer and see what random gems you can find. Let me know if you spot anything cool! [If the formatting of this post looks odd in your feed reader, visit the original article]

7 hours ago 1 votes
Choosing Languages
yesterday 2 votes
05 · Syncing Keyhive

How we sync Keyhive and Automerge

yesterday 1 votes