More from elementary Blog
It’s only been a little over 2 weeks since we released elementary OS 8, but we’re already back with updates just in time for the holidays! Terminal The headliner this month is Terminal which comes with a bunch of fixes and new features thanks to Jeremy. It now uses the more modern tab bar widget you’re used to from Web, Files, and Code. There’s an overlay bar that shows the current zoom level when it changes. We do a better job of handling URIs which contain spaces. And we now show unsafe paste warnings for Drag n Drop operations. Plus, we now show the unsafe paste warning for more commands like doas thanks to Elsie and there’s a new option in the gear menu to toggle unsafe paste alerts thanks to Stella and Charlie. Michal upped the contrast for gray in our default style and Igor made sure we focus the relevant tab when notifications are clicked. Plus, we now replace notifications from the same tab and withdraw notifications when a tab is focused, so your notification center should be a lot less noisy. This release was really a group effort with several new contributors, so major shoutouts to everyone who worked on it! AppCenter AppCenter will use Dark Mode screenshots when available Thanks to Italo, AppCenter will now use provided dark mode screenshots and brand colors when developers provide them. Plus, he addressed a visual bug with release notes. And Juan added support for the latest Appstream Developer tag, so we’re staying up on standards. Window Manager & Dock In the Window Manager, Leo fixed an issue where the dock could sometimes still be clicked when hidden in the Classic session, while Leonhard contributed some performance improvements. In the Dock, Leonhard made sure launcher bounces don’t run too long for apps that don’t notify on startup. Leo fixed an issue where launchers with large icons could become clipped while they bounce and made sure running indicators have a bit more room to breath. Plus the dock now also respects the “Panel Translucency” setting, making it completely solid when requested for added contrast. System Settings Alain added some visual polish to the System view as well as a new progress bar that represents how close we are to meeting our monthly sponsorship goal. Plus Leonhard made sure automatic updates won’t download on metered networks, and we avoid checking for system updates altogether in Demo Mode. We now show monthly funding goal progress right in System Settings You can now prevent Apps from sending notifications from Applications → Permissions, even for apps that don’t report their notification usage in Notification settings. and the check mark next to the current language in Language & Region settings will now follow your accent color thanks to Leo. Installation & Onboarding David fixed a crash with certain partitioning schemes in the Installer’s custom install view, and the encrypt view was simplified. Onboarding will now always stay centered on the screen, even when resized. Icon Browser A new version of the Icon Browser for app developers is available in AppCenter that includes the latest icons for Platform 8 as well as a quick button for copying code snippets thanks to Ryo. And we now focus the search automatically when you start typing, thanks to Alain. And More You can now close the captive network assistant with the keyboard shortcut Ctrl + Q, thanks to Stanisław. Alain fixed copying screenshots to the clipboard. And there a ton of translation updates, especially including traditional Chinese thanks to Kisaragi. Sponsors At the moment we’re at 22% of our monthly funding goal and 430 Sponsors on GitHub! Shoutouts to everyone helping us reach our goals here. Your monthly sponsorship funds development and makes sure we have the resources we need to give you the best version of elementary OS we can! Monthly release candidate builds and daily Early Access builds are available to GitHub Sponsors from any tier! Beware that Early Access builds are not considered stable and you will encounter fresh issues when you run them. We’d really appreciate reporting any problems you encounter with the Feedback app or directly on GitHub.
We’re proud to announce that elementary OS 8 is available to download later today and shipping on several high-quality computers! With OS 8, we’ve focused in on: Creating a new Secure Session that ensures applications respect your privacy and require your consent A brand new Dock with productive multitasking and window management features Empowering our diverse community through Inclusive Design To get elementary OS 8, head to elementary.io later today for the download—or read on for an overview of what’s new. Privacy, Security & Consent Over the past several years we’ve been building features to improve the trust relationship with your computer by requiring your explicit informed consent and disallowing untrustworthy behavior on a technical level. We’ve done that by embracing Flatpak as the way to install apps on elementary OS and Portals for confining them to a safer sandbox. Now we’re extending that story with both new settings to put you in control of the system features apps can access and a new Secure Session powered by Wayland. In the Secure Session apps need your explicit permission for more things On the lock screen, you’ll now see a gear menu next to the password field that gives you the option of Classic or Secure sessions. If you select the Secure Session, elementary OS will use Wayland, a modern and secure method for apps to draw themselves and accept your input. In the Secure Session, apps will be more restricted and will require your consent for access to system features. When an app wants to listen in the background for your keystrokes, take a screenshot, record the screen, or even pick up the color from a single pixel, you will be asked first to make sure that it’s okay. The Secure Session also comes with other modern features like support for Mixed DPI modes—A hotly requested feature for folks using a HiDPI notebook or tablet with a LoDPI external display—and improved support for multi-touch gestures on touch screens and tablets. You might also experience improved performance and smoothness, especially on low-powered hardware. OS 8 will use the Classic Session by default and apps will work and behave as they always have Portals are the standardized system interfaces that apps use to access features in a way that respects your privacy and requires your explicit consent. Four new Portals are now supported in OS 8: Color Picker, Screenshot, Screencast, and Wallpaper. These Portals are essential for enabling modern apps to work in the Secure Session when they don’t have direct access to the pixels on your display. Since some apps haven’t yet made use of the Portals required to operate under the Secure Session, OS 8 will continue to use the Classic Session by default. Apps will work and behave as they always have there, with the same level of system access you’re used to from OS 7 and before. If you rely on certain accessibility features, you may find that those are not yet available under the new Secure Session as well. However, we highly encourage you to give the Secure Session a try and you might be surprised to find that the apps and features you use are already compatible. System Settings → Applications has expanded options Application settings has an all-new design that expands your control over permissions. We now support adjusting the run-time permissions in Flatpak’s Permissions Store—these are set when an app explicitly asks for your permission to access a feature while it’s running. So if you’ve previously denied an app access to run in the background or granted an app permission to set the wallpaper, you can change your mind at any time and adjust permissions here. We’ve also adjusted the language of install-time permissions—aka sandbox holes—to be more clear that these represent advanced system access and the implications of adjusting them. Plus the descriptions of several individual items were changed based on your feedback to use less technical language. And app permission pages now show the app’s icon and description. Getting Apps You Need & Staying Up to Date In 2017 we shipped AppCenter, the Open Source pay-what-you-can app store and in 2021 we revamped that store to use Flatpak, an app distribution technology that is decentralized by design and makes cross-platform app distribution on Linux-based operating systems a breeze. Since the move to Flatpak, you’ve always had the option to easily sideload apps directly from developers or use entire alternative app stores. In OS 8 we’re expanding your access to apps even further by including the most popular app store for Linux out of the box: Flathub. We’re expanding your access to apps even further by including Flathub out of the box This means you’ll be able to access apps made specifically for elementary OS, apps made for Linux, and popular cross-platform apps like Discord and Spotify all directly from AppCenter without having to manually sideload or configure an alt store. To support this change, we’ve made a few changes to App info pages in AppCenter. We’ve removed the “non-curated” badge based on your feedback and instead show a “Made for elementary OS” badge when appropriate. The links section has also been redesigned, featuring colorful iconography. We now show a Sponsor link for app developers that fund the development of their app using third-party platforms like GitHub or Patreon and we show a link directly to the app’s source code for apps that provide it. With the introduction of the Secure Session and new Portals to support it, expanded permissions settings, and sandbox warnings in AppCenter we feel much more confident in providing this expanded app access out of the box while upholding the expectation that the apps you get from AppCenter are reasonably safe, will ask for your consent, and respect your privacy. In elementary OS there are two different kinds of updates. Updates to the operating system itself are installed offline, when your computer restarts, to make sure services are restarted correctly and to prevent issues. Updates to apps, on the other hand, are quickly installed while your computer is running. In OS 7, both of these types of updates appear side-by-side in AppCenter, but in OS 8 operating system updates will now appear in System Settings. Operating system updates now appear in System Settings Splitting apart these two update systems makes it faster to check for updates, more reliable to install them, and clearer which updates will require a restart: updates in AppCenter will never require a restart, while updates in System Settings will always require a restart. Updates in AppCenter will never require a restart, while updates in System Settings will always require a restart. The new system updates mechanism is super fast and includes the option to download updates automatically—which you can now opt-in to during Onboarding. It will also let you know if the updates package contains security updates and has improved error handling if things go wrong. Plus there are new options in the system shutdown dialog so you can install updates before shutting down or choose to skip a pending update, even when automatic updates are enabled. Multitasking & Window Management When planning for the Secure Session we realized that our Dock would need to be completely rewritten. So we took the opportunity a few years ago to run a survey and get better insights into the way you multitask on elementary OS and other operating systems. We then combined those new insights with the feedback we’ve received in GitHub over the years and carefully reconsidered the role of the Dock in our desktop alongside other desktop features which have appeared over the years. This has resulted in a Dock that retains the features you love from OS 7 and before and introduces whole new features to improve your multitasking workflow. Cassidy James Blaede Former Co-founder & CXO Thu, Jan 27, 2022 15 min read In particular, we’ve revisited the way we handle multi-window apps and made the behavior of clicking app icons more predictable. When an app isn’t open yet, a single-click of its icon will still launch it. When an app has a single window open, a single-click will always focus that window, even switching workspaces if necessary. And, when an app has multiple windows open, a single-click will show a window spread so you can quickly select the right window, even outside of the Multitasking View. In this way, a single-click always takes you to an app window instead of sometimes opening a new window or even hiding windows. When an app has multiple windows, clicking shows a window spread For apps that support multiple windows, we’ve implemented a new system that is aware of the FreeDesktop.org standard for hinting this feature, so we can now reliably open new windows when middle-clicking an app’s icon. Plus you can still scroll over an app icon to cycle through open windows. And, you can now launch pinned apps with ⌘ + 1—9, a hotly requested feature. We’ve also added several new optional multitasking features including the ability to switch between windows with a horizontal swipe gesture, the ability to disable hotcorners when on a workspace that contains a fullscreen app, and the ability to switch between workspaces by scrolling over the panel Designing for Inclusivity We sat down this summer with self-described fully-blind cybersecurity enthusiast Florian Beijers to evaluate our experience for blind folks and identify areas of improvement. A particular showstopper we noticed was keyboard navigation and screen reader support during Onboarding, which has now been completely rewritten. We also took a second look at keyboard navigation and screen reader support during Installation and Initial Setup and the entire first run experience has been much improved for blind folks in OS 8. We also now have screen reader support in the Alt + Tab window switcher and we’ve made sure that there’s audio—or visual depending on your settings—feedback when we’re unable to complete window management tasks like cycling workspaces in response to the keyboard shortcut. Navigation has been rewritten in Onboarding System Settings has been refreshed with a modern space-saving dual-pane design that is more responsive for small and large displays. We’ve also vastly improved support for text scaling, screen readers, keyboard navigation, right-to-left language layouts, and improved contrast in illustrations. Plus search now returns more relevant results and the titles of those results now reflect both the exact setting name they’re matching and the path to that setting. Instead of removing features during this redesign, we’ve added new ones. For example, if you’re not a fan of overlaid scrollbars or have a motor disability that makes them difficult to use, there’s a new setting to always show scrollbars in Desktop → Appearance. Language & Region settings has a new option to automatically select the temperature unit based on locale. And there are new keyboard shortcut options for switching between keyboard layouts or using features like emoji or unicode typing. Instead of removing features during this redesign, we’ve added new ones Settings that use dropdowns are now frequently searchable. We’ve also improved setting descriptions, added new ones based on your feedback, and made sure help text is less frequently hidden behind a mouse hover. Plus, System got a redesign of external links similar to the one in AppCenter, with clearer help and documentation links as well as a better call for contributions. Quick Settings improves access to features while reducing clutter OS 8 also brings a new Quick Settings menu that improves access to features while reducing clutter in the panel. We’ve started by combining the accessibility and session menus which contain useful controls, but don’t indicate a change in status. We’ve also added hotly requested controls like Dark Mode and Rotation Lock. Features like the Screen Reader and Onscreen Keyboard are now available from the Quick Settings menu by default, but you can still choose to hide them in System Settings → Desktop → Dock & Panel. By popular demand, we’re making a major change to our default keyboard shortcuts: pressing ⌘ will now open the Applications menu instead of the Shortcuts overlay and ⌘ + Space will now switch keyboard layouts by default. This brings us more in line with the defaults from other desktops and operating systems and will hopefully be more comfortable for folks who rely on these shortcuts to get around. Of course you can always change the ⌘ key behavior and keyboard shortcuts in general in System Settings → Keyboard. Visual design plays a huge role in the appeal of our operating system and elementary has always had a strong identity in using colorful and playful design to convey a sense of friendliness and fun. In OS 8 we’ve maintained our careful balance of learning and evolving while avoiding chasing design trends to retain our unique personality. Pointers are more consistent and make better use of color A perfect example of this is our new pointers. Pointers were completely redrawn to be more consistent, make better use of color, and be more precise. The new design is more fun and playful with softer edges and rounder corners while maintaining high contrast and legibility. The new design feels extremely familiar but also more modern. We have two new wallpapers to share, “A Large Body of Water Surrounded By Mountains” by Peter Thomas and “A Trail of Footprints In The Sand” by David Emrich. Both of these images have been slightly edited for use as wallpapers in elementary OS and are distributed under the permissive Unsplash license. Instead of a plain dark gray background, Multitasking View now features a blurred version of your wallpaper that is adjusted for light and dark modes. Workspace cards now have rounded corners and the switcher at the bottom of the screen has been updated for light and dark modes as well. The Login & Lock Screen also features a blurred background similar to the Multitasking View as well as a larger and bolder clock Several applications have a noticeably more modern design as well. Notably, Videos has a completely redesigned player page and now follows the system light and dark style preference. The new Fonts looks fantastic and has much better performance. And Web 46 brings its own set of performance improvements along with a more minimal appearance. Hardware Support OS 8 includes the latest long-term support Hardware Enablement stack from Ubuntu, including Linux 6.8. We’re also shipping with Pipewire which improves latency and bluetooth audio quality while being architected for the world of sandboxed Flatpak apps running in the Secure Session. This is an especially big deal for folks doing audio production tasks on elementary OS. Drivers moved to System Settings → System Driver management has moved from AppCenter to System Settings → System. The new design for drivers is more in line with how drivers are managed on other operating systems and is easier to work with, especially for hardware that has multiple driver options like NVIDIA® graphics. Power Settings now shows battery charging levels Power settings now shows the charging level and status for both internal batteries and connected battery devices like mice and keyboards. You can also choose to automatically set different power profiles based on whether your device is plugged in or on battery power, and power modes can be quickly changed from the power menu in the panel. Plus the battery icon in the panel will now show much more accurate battery levels for mobile computers. Power modes can be changed from the power menu Get elementary OS 8 elementary OS 8 is available as a pay-what-you-can purchase at elementary.io later today. Localized direct downloads and a torrent magnet link are provided. OS 8 FAQ Download elementary OS 8 OS 8 will receive additional feature and bug fix updates on a monthly schedule that will be reported on here on our blog, so stay tuned for even more updates in the future! Get A New Computer Our hardware retailers Laptop with Linux, Star Labs, and Slimbook are offering elementary OS 8 out of the box starting today! Visit retailers’ individual sites for more information. Shop Devices Special Thanks I want to give special thanks to all of our volunteer contributors for working hard over the last 13 months to make this an incredible release. We set some really ambitious goals and have made major architectural changes to accomplish them that required a lot of planning and coordination. Some of the features landed in this cycle have been years in the making. Our monthly blog posts highlight more of our individual contributors and it’s worth reading through them to admire their passion and dedication. I’m also eternally grateful to our individual Early Access sponsors for providing consistent funding to keep producing our operating system and distributing it under our pay-what-you-can model. We’re funded almost entirely by the good will of individuals without any VC funding or major corporate backing. The only partnerships we have is with our indie hardware vendors. Choosing to support an operating system made by a community like ours is an act of protest in the world we currently find ourselves in and your solidarity means everything.
This month we have a bunch of surprise updates for OS 7 and as always a progress update on OS 8. We’re getting very close to releasing the latest version of our operating system and that means releasing new versions of all of the projects we maintain! That means big new versions of apps and new platform features, some of which we’re also able to release as an update for OS 7. Community Just a little follow up on our Discord community: we’re now just over 550 members! It’s quickly becoming a great place to ask questions and get help, share ideas, and generally hang out and chat with other people who use elementary OS or run Pantheon on other Linux distributions. It’s been really fun to watch this server grow and I’m really excited to participate more in a much less formal way with our community. Join us on Discord OS 7 Updates While most of the releases going out at the moment are exclusive to OS 8, there were still a number of significant updates that we were able to release for OS 7! This is in large part due to shipping many of our apps as Flatpak packages, but it also includes a sneaky platform update that you might not even know you needed. Videos Videos has a new modern and minimalist design Videos now sports a more modern and minimal design which is especially apparent on the player page. On the library page it uses larger landscape format thumbnails and the app now follows the system light and dark style preference. The upgrade to GTK4 also brings performance improvements. Major shoutouts to Leonhard for his work modernizing this code base. Videos—like all of our Flatpak apps—is of course also available to download as a Flatpak for folks running Linux distros other than elementary OS: Download Videos as Flatpak Developer Tools Code now uses the LibHandy tab bar widget which brings improved animations and drag-n-drop behavior. The Terminal pane can now open to your preference of project subdirectory by default. The preferences dialog was slightly redesigned to fit more modern platform conventions and improve screen reader compatibility. And we fixed a dozen reported issues! There’s a new setting in Terminal for event alerts on invalid input and we now do a better job saving tab state. Plus man page documentation has been improved. Restoring tabs from last time is now optional in Files and it now supports hiding files and folders via a .hidden file, a feature you may be familiar with from other file manager apps. Special thanks go to Colin, Gustavo, and Jeremy for working on our developer tools. Portals Portals are special API that apps can use to access system features in a way that respects your privacy and requires your explicit consent. Three new Portals are now supported in OS 7: Color Picker, Screenshot, and Screencast. These portals are essential for maintaining compatibility with modern apps which are written to work in a Wayland world and don’t have direct access to the pixels on your display. If you’ve previously experienced trouble using modern color picker or screen recording apps from alt stores like Flathub on elementary OS 7, this update should fix that for you! Thanks to Davidand Leonhard for their work here! And More Music can now open individual audio files from within the app instead of requiring you to open them from within Files and it gains the now-familiar sticky toolbar style when scrolling in the queue. Camera has been updated to use GTK4 which for now simply means improved performance. And a new Tasks release fixes an issue where it would crash when the system style was changed from light to dark. OS 8 Updates We’ve landed a rename of the session options on the Lock Screen to hopefully improve clarity for folks that aren’t sure if they should be using a Wayland or X11 session. The X11 session is now called the “Compatibility Session” since it offers improved compatibility with legacy apps and some accessibility tools. The Wayland session is now called the “Secure Session” since it requires apps to use modern APIs that improve your security and respect your privacy. There was a lot of back-and-forth discussion about the best way to concisely describe these sessions in a non-technical way—we’re aware these descriptions are not perfect—and we think that for now this the best way to sum up the trade offs when selecting a session. In the future this may change as new features may rely on Wayland and the performance benefits of Wayland become more distinct. But for now we want to make sure that folks who rely on the X11 session for certain workflows aren’t being discouraged by a word choice that doesn’t reflect their reality. And speaking of the compatibility session—pending approval of a new window manager protocol—we will be able to ship the new Dock in both the Secure and Compatibility sessions in OS 8. This is particularly great news since the new Dock offers a much better multitasking workflow based on the feedback we gathered in our survey; For those times you may need to switch back to the Compatibility session for certain apps you won’t need to manage disparate dock settings. Navigation in Onboarding has been rewritten for improved accessibility and with a neat progress bar On the heels of some of our recent accessibility work, I’m proud to say that navigation in Onboarding has been rewritten for much improved keyboard navigation and screen reader compatibility. This was a show stopper when Florian took a look at OS 8 in June. Onboarding is such an important part of introducing a new operating system and making sure people new to elementary OS have a great time, so I’m particularly glad to improve this first impression for folks with vision-related disabilities. The Bluetooth Daemon was previously shipped in the same package as the Bluetooth Indicator and it now lives in its own separate package and has its own project. This should both make it clearer which components are responsible for which parts of Bluetooth features on elementary OS and make things a bit easier to maintain. For now features remain completely unchanged and this is purely organizational. We’re also now shipping Font Viewer as a Flatpak app. Previously we had packaged it and released it to AppCenter, but it is now pulled into OS 8 daily by default. This means we can continually ship the latest GNOME Font Viewer in elementary OS built against our Flatpak runtime so that it fits in stylistically. Release Planning Nearly everything is now released in the OS 8 stable repository, which means we’re very close to building stable release candidate quality builds for OS 8 in Early Access. At the moment we’re mainly waiting on the new Window Manager protocol for the Dock in the compatibility session which will unblock releases for the Dock and Panel. As always you can follow along with our progress towards the release of OS 8 in this GitHub project. At this rate we may be looking at a September release of OS 8 if everything goes smoothly; keep your fingers crossed! Sponsors At the moment we’re just above 21% of our monthly funding goal and we’re at 382 Sponsors on GitHub! Shoutouts to everyone helping us reach our goals here. Your monthly sponsorship funds development and makes sure we have the resources we need to give you the best version of elementary OS we can! Monthly release candidate builds and daily Early Access builds are available to GitHub Sponsors from any tier! Beware that Early Access builds are not considered stable and you will encounter fresh issues when you run them. We’d really appreciate reporting any problems you encounter with the Feedback app or directly on GitHub.
This month we have several community updates, a couple of Flatpak releases available on OS 7, and plenty of OS 8 news. Disability Pride Month It’s disability pride month, which means making space to talk about how we can build communities and systems that better accommodate people with disabilities. Last year for disability pride month we announced color deficiency assistance filters and a new feature that reads out the keyboard shortcut for the screen reader during Installation & Initial Setup. You might also be familiar with our Curb Cuts initiative that was started a few years ago and how we completed it in OS 7.1. We also expanded the number of places affected by the Reduce Motion setting, increased the upper intensity limit for Night Light, and made sure accommodations were present on the Lock Screen. This year we had the pleasure of being introduced to Florian Biejers who describes himself as a fully blind cybersecurity enthusiast. You may have heard him recently on Linux After Dark talking about accessibility on Linux desktops generally. A couple weeks ago, he took some time to take a look at the state of OS 8 with regards to accessibility for someone who has no vision at all, and I’ll be honest we were definitely humbled by his experience. We took notes, filed issue reports, and I’m proud to say we’ve already taken meaningful steps towards addressing his feedback with fixes across the desktop and in our platform libraries, but there’s clearly a long way for us to go! If you want to follow along or help us address accessibility issues in elementary OS, we’d love your help! We’re tracking issues in this GitHub project. If you discover a new issue—accessibility related or otherwise—we’d love to get your feedback and we have a handy contributor guide to help you file a report here. Community We’ve moved our recommended community chat from Slack to Discord! You may know that we’ve had quite a bit of trouble keeping the invite link for Slack active which has really prevented that community from growing and staying vibrant. There’s also been issues with the policies for Slack’s free tier around keeping/deleting messages and generally we recognize that at its core Slack is a product for workplaces, not for building communities. So after a bit of poking around and discussion, we’re now recommending folks join the community Discord server instead. So far we’re already at over 330 members and it’s been a much more active and lively place than the Slack was. Plus, we have dedicated moderators besides the core team of developers. Thanks to Tiana and Tony for creating this community and being our mods! Join us on Discord {: .button.suggested } Fedora is asking for contributors to join the Pantheon special interest group. If you’re interested in running our desktop environment on Fedora or contributing to maintaining it on Fedora, they’d love to hear from you! OS 7 Updates Photos 8 has been released to AppCenter as a Flatpak app. This means you can continue to receive updates to Photos by installing the Flatpak version from AppCenter even on older versions of elementary OS, and Photos is now easily available to folks running Linux distributions other than elementary OS. This new version of Photos is largely a maintenance release so don’t expect any major design changes or new features, but it now uses the Wallpaper and Open Directory portals improving cross-platform compatibility. The Captive Network Assistant was also updated to version 8. This release contains the update to GTK4 and the latest WebKit which improves its performance and web compatibility as well as a couple of bug fixes. AppCenter Another couple of big design updates landed in AppCenter in June! Pages now draw their own individual headers, which means we can show more contextual controls and have more design freedom. You’ll notice that options related to updates have now moved to the Updates & Installed Apps page, for example. On App info pages, main action buttons like Install and Open are now always available from the headerbar, and when you scroll past an app’s banner a smaller icon and app title will appear. AppCenter has a flatter design where each page has unique headerbar contents The links section of App Info pages has also been redesigned, featuring colorful iconography and an expanded set of supported links. We now show a Sponsor link for apps who monetize outside of AppCenter and we show a link directly to the app’s source code for apps that provide it. Plus we’ve made a ton of cleanups, bug fixes, and performance improvements, especially around updates. And AppCenter now starts much faster thanks to Leonhard. System Settings Locale settings saw the biggest improvements with a new setting for automatically selecting the temperature unit based on locale, fixed freezing while getting advanced permissions, and it will no longer prompt system administrators for a password unnecessarily for setting the system language. Plus we made some improvements to error handling and other feedback. Operating System settings has a redesigned links section System got a redesign of external links similar to the one in AppCenter, with clearer help and documentation links as well as a better call for contributions. Plus, network settings now shows the name of connected wireless networks in the sidebar and we fixed a missing icon for some wireless headphones in Bluetooth settings. And More The Screencast portal landed this past month, meaning screen recording applications are now able to capture the screen in the Wayland session, thanks again to Leonhard. Code now uses the TabBar widget from LibHandy instead of the deprecated widget from Granite, an important step in porting to GTK 4. There’s also been a lot of progress towards using GLib.Action and modernizing menus thanks to Colin. And we have a new GitHub project for tracking issues with hardware that ships with elementary OS, including the StarLite Mk V. Release Planning At this point OS 8 is almost finished! We’re currently making releases for every single one of the nearly 150 repositories that make up Pantheon and elementary OS. This involves things like writing release notes, taking screenshots, checking for regressions and merging last minute fixes etc. Some of these releases will trickle back to OS 7, but many of them will be OS 8 exclusive as they rely on big under-the-hood changes that can’t easily be backported. We’re also putting the finishing touches on our Wayland session and fixing the last few major bugs and regressions there. We’re nearly at parity with the X11 session and I’ve personally been using it every day with only minor issues! But don’t worry the X11 session will be sticking around if you need it. One small concession we’ve needed to make is that we’ll almost certainly be shipping our fork of Plank for use in the X11 session. So the dock experience between Wayland and X11 will be slightly different in this release and you’ll only get the features of the new dock when running under Wayland. As always you can follow along with our progress towards the release of OS 8 in this GitHub project. Hang tight, we’re almost there! Sponsors At the moment we’re just above 21% of our monthly funding goal and we’re at 365 Sponsors on GitHub! Shoutouts to everyone helping us reach our goals here. Your monthly sponsorship funds development and makes sure we have the resources we need to give you the best version of elementary OS we can! We’re also now automatically building monthly release candidate quality stable builds! These builds are created on the 1st of every month and include all stable updates for the current stable OS series. They have not been reviewed by a human, but should usually be of high quality. Monthly release candidate builds and daily Early Access builds are available to GitHub Sponsors from any tier! Beware that Early Access builds are not considered stable and you will encounter fresh issues when you run them. We’d really appreciate reporting any problems you encounter with the Feedback app or directly on GitHub.
More in programming
This is a re-publishing of a blog post I originally wrote for work, but wanted on my own blog as well. AI is everywhere, and its impressive claims are leading to rapid adoption. At this stage, I’d qualify it as charismatic technology—something that under-delivers on what it promises, but promises so much that the industry still leverages it because we believe it will eventually deliver on these claims. This is a known pattern. In this post, I’ll use the example of automation deployments to go over known patterns and risks in order to provide you with a list of questions to ask about potential AI solutions. I’ll first cover a short list of base assumptions, and then borrow from scholars of cognitive systems engineering and resilience engineering to list said criteria. At the core of it is the idea that when we say we want humans in the loop, it really matters where in the loop they are. My base assumptions The first thing I’m going to say is that we currently do not have Artificial General Intelligence (AGI). I don’t care whether we have it in 2 years or 40 years or never; if I’m looking to deploy a tool (or an agent) that is supposed to do stuff to my production environments, it has to be able to do it now. I am not looking to be impressed, I am looking to make my life and the system better. Another mechanism I want you to keep in mind is something called the context gap. In a nutshell, any model or automation is constructed from a narrow definition of a controlled environment, which can expand as it gains autonomy, but remains limited. By comparison, people in a system start from a broad situation and narrow definitions down and add constraints to make problem-solving tractable. One side starts from a narrow context, and one starts from a wide one—so in practice, with humans and machines, you end up seeing a type of teamwork where one constantly updates the other: The optimal solution of a model is not an optimal solution of a problem unless the model is a perfect representation of the problem, which it never is. — Ackoff (1979, p. 97) Because of that mindset, I will disregard all arguments of “it’s coming soon” and “it’s getting better real fast” and instead frame what current LLM solutions are shaped like: tools and automation. As it turns out, there are lots of studies about ergonomics, tool design, collaborative design, where semi-autonomous components fit into sociotechnical systems, and how they tend to fail. Additionally, I’ll borrow from the framing used by people who study joint cognitive systems: rather than looking only at the abilities of what a single person or tool can do, we’re going to look at the overall performance of the joint system. This is important because if you have a tool that is built to be operated like an autonomous agent, you can get weird results in your integration. You’re essentially building an interface for the wrong kind of component—like using a joystick to ride a bicycle. This lens will assist us in establishing general criteria about where the problems will likely be without having to test for every single one and evaluate them on benchmarks against each other. Questions you'll want to ask The following list of questions is meant to act as reminders—abstracting away all the theory from research papers you’d need to read—to let you think through some of the important stuff your teams should track, whether they are engineers using code generation, SREs using AIOps, or managers and execs making the call to adopt new tooling. Are you better even after the tool is taken away? An interesting warning comes from studying how LLMs function as learning aides. The researchers found that people who trained using LLMs tended to fail tests more when the LLMs were taken away compared to people who never studied with them, except if the prompts were specifically (and successfully) designed to help people learn. Likewise, it’s been known for decades that when automation handles standard challenges, the operators expected to take over when they reach their limits end up worse off and generally require more training to keep the overall system performant. While people can feel like they’re getting better and more productive with tool assistance, it doesn’t necessarily follow that they are learning or improving. Over time, there’s a serious risk that your overall system’s performance will be limited to what the automation can do—because without proper design, people keeping the automation in check will gradually lose the skills they had developed prior. Are you augmenting the person or the computer? Traditionally successful tools tend to work on the principle that they improve the physical or mental abilities of their operator: search tools let you go through more data than you could on your own and shift demands to external memory, a bicycle more effectively transmits force for locomotion, a blind spot alert on your car can extend your ability to pay attention to your surroundings, and so on. Automation that augments users therefore tends to be easier to direct, and sort of extends the person’s abilities, rather than acting based on preset goals and framing. Automation that augments a machine tends to broaden the device’s scope and control by leveraging some known effects of their environment and successfully hiding them away. For software folks, an autoscaling controller is a good example of the latter. Neither is fundamentally better nor worse than the other—but you should figure out what kind of automation you’re getting, because they fail differently. Augmenting the user implies that they can tackle a broader variety of challenges effectively. Augmenting the computers tends to mean that when the component reaches its limits, the challenges are worse for the operator. Is it turning you into a monitor rather than helping build an understanding? If your job is to look at the tool go and then say whether it was doing a good or bad job (and maybe take over if it does a bad job), you’re going to have problems. It has long been known that people adapt to their tools, and automation can create complacency. Self-driving cars that generally self-drive themselves well but still require a monitor are not effectively monitored. Instead, having AI that supports people or adds perspectives to the work an operator is already doing tends to yield better long-term results than patterns where the human learns to mostly delegate and focus elsewhere. (As a side note, this is why I tend to dislike incident summarizers. Don’t make it so people stop trying to piece together what happened! Instead, I prefer seeing tools that look at your summaries to remind you of items you may have forgotten, or that look for linguistic cues that point to biases or reductive points of view.) Does it pigeonhole what you can look at? When evaluating a tool, you should ask questions about where the automation lands: Does it let you look at the world more effectively? Does it tell you where to look in the world? Does it force you to look somewhere specific? Does it tell you to do something specific? Does it force you to do something? This is a bit of a hybrid between “Does it extend you?” and “Is it turning you into a monitor?” The five questions above let you figure that out. As the tool becomes a source of assertions or constraints (rather than a source of information and options), the operator becomes someone who interacts with the world from inside the tool rather than someone who interacts with the world with the tool’s help. The tool stops being a tool and becomes a representation of the whole system, which means whatever limitations and internal constraints it has are then transmitted to your users. Is it a built-in distraction? People tend to do multiple tasks over many contexts. Some automated systems are built with alarms or alerts that require stealing someone’s focus, and unless they truly are the most critical thing their users could give attention to, they are going to be an annoyance that can lower the effectiveness of the overall system. What perspectives does it bake in? Tools tend to embody a given perspective. For example, AIOps tools that are built to find a root cause will likely carry the conceptual framework behind root causes in their design. More subtly, these perspectives are sometimes hidden in the type of data you get: if your AIOps agent can only see alerts, your telemetry data, and maybe your code, it will rarely be a source of suggestions on how to improve your workflows because that isn’t part of its world. In roles that are inherently about pulling context from many disconnected sources, how on earth is automation going to make the right decisions? And moreover, who’s accountable for when it makes a poor decision on incomplete data? Surely not the buyer who installed it! This is also one of the many ways in which automation can reinforce biases—not just based on what is in its training data, but also based on its own structure and what inputs were considered most important at design time. The tool can itself become a keyhole through which your conclusions are guided. Is it going to become a hero? A common trope in incident response is heroes—the few people who know everything inside and out, and who end up being necessary bottlenecks to all emergencies. They can’t go away for vacation, they’re too busy to train others, they develop blind spots that nobody can fix, and they can’t be replaced. To avoid this, you have to maintain a continuous awareness of who knows what, and crosstrain each other to always have enough redundancy. If you have a team of multiple engineers and you add AI to it, having it do all of the tasks of a specific kind means it becomes a de facto hero to your team. If that’s okay, be aware that any outages or dysfunction in the AI agent would likely have no practical workaround. You will essentially have offshored part of your ops. Do you need it to be perfect? What a thing promises to be is never what it is—otherwise AWS would be enough, and Kubernetes would be enough, and JIRA would be enough, and the software would work fine with no one needing to fix things. That just doesn’t happen. Ever. Even if it’s really, really good, it’s gonna have outages and surprises, and it’ll mess up here and there, no matter what it is. We aren’t building an omnipotent computer god, we’re building imperfect software. You’ll want to seriously consider whether the tradeoffs you’d make in terms of quality and cost are worth it, and this is going to be a case-by-case basis. Just be careful not to fix the problem by adding a human in the loop that acts as a monitor! Is it doing the whole job or a fraction of it? We don’t notice major parts of our own jobs because they feel natural. A classic pattern here is one of AIs getting better at diagnosing patients, except the benchmarks are usually run on a patient chart where most of the relevant observations have already been made by someone else. Similarly, we often see AI pass a test with flying colors while it still can’t be productive at the job the test represents. People in general have adopted a model of cognition based on information processing that’s very similar to how computers work (get data in, think, output stuff, rinse and repeat), but for decades, there have been multiple disciplines that looked harder at situated work and cognition, moving past that model. Key patterns of cognition are not just in the mind, but are also embedded in the environment and in the interactions we have with each other. Be wary of acquiring a solution that solves what you think the problem is rather than what it actually is. We routinely show we don’t accurately know the latter. What if we have more than one? You probably know how straightforward it can be to write a toy project on your own, with full control of every refactor. You probably also know how this stops being true as your team grows. As it stands today, a lot of AI agents are built within a snapshot of the current world: one or few AI tools added to teams that are mostly made up of people. By analogy, this would be like everyone selling you a computer assuming it were the first and only electronic device inside your household. Problems arise when you go beyond these assumptions: maybe AI that writes code has to go through a code review process, but what if that code review is done by another unrelated AI agent? What happens when you get to operations and common mode failures impact components from various teams that all have agents empowered to go fix things to the best of their ability with the available data? Are they going to clash with people, or even with each other? Humans also have that ability and tend to solve it via processes and procedures, explicit coordination, announcing what they’ll do before they do it, and calling upon each other when they need help. Will multiple agents require something equivalent, and if so, do you have it in place? How do they cope with limited context? Some changes that cause issues might be safe to roll back, some not (maybe they include database migrations, maybe it is better to be down than corrupting data), and some may contain changes that rolling back wouldn’t fix (maybe the workload is controlled by one or more feature flags). Knowing what to do in these situations can sometimes be understood from code or release notes, but some situations can require different workflows involving broader parts of the organization. A risk of automation without context is that if you have situations where waiting or doing little is the best option, then you’ll need to either have automation that requires input to act, or a set of actions to quickly disable multiple types of automation as fast as possible. Many of these may exist at the same time, and it becomes the operators’ jobs to not only maintain their own context, but also maintain a mental model of the context each of these pieces of automation has access to. The fancier your agents, the fancier your operators’ understanding and abilities must be to properly orchestrate them. The more surprising your landscape is, the harder it can become to manage with semi-autonomous elements roaming around. After an outage or incident, who does the learning and who does the fixing? One way to track accountability in a system is to figure out who ends up having to learn lessons and change how things are done. It’s not always the same people or teams, and generally, learning will happen whether you want it or not. This is more of a rhetorical question right now, because I expect that in most cases, when things go wrong, whoever is expected to monitor the AI tool is going to have to steer it in a better direction and fix it (if they can); if it can’t be fixed, then the expectation will be that the automation, as a tool, will be used more judiciously in the future. In a nutshell, if the expectation is that your engineers are going to be doing the learning and tweaking, your AI isn’t an independent agent—it’s a tool that cosplays as an independent agent. Do what you will—just be mindful All in all, none of the above questions flat out say you should not use AI, nor where exactly in the loop you should put people. The key point is that you should ask that question and be aware that just adding whatever to your system is not going to substitute workers away. It will, instead, transform work and create new patterns and weaknesses. Some of these patterns are known and well-studied. We don’t have to go rushing to rediscover them all through failures as if we were the first to ever automate something. If AI ever gets so good and so smart that it’s better than all your engineers, it won’t make a difference whether you adopt it only once it’s good. In the meanwhile, these things do matter and have real impacts, so please design your systems responsibly. If you’re interested to know more about the theoretical elements underpinning this post, the following references—on top of whatever was already linked in the text—might be of interest: Books: Joint Cognitive Systems: Foundations of Cognitive Systems Engineering by Erik Hollnagel Joint Cognitive Systems: Patterns in Cognitive Systems Engineering by David D. Woods Cognition in the Wild by Edwin Hutchins Behind Human Error by David D. Woods, Sydney Dekker, Richard Cook, Leila Johannesen, Nadine Sarter Papers: Ironies of Automation by Lisanne Bainbridge The French-Speaking Ergonomists’ Approach to Work Activity by Daniellou How in the World Did We Ever Get into That Mode? Mode Error and Awareness in Supervisory Control by Nadine Sarter Can We Ever Escape from Data Overload? A Cognitive Systems Diagnosis by David D. Woods Ten Challenges for Making Automation a “Team Player” in Joint Human-Agent Activity by Gary Klein and David D. Woods MABA-MABA or Abracadabra? Progress on Human–Automation Co-ordination by Sidney Dekker Managing the Hidden Costs of Coordination by Laura Maguire Designing for Expertise by David D. Woods The Impact of Generative AI on Critical Thinking by Lee et al.
AMD is sending us the two MI300X boxes we asked for. They are in the mail. It took a bit, but AMD passed my cultural test. I now believe they aren’t going to shoot themselves in the foot on software, and if that’s true, there’s absolutely no reason they should be worth 1/16th of NVIDIA. CUDA isn’t really the moat people think it is, it was just an early ecosystem. tiny corp has a fully sovereign AMD stack, and soon we’ll port it to the MI300X. You won’t even have to use tinygrad proper, tinygrad has a torch frontend now. Either NVIDIA is super overvalued or AMD is undervalued. If the petaflop gets commoditized (tiny corp’s mission), the current situation doesn’t make any sense. The hardware is similar, AMD even got the double throughput Tensor Cores on RDNA4 (NVIDIA artificially halves this on their cards, soon they won’t be able to). I’m betting on AMD being undervalued, and that the demand for AI has barely started. With good software, the MI300X should outperform the H100. In for a quarter million. Long term. It can always dip short term, but check back in 5 years.
Earlier this weekGuileWhippet But now I do! Today’s note is about how we can support untagged allocations of a few different kinds in Whippet’s .mostly-marking collector Why bother supporting untagged allocations at all? Well, if I had my way, I wouldn’t; I would just slog through Guile and fix all uses to be tagged. There are only a finite number of use sites and I could get to them all in a month or so. The problem comes for uses of from outside itself, in C extensions and embedding programs. These users are loathe to adapt to any kind of change, and garbage-collection-related changes are the worst. So, somehow, we need to support these users if we are not to break the Guile community.scm_gc_malloclibguile The problem with , though, is that it is missing an expression of intent, notably as regards tagging. You can use it to allocate an object that has a tag and thus can be traced precisely, or you can use it to allocate, well, anything else. I think we will have to add an API for the tagged case and assume that anything that goes through is requesting an untagged, conservatively-scanned block of memory. Similarly for : you could be allocating a tagged object that happens to not contain pointers, or you could be allocating an untagged array of whatever. A new API is needed there too for pointerless untagged allocations.scm_gc_mallocscm_gc_mallocscm_gc_malloc_pointerless Recall that the mostly-marking collector can be built in a number of different ways: it can support conservative and/or precise roots, it can trace the heap precisely or conservatively, it can be generational or not, and the collector can use multiple threads during pauses or not. Consider a basic configuration with precise roots. You can make tagged pointerless allocations just fine: the trace function for that tag is just trivial. You would like to extend the collector with the ability to make pointerless allocations, for raw data. How to do this?untagged Consider first that when the collector goes to trace an object, it can’t use bits inside the object to discriminate between the tagged and untagged cases. Fortunately though . Of those 8 bits, 3 are used for the mark (five different states, allowing for future concurrent tracing), two for the , one to indicate whether the object is pinned or not, and one to indicate the end of the object, so that we can determine object bounds just by scanning the metadata byte array. That leaves 1 bit, and we can use it to indicate untagged pointerless allocations. Hooray!the main space of the mostly-marking collector has one metadata byte for each 16 bytes of payloadprecise field-logging write barrier However there is a wrinkle: when Whippet decides the it should evacuate an object, it tracks the evacuation state in the object itself; the embedder has to provide an implementation of a , allowing the collector to detect whether an object is forwarded or not, to claim an object for forwarding, to commit a forwarding pointer, and so on. We can’t do that for raw data, because all bit states belong to the object, not the collector or the embedder. So, we have to set the “pinned” bit on the object, indicating that these objects can’t move.little state machine We could in theory manage the forwarding state in the metadata byte, but we don’t have the bits to do that currently; maybe some day. For now, untagged pointerless allocations are pinned. You might also want to support untagged allocations that contain pointers to other GC-managed objects. In this case you would want these untagged allocations to be scanned conservatively. We can do this, but if we do, it will pin all objects. Thing is, conservative stack roots is a kind of a sweet spot in language run-time design. You get to avoid constraining your compiler, you avoid a class of bugs related to rooting, but you can still support compaction of the heap. How is this, you ask? Well, consider that you can move any object for which we can precisely enumerate the incoming references. This is trivially the case for precise roots and precise tracing. For conservative roots, we don’t know whether a given edge is really an object reference or not, so we have to conservatively avoid moving those objects. But once you are done tracing conservative edges, any live object that hasn’t yet been traced is fair game for evacuation, because none of its predecessors have yet been visited. But once you add conservatively-traced objects back into the mix, you don’t know when you are done tracing conservative edges; you could always discover another conservatively-traced object later in the trace, so you have to pin everything. The good news, though, is that we have gained an easier migration path. I can now shove Whippet into Guile and get it running even before I have removed untagged allocations. Once I have done so, I will be able to allow for compaction / evacuation; things only get better from here. Also as a side benefit, the mostly-marking collector’s heap-conservative configurations are now faster, because we have metadata attached to objects which allows tracing to skip known-pointerless objects. This regains an optimization that BDW has long had via its , used in Guile since time out of mind.GC_malloc_atomic With support for untagged allocations, I think I am finally ready to start getting Whippet into Guile itself. Happy hacking, and see you on the other side! inside and outside on intent on data on slop fin
I’ve been working on a project where I need to plot points on a map. I don’t need an interactive or dynamic visualisation – just a static map with coloured dots for each coordinate. I’ve created maps on the web using Leaflet.js, which load map data from OpenStreetMap (OSM) and support zooming and panning – but for this project, I want a standalone image rather than something I embed in a web page. I want to put in coordinates, and get a PNG image back. This feels like it should be straightforward. There are lots of Python libraries for data visualisation, but it’s not an area I’ve ever explored in detail. I don’t know how to use these libraries, and despite trying I couldn’t work out how to accomplish this seemingly simple task. I made several attempts with libraries like matplotlib and plotly, but I felt like I was fighting the tools. Rather than persist, I wrote my own solution with “lower level” tools. The key was a page on the OpenStreetMap wiki explaining how to convert lat/lon coordinates into the pixel system used by OSM tiles. In particular, it allowed me to break the process into two steps: Get a “base map” image that covers the entire world Convert lat/lon coordinates into xy coordinates that can be overlaid on this image Let’s go through those steps. Get a “base map” image that covers the entire world Let’s talk about how OpenStreetMap works, and in particular their image tiles. If you start at the most zoomed-out level, OSM represents the entire world with a single 256×256 pixel square. This is the Web Mercator projection, and you don’t get much detail – just a rough outline of the world. We can zoom in, and this tile splits into four new tiles of the same size. There are twice as many pixels along each edge, and each tile has more detail. Notice that country boundaries are visible now, but we can’t see any names yet. We can zoom in even further, and each of these tiles split again. There still aren’t any text labels, but the map is getting more detailed and we can see small features that weren’t visible before. You get the idea – we could keep zooming, and we’d get more and more tiles, each with more detail. This tile system means you can get detailed information for a specific area, without loading the entire world. For example, if I’m looking at street information in Britain, I only need the detailed tiles for that part of the world. I don’t need the detailed tiles for Bolivia at the same time. OpenStreetMap will only give you 256×256 pixels at a time, but we can download every tile and stitch them together, one-by-one. Here’s a Python script that enumerates all the tiles at a particular zoom level, downloads them, and uses the Pillow library to combine them into a single large image: #!/usr/bin/env python3 """ Download all the map tiles for a particular zoom level from OpenStreetMap, and stitch them into a single image. """ import io import itertools import httpx from PIL import Image zoom_level = 2 width = 256 * 2**zoom_level height = 256 * (2**zoom_level) im = Image.new("RGB", (width, height)) for x, y in itertools.product(range(2**zoom_level), range(2**zoom_level)): resp = httpx.get(f"https://tile.openstreetmap.org/{zoom_level}/{x}/{y}.png", timeout=50) resp.raise_for_status() im_buffer = Image.open(io.BytesIO(resp.content)) im.paste(im_buffer, (x * 256, y * 256)) out_path = f"map_{zoom_level}.png" im.save(out_path) print(out_path) The higher the zoom level, the more tiles you need to download, and the larger the final image will be. I ran this script up to zoom level 6, and this is the data involved: Zoom level Number of tiles Pixels File size 0 1 256×256 17.1 kB 1 4 512×512 56.3 kB 2 16 1024×1024 155.2 kB 3 64 2048×2048 506.4 kB 4 256 4096×4096 2.7 MB 5 1,024 8192×8192 13.9 MB 6 4,096 16384×16384 46.1 MB I can just about open that zoom level 6 image on my computer, but it’s struggling. I didn’t try opening zoom level 7 – that includes 16,384 tiles, and I’d probably run out of memory. For most static images, zoom level 3 or 4 should be sufficient – I ended up a base map from zoom level 4 for my project. It takes a minute or so to download all the tiles from OpenStreetMap, but you only need to request it once, and then you have a static image you can use again and again. This is a particularly good approach if you want to draw a lot of maps. OpenStreetMap is provided for free, and we want to be a respectful user of the service. Downloading all the map tiles once is more efficient than making repeated requests for the same data. Overlay lat/lon coordinates on this base map Now we have an image with a map of the whole world, we need to overlay our lat/lon coordinates as points on this map. I found instructions on the OpenStreetMap wiki which explain how to convert GPS coordinates into a position on the unit square, which we can in turn add to our map. They outline a straightforward algorithm, which I implemented in Python: import math def convert_gps_coordinates_to_unit_xy( *, latitude: float, longitude: float ) -> tuple[float, float]: """ Convert GPS coordinates to positions on the unit square, which can be plotted on a Web Mercator projection of the world. This expects the coordinates to be specified in **degrees**. The result will be (x, y) coordinates: - x will fall in the range (0, 1). x=0 is the left (180° west) edge of the map. x=1 is the right (180° east) edge of the map. x=0.5 is the middle, the prime meridian. - y will fall in the range (0, 1). y=0 is the top (north) edge of the map, at 85.0511 °N. y=1 is the bottom (south) edge of the map, at 85.0511 °S. y=0.5 is the middle, the equator. """ # This is based on instructions from the OpenStreetMap Wiki: # https://wiki.openstreetmap.org/wiki/Slippy_map_tilenames#Example:_Convert_a_GPS_coordinate_to_a_pixel_position_in_a_Web_Mercator_tile # (Retrieved 16 January 2025) # Convert the coordinate to the Web Mercator projection # (https://epsg.io/3857) # # x = longitude # y = arsinh(tan(latitude)) # x_webm = longitude y_webm = math.asinh(math.tan(math.radians(latitude))) # Transform the projected point onto the unit square # # x = 0.5 + x / 360 # y = 0.5 - y / 2π # x_unit = 0.5 + x_webm / 360 y_unit = 0.5 - y_webm / (2 * math.pi) return x_unit, y_unit Their documentation includes a worked example using the coordinates of the Hachiko Statue. We can run our code, and check we get the same results: >>> convert_gps_coordinates_to_unit_xy(latitude=35.6590699, longitude=139.7006793) (0.8880574425, 0.39385379958274735) Most users of OpenStreetMap tiles will use these unit positions to select the tiles they need, and then dowload those images – but we can also position these points directly on the global map. I wrote some more Pillow code that converts GPS coordinates to these unit positions, scales those unit positions to the size of the entire map, then draws a coloured circle at each point on the map. Here’s the code: from PIL import Image, ImageDraw gps_coordinates = [ # Hachiko Memorial Statue in Tokyo {"latitude": 35.6590699, "longitude": 139.7006793}, # Greyfriars Bobby in Edinburgh {"latitude": 55.9469224, "longitude": -3.1913043}, # Fido Statue in Tuscany {"latitude": 43.955101, "longitude": 11.388186}, ] im = Image.open("base_map.png") draw = ImageDraw.Draw(im) for coord in gps_coordinates: x, y = convert_gps_coordinates_to_unit_xy(**coord) radius = 32 draw.ellipse( [ x * im.width - radius, y * im.height - radius, x * im.width + radius, y * im.height + radius, ], fill="red", ) im.save("map_with_dots.png") and here’s the map it produces: The nice thing about writing this code in Pillow is that it’s a library I already know how to use, and so I can customise it if I need to. I can change the shape and colour of the points, or crop to specific regions, or add text to the image. I’m sure more sophisticated data visualisation libraries can do all this, and more – but I wouldn’t know how. The downside is that if I need more advanced features, I’ll have to write them myself. I’m okay with that – trading sophistication for simplicity. I didn’t need to learn a complex visualization library – I was able to write code I can read and understand. In a world full of AI-generating code, writing something I know I understand feels more important than ever. [If the formatting of this post looks odd in your feed reader, visit the original article]
This website has a new section: blogroll.opml! A blogroll is a list of blogs - a lightweight way of people recommending other people’s writing on the indieweb. What it includes The blogs that I included are just sampled from my many RSS subscriptions that I keep in my Feedbin reader. I’m subscribed to about 200 RSS feeds, the majority of which are dead or only publish once a year. I like that about blogs, that there’s no expectation of getting a post out every single day, like there is in more algorithmically-driven media. If someone who I interacted with on the internet years ago decides to restart their writing, that’s great! There’s no reason to prune all the quiet feeds. The picks are oriented toward what I’m into: niches, blogs that have a loose topic but don’t try to be general-interest, people with distinctive writing. If you import all of the feeds into your RSS reader, you’ll probably end up unsubscribing from some of them because some of the experimental electric guitar design or bonsai news is not what you’re into. Seems fine, or you’ll discover a new interest! How it works Ruben Schade figured out a brilliant way to show blogrolls and I copied him. Check out his post on styling OPML and RSS with XSLT to XHTML for how it works. My only additions to that scheme were making the blogroll page blend into the rest of the website by using an include tag with Jekyll to add the basic site skeleton, and adding a link with the download attribute to provide a simple way to download the OPML file. Oddly, if you try to save the OPML page using Save as… in Firefox, Firefox will save the transformed output via the XSLT, rather than the raw source code. XSLT is such an odd and rare part of the web ecosystem, I had to use it.