Full Width 
Shortcuts 
Sign Up 
Log In 
Articles 
Reading List More from anderegg.ca
Yesterday, a group of MIT researchers released an article titled “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights”. The piece describes a set of experiments which convincingly show OpenAI’s latest Astra model is able to understand and successfully alter raw weight data. Wowzers! It’s a rough analogy, but an LLM’s weights are sort of like its brain. A set of weights is a large pile of numbers generated during training, and an LLM’s output is derived by running input through the weights using some matrix multiplication and a bit of randomization. While the steps to generate weights are well understood, the weights themselves are extremely opaque. There’s ongoing work to try and explain how weights work, but researchers currently have a limited understanding of what these numbers do. Based on this, I was quite surprised to learn that LLMs are able to act on raw weight data — even if it’s still just in limited cases. It’s unclear if OpenAI’s model was trained on weight data. Even if it was, I still find these results surprising. The article shows the model accurately predicting what a never-before-seen set of weights was built to do, fixing another model by updating its weights, and creating new sets of weights that performed simple tasks. I would not have guessed any of that would be possible a couple of days ago. This whole thing is slightly unnerving. The whole “weights as a black box” thing has always felt a bit sketchy to me, even when I was working with much simpler machine learning models. Now these experiments hint at a future where LLMs could be self-improving in ways that aren’t human-verifiable. Machines building machines, and we can only hope that they’re not doing anything weird. I don’t know if this strategy is scalable, but I’d guess that it’s something more people will be exploring very soon. More and more often these days, I wish we were living in less interesting times.
After living with Apple’s 27.0 OSs since launch, I have some more annoyances to get off my chest. This time, it’s all about how tabs and menus have gotten worse. I’ve already ranted about the Liquid Glass material in general, but these two design changes in particular have really been grinding my gears. I’ll reiterate that Apple’s latest OSs look substantially nicer to me than the previous set… but that only makes these setbacks more glaring. Also, many of these issues aren’t nearly as bad in light mode — but I use dark mode exclusively on all platforms. Apple offers this appearance setting, so I think it’s fair to criticize them when it’s not holding up. First up, let’s talk tab bars. I think these looked awful in the original Liquid Glass redesign, and in 27.0 they look even worse. Below is an example of three tab bars from Safari in macOS. All of them are in dark mode. The top example is from macOS 26 with the “clear” Liquid Glass setting, the middle is macOS 27 with the default (mid-slider) version of Liquid Glass, and the bottom is macOS 27 with Liquid Glass at its most tinted. In each, the middle tab is selected (though I think the word “tab” is being quite generous to these globs). In macOS 26, there was practically no difference between the clear and tinted versions of tabs. Similarly, the clearest and default/middle tabs in macOS 27 are effectively the same. Because of this, I’m leaving out the redundant examples. Even though I still think it’s ugly, I vastly prefer the macOS 26 version of these three options. It offers the most contrast, and it makes more sense in dark mode: the background is darker and the foreground of the active tab is lighter. The middle example is what tabs look like in the default (mid-slider) version of Liquid Glass in macOS 27. There’s now only a very faint outline around the active tab, and practically no difference in background colours. To me, this is unreasonably subtle. The effect is even worse when there are a lot of tabs open. Lastly, there’s macOS 27 with the fully tinted Liquid Glass setting. It’s better, but it still looks less correct to me than the tab design from macOS 26. It’s difficult to put into words how much I loathe the look of this new tab bar design. I don’t mind the more “bubbly” look of Liquid Glass throughout the 27 OSs for the most part. It gives UI elements more dimension than in the 26 OSs. But it doesn’t work for tabs. Because the bubbly look is inside a trough, the active tab’s glass effect ends up looking like a blur on the top and bottom. This reduces contrast further and makes the active tab harder for me to pick out. Even in dark mode with full tint, I find the active tab less visually clear than in 26’s clear mode. Now, I’m sure there are at least a few people reading who don’t see what the fuss is about. If that’s you, I assure you that the difference is more stark when you’re not comparing things side by side. It’s not impossible for me to pick out the active tab, but I think it’s trickier than it needs to be! But, if you still don’t believe me, here’s a little experiment. Which of these do you think is most legible? The text/background colours in the above image are based on the foreground/background colours used in the tab bar instances above. First is clear in macOS 26, then the default from macOS 27, then fully tinted in macOS 27. I think they’re all pretty awful, but I prefer the macOS 26 clear version. Again, this is because I’m using dark mode. In dark mode, light text appears on a darker background. Similarly, active UI elements have a lighter background than their surrounding elements. I’m sure there are counter-examples, but this is how just about everything else works in Apple’s own apps! It should be noted that Safari uses the system default tab bar design. I also see this design in Apple’s Terminal app, in Pixelmator Pro, and elsewhere. I don’t use Xcode daily anymore, but you’ll also find them there — though in true Xcode fashion, they’re ever so slightly nonstandard and also don’t respect your tint setting. Below is a screenshot of Xcode using my current settings of dark mode with fully tinted Liquid Glass. Up next: menus. Below is an image showing four versions of the same menu in macOS. Top left is macOS 26 clear, top right is macOS 26 tinted, bottom left is macOS 27 default (mid-slider), and bottom right is macOS 27 fully tinted. It’s a similar story here. In macOS 26’s dark mode, I had no problem with system menus even when Liquid Glass was set to clear. In macOS 27, even in the fully tinted mode, the menus have much lower contrast. They also now lose all of Liquid Glass’s refractive effects when at their most tinted. I think this is less of an issue than the tab design changes, but it’s still a downgrade. Again, I’m certain many people don’t care about this. Some might wonder why I’m not turning on accessibility settings to help with these things, if they bother me so much. I’ve flirted with this (especially the “Increase Contrast” setting), but those settings have many knock-on effects. 1 But honestly, I don’t think accessibility settings should be required to have a reasonable amount of contrast in a design system. Maybe Apple disagrees, but I really hope more dark mode tweaks are coming. The “Increase Contrast” setting is under System Settings -> Accessibility -> Display -> Increase Contrast. Interestingly, you can use this setting along with the clearest version of Liquid Glass to almost get back to how things looked in macOS 26’s version of tinted. However, it adds contrast-y lines around many UI elements that I find extremely distracting. It also alters colours on some elements to, strangely, make them less contrast-y. It feels unevenly applied and poorly implemented in several apps. ↩
I really enjoy John Gruber’s podcast The Talk Show. However, I’ve noticed that it’s been a while since it was in my feed. I decided to chart previous episode dates to check some of my assumptions about release frequency. Cutting to the chase, here’s a link to The Talk Show Timeline. You can click/tap on days to get a pinned tooltip, which lets you link out to a particular day’s episode. I’ve tried to make this work well on mobile devices, but you’ll likely find it works best on desktop. A note about this: Gruber has mentioned in the past that there has been a personal situation that he’s been working through. He’s also mentioned something to this effect on the podcast at points, but hasn’t gone into detail. I don’t need to know more than has been said, and I just hope things are alright. I’ve read Gruber’s site since 2002 1 and listened to his podcasts since 2007. In that time, I’ve learned that he operates on his own timeline… and has a penchant for procrastination. No shade here! He’s set himself up with a work situation that I envy. I also don’t want to seem grumpy about the lack of podcast episodes; I’m just trying to understand how this current break compares to those in the past. Because it was easiest, and because it would answer my immediate questions, I only looked at the current iteration of The Talk Show. Happily, there’s a single page which lists all the episodes. It had a tiny bit of data weirdness, 2 but I was able to turn this into a JSON file with the details I cared about. With that, I used the calendar heatmap chart in Apache ECharts to display the episodes over time. Some things I noticed: While it’s been a while (29 days) since the last episode (on August 10), this isn’t unheard of. The average is about 12 days between episodes, but there have been 6 other times when the episode gap has been at least this long. The longest gap was 62 days between December 24, 2024 and February 24, 2025. There’s exactly one day with two episodes: July 19, 2014. It was “Cat Pictures” with Marco Arment, which got split into two “sides”. Here’s a link to side 1, and here’s side 2. I mentioned procrastination: 73% of episodes take place on the 16th of a month or later. 28% of episodes take place on either the last or second-last day of the month. Again, no shade! It’s something I chuckle about whenever I notice it. At the current rate, this year is about in line with output going back to 2023. While it has been a while since a new episode dropped, it’s not out of whack statistically. Anyway, this was a Labour Day project that was mostly for my own amusement. I expect there will likely be a new episode of The Talk Show some time fairly soon after tomorrow’s Apple event. I also don’t know if I’ll keep this chart up to date… though it’s not terribly onerous to do now, so who knows. I found Daring Fireball because I was researching the purchase of my first Mac around this time. It wasn’t until October of 2002, but I ended up buying one of the machines he wrote about in his first post that August. ↩ The following dates don’t match the main format: Sunday 30 November, 2025 Mon, 24 Feb 2020 16:05:34 EDT Tuesday 31 December 2019 Tuesday, June 25 2019 Sat, 16 Mar 2019 19:43:49 EDT Wednesday 31 December 2014 Monday, 30 Jun 2014 ↩
I mostly use my web stats service because I get a lizard-brain kick from watching numbers go up, but it also provides some fun insights. For example, it’s how I found out about bubbles.town. At first I thought this might be a Mastodon instance. I get a lot of referral traffic from Mastodon, but each instance has its own URL. You can guess at mas.to, but dice.camp doesn’t provide many clues. Bubbles, as it’s known, is a very nice service. It aggregates a curated list of indie blogs. People can sign up and vote for posts they enjoy, but they can’t submit new articles. Authentication and comments are both handled using Fediverse technologies. In my case, I was able to log in via my Mastodon instance. The idea is to highlight cool indie writers around the web. I love this sort of thing so very much. Anyway, this is all to say that you should check out Bubbles. Seeing a blog-based reaction to one of my posts on the service filled my heart with immense joy.
More in technology
On Sunday morning, a lot of us woke up to Grok Bot, the always-on AI assistant, decidedly off. “100% used” read my quota meter. It’d been 45% late the night before. It was an error, but of the Freudian slip sort, and that’s a problem if tools like this are to be taken seriously.
A Pika author was editing the same post in two browsers. They added six images in Browser A, then later they referenced an old tab editing the post in Browser B. That copy of the editor did not include those images. Browser B saved and removed all six images from Active Storage. A short time later, Browser A saved its newer copy of the post and put the image references back, but at that point the originally uploaded files were gone due to Browser B saving the post without the images. This exposed two separate problems: Missing images produced uncached 404 responses that the editor requested repeatedly. Action Text purged an image as soon as one save removed it from the body. Here’s how we fixed those problems. Why Action Text deleted the images Action Text keeps a rich text record’s embeds synchronized with the attachments in its body. When a submitted body no longer contains an image, Action Text removes its ActiveStorage::Attachment. Active Storage then purges the dependent blob, deleting both its database record and its file. That behavior makes sense when there is only one current copy of a document. It is unsafe when the same post can be open in multiple tabs, browsers, or devices. One stale save can delete an image that another editor is still using. Why the missing images flooded the server Browser A still had the six images in its editor, each pointing at an Active Storage redirect URL: /rails/active_storage/blobs/redirect/:signed_id/:filename Pika uses the ProseMirror-based Rhino editor. Rhino rebuilt an image’s DOM node when its caption or selection changed. Each new <img> element requested its URL again. Healthy images did not cause trouble since Active Storage makes its redirect cacheable. For a purged blob, Rails returned a 404 without Cache-Control, so the browser kept sending the request to Rails instead of reusing the failure. In a gallery, typing could re-request several missing images per keystroke. In these scenarios, this issue led to hundreds of requests hitting our servers. Fix 1: Cache permanent Active Storage 404s A purged blob’s redirect URL can never start working again. Blob IDs are not reused, and uploading the image again creates a new blob with a new signed URL. Sinc there is no concern of a 404 becoming stale, we made a specific 404 cacheable for the same period as a successful redirect: # app/controllers/concerns/active_storage_missing_blob_cache.rb module ActiveStorageMissingBlobCache extend ActiveSupport::Concern included do rescue_from ActiveRecord::RecordNotFound, with: :render_cacheable_not_found end private def render_cacheable_not_found expires_in ActiveStorage.service_urls_expire_in render file: Rails.public_path.join("404.html"), layout: false, status: :not_found end end We included the concern only in the controller serving these blob redirects: # config/initializers/active_storage_missing_blob_cache.rb Rails.application.config.to_prepare do ActiveStorage::Blobs::RedirectController.include(ActiveStorageMissingBlobCache) end (We also offered an update to Rhino that avoids it being so aggressive re-requesting images.) Fix 2: Detach dropped images instead of purging them When a save removes an image from an Action Text body, we now remove the attachment while temporarily keeping its blob. Another browser can save the image again and reattach that same blob. If nothing brings it back, a cleanup job purges it later. First, we added a timestamp for when the image was dropped: class AddDetachedAtToActiveStorageBlobs < ActiveRecord::Migration[8.1] def change add_column :active_storage_blobs, :detached_at, :datetime end end We then intercepted the private callback Active Storage uses to purge a dependent blob: # config/initializers/detach_dropped_embeds.rb module DetachDroppedEmbeds private def purge_dependent_blob if dropped_from_rich_text? blob&.update_columns(detached_at: Time.now) else super end end def dropped_from_rich_text? record.is_a?(ActionText::RichText) && !destroyed_by_association end end ActiveSupport.on_load(:active_storage_attachment) do prepend DetachDroppedEmbeds end The destroyed_by_association check preserves intentional deletion. Removing an image from a saved body detaches it, but deleting the post and its rich text still purges its images normally. Purge detached blobs after a grace period The cleanup query considers both when the blob was created and when it was detached: ActiveSupport.on_load(:active_storage_blob) do scope :unattached_since, ->(time) { unattached.where( created_at: ..time, detached_at: [nil, ..time] ) } end That scope handles both kinds of unattached blobs: A new upload that has never been attached becomes eligible based on created_at. An image removed from an old post becomes eligible based on detached_at. Pika gives both a 30-day grace period: class PurgeUnattachedUploadsJob < ApplicationJob GRACE_PERIOD = 30.days def perform ActiveStorage::Blob.unattached_since(GRACE_PERIOD.ago).find_each do |blob| blob.purge_later end end end If another browser saves the image during that period, it is attached again and the unattached scope excludes it from cleanup. The result Replaying the original two-browser edit now works safely: Browser B saves a body without the six images. The attachments are removed, but their blobs are only marked as detached. Browser A saves its newer body and reattaches the existing blobs. The images remain available and do not need to be uploaded again. If no editor brings a detached image back, the cleanup job eventually purges it. Much better! Reply by email
Yesterday, a group of MIT researchers released an article titled “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights”. The piece describes a set of experiments which convincingly show OpenAI’s latest Astra model is able to understand and successfully alter raw weight data. Wowzers! It’s a rough analogy, but an LLM’s weights are sort of like its brain. A set of weights is a large pile of numbers generated during training, and an LLM’s output is derived by running input through the weights using some matrix multiplication and a bit of randomization. While the steps to generate weights are well understood, the weights themselves are extremely opaque. There’s ongoing work to try and explain how weights work, but researchers currently have a limited understanding of what these numbers do. Based on this, I was quite surprised to learn that LLMs are able to act on raw weight data — even if it’s still just in limited cases. It’s unclear if OpenAI’s model was trained on weight data. Even if it was, I still find these results surprising. The article shows the model accurately predicting what a never-before-seen set of weights was built to do, fixing another model by updating its weights, and creating new sets of weights that performed simple tasks. I would not have guessed any of that would be possible a couple of days ago. This whole thing is slightly unnerving. The whole “weights as a black box” thing has always felt a bit sketchy to me, even when I was working with much simpler machine learning models. Now these experiments hint at a future where LLMs could be self-improving in ways that aren’t human-verifiable. Machines building machines, and we can only hope that they’re not doing anything weird. I don’t know if this strategy is scalable, but I’d guess that it’s something more people will be exploring very soon. More and more often these days, I wish we were living in less interesting times.
Harry threw me off course in episode 48 of Traction Heroes. At first, I thought was reading from a book about mind mapping. Soon, it became clear it was about something else: how to define a clear vision for a team or organization. Visions are notoriously hard to articulate. Drawing a picture — especially a picture of the whole forest, plus the trees — can help. The passage Harry read from Cameron Herold’s Vivid Vision explores this idea: So to visualize something a little more tangible, think about a tree trunk in the earth, inner core with its branches stretching out into the sky. Each of those branches contains twigs extend further out. The mind map works in a similar way, a core that branches out into smaller components. Now, apply this to the world of business. As the leader of an organization, you’re firmly planted at the center, and the various areas that contribute to the business branch out: marketing, sales, engineering, and so forth. With that in place, build off that by describing those branches in about three or four bullet points each. In other words, describe what your marketing looks like and sales and so on until you’ve captured every idea. Remember too that among the branches of your business, you want to consider every aspect of the organization: the vibe, informal codes of conduct, the fiscal environment, the scenery, all of that. Deliberately leave out numbers and data. Get a feel for the place. When you step out of your time machine three years from now and walk around the office, what’s happening? What can you say that is typical of the business three years from now? The mind map is an initial foray into the vivid vision and will act as an outline for the rough draft that you, the leader, will then write. I’ve sketched diagrams before when outlining articles, workshops, and presentations, never in service of aligning around a company’s vision. Harry’s done it several times and found it easier than trying to write it out. Seems useful — check out our conversation for more: Traction Heroes episode 48: Visions
Sharing a few data points because some people did not expect these drives to last that long.
