Full Width [alt+shift+f] Shortcuts [alt+shift+k]
Sign Up [alt+shift+s] Log In [alt+shift+l]

Posts on Made of Bugs

Posts on Made of...
Finding near-duplicates with Jaccard similarity and MinHash Suppose we have a large collection of documents, and we wish you identify which documents are...
7 months ago
18
7 months ago
Suppose we have a large collection of documents, and we wish you identify which documents are approximately the same as each other. For instance, we may have crawled the web over some period of time, and expect to have fetched the “same page” several times, but to see slight...
Posts on Made of...
Building personal software with Claude Earlier this month, I used Claude to port (parts of) an Emacs package into Rust, shrinking the...
3 weeks ago
18
3 weeks ago
Earlier this month, I used Claude to port (parts of) an Emacs package into Rust, shrinking the execution time by a factor of 1000 or more (in one concrete case: from 90s to about 15ms). This is a variety of yak-shave that I do somewhat routinely, both professionally and in...
Posts on Made of...
Advent of Code in C++ Template Metaprogramming This December, the imp of the perverse struck me, and I decided to see how many days of Advent of...
a year ago
16
a year ago
This December, the imp of the perverse struck me, and I decided to see how many days of Advent of Code I could do purely in compile-time C++ metaprogramming. As of this writing, I’ve done two days, and I’m not sure I’ll make it any further. However, that’s one more day than I...
Posts on Made of...
Performance engineering, profilers, and seeing the invisible I was recently introduced to the paper “Seeing the Invisible: Perceptual-Cognitive Aspects of...
a year ago
14
a year ago
I was recently introduced to the paper “Seeing the Invisible: Perceptual-Cognitive Aspects of Expertise” by Gary Klein and Robert Hoffman. It’s excellent and I recommend you read it when you have a chance. Klein and Hoffman discuss the ability of experts to “see what is not...
Posts on Made of...
Stripe's monorepo developer environment I worked at Stripe for about seven years, from 2012 to 2019. Over that time, I used and contributed...
9 months ago
12
9 months ago
I worked at Stripe for about seven years, from 2012 to 2019. Over that time, I used and contributed to many generations of Stripe’s developer environment – the tools that engineers used daily to write and test code. I think Stripe did a pretty good job designing and building that...
Posts on Made of...
A Brief Introduction to termios: termios(3) and stty (This is part two of a multi-part introduction to termios and terminal emulation on UNIX. Read part...
over a year ago
11
over a year ago
(This is part two of a multi-part introduction to termios and terminal emulation on UNIX. Read part 1 if you’re new here) In this entry, we’ll look at the interfaces that are used to control the behavior of the “termios” box sitting between the master and slave pty. The behaviors...
Posts on Made of...
Towards solving Ultimate Tic Tac Toe Summary: Read about my efforts to solve the game of Ultimate Tic Tac Toe. It’s been a fun journey...
over a year ago
11
over a year ago
Summary: Read about my efforts to solve the game of Ultimate Tic Tac Toe. It’s been a fun journey into interesting algorithms and high-performance parallel programming in Rust. Backstory Starting around the beginning of the COVID-19 lockdown, I’ve gotten myself deeply nerdsniped...
Posts on Made of...
Disable Transparent Hugepages tl;dr “Transparent Hugepages” is a Linux kernel feature intended to improve performance by making...
over a year ago
11
over a year ago
tl;dr “Transparent Hugepages” is a Linux kernel feature intended to improve performance by making more efficient use of your processor’s memory-mapping hardware. It is enabled ("enabled=always") by default in most Linux distributions. Transparent Hugepages gives some applications...
Posts on Made of...
A Go/C Polyglot Writing a Go/C polyglot Someone on a Slack I’m on recently raised the question of how you might...
over a year ago
11
over a year ago
Writing a Go/C polyglot Someone on a Slack I’m on recently raised the question of how you might write a source file that’s both valid C and Go, commenting that it wasn’t immediately obvious if this was even possible. I got nerdsniped, and succeeded in producing one, which you can...
Posts on Made of...
Using wpa_supplicant on Debian/Ubuntu I’ve been using wpa_supplicant to manage wifi on my Ubuntu laptop for a while, and have found that...
over a year ago
11
over a year ago
I’ve been using wpa_supplicant to manage wifi on my Ubuntu laptop for a while, and have found that it’s pretty close to what I want for managing wireless — closer than anything else I’ve found, at least. I figured I should document my setup and experiences. Some Background You...
Posts on Made of...
Indices point between elements If you’re familiar with nearly any mainstream programming language, and I asked you to draw a...
over a year ago
11
over a year ago
If you’re familiar with nearly any mainstream programming language, and I asked you to draw a diagram of an array, the array indices, and the array elements, odds are good you’d produce a diagram something like this: In this post, I want to persuade you to replace that image, or,...
Posts on Made of...
Suggestion time: What should I blog about? I haven’t been feeling very motivated to blog lately – I’ve missed the last two weeks of Iron...
over a year ago
10
over a year ago
I haven’t been feeling very motivated to blog lately – I’ve missed the last two weeks of Iron Blogger, and I’m not totally enthusiastic about any of the items on my “to blog” list. But, I do enjoy blogging when I actually get into posts, and I’d like to keep updating this blog....
Posts on Made of...
reptyr: Attach a running process to a new terminal Over the last week, I’ve written a nifty tool that I call reptyr. reptyr is a utility for taking an...
over a year ago
10
over a year ago
Over the last week, I’ve written a nifty tool that I call reptyr. reptyr is a utility for taking an existing running program and attaching it to a new terminal. Started a long-running process over ssh, but have to leave and don’t want to interrupt it? Just start a screen, use...
Posts on Made of...
Property-Based Testing Is Fuzzing “Property-based testing” refers to the idea of writing statements that should be true of your code...
over a year ago
10
over a year ago
“Property-based testing” refers to the idea of writing statements that should be true of your code (“properties”), and then using automated tooling to generate test inputs (typically, randomly-generated inputs of an appropriate type), and observe whether the properties hold for...
Posts on Made of...
6.170, CVS, and SVN I’m taking 6.170 Lab in Software Engineering this semester. The course sucks in various ways, but...
over a year ago
10
over a year ago
I’m taking 6.170 Lab in Software Engineering this semester. The course sucks in various ways, but one of the most egregious, in my opinion, is that they force you to use CVS for your version control. Problem sets are distributed by the TAs importing them into your repository, and...
Posts on Made of...
Using Haskell's 'newtype' in C A common problem in software engineering is avoiding confusion and errors when dealing with multiple...
over a year ago
10
over a year ago
A common problem in software engineering is avoiding confusion and errors when dealing with multiple types of data that share the same representation. Classic examples include differentiating between measurements stored in different units, distinguishing between a string of HTML...
Posts on Made of...
Record/Replay testing in Sorbet In 2017 and 2018, I (along with Paul Tarjan and Dmitry Petrashko) was a founding member of the...
over a year ago
10
over a year ago
In 2017 and 2018, I (along with Paul Tarjan and Dmitry Petrashko) was a founding member of the Sorbet project at Stripe to build a gradual static typechecking system for Ruby, with the aim of enhancing productivity on Stripe’s millions of lines of Ruby, and eventually producing a...
Posts on Made of...
Confessions of a programmer: I hate code review Most of the projects I've been working on today have fairly strict code review policies. My work...
over a year ago
10
over a year ago
Most of the projects I've been working on today have fairly strict code review policies. My work requires code review on most of our code, and as we bring on an army of interns for the summer, I've been responsible for reviewing lots of code. Additionally, about five months ago...
Posts on Made of...
Some Android reverse-engineering tools I’ve spent a lot of time this last week staring at decompiled Dalvik assembly. In the process, I...
over a year ago
10
over a year ago
I’ve spent a lot of time this last week staring at decompiled Dalvik assembly. In the process, I created a couple of useful tools that I figure are worth sharing. I’ve been using dedexer instead of baksmali, honestly mainly because the former’s output has fewer blank lines and so...
Posts on Made of...
CVE-2010-4258: Turning denial-of-service into privilege escalation Dan Rosenberg recently released a privilege escalation bug for Linux, based on three different...
over a year ago
10
over a year ago
Dan Rosenberg recently released a privilege escalation bug for Linux, based on three different kernel vulnerabilities I reported recently. This post is about CVE-2010-4258, the most interesting of them, and, as Dan writes, the reason he wrote the exploit in the first place. In...
Posts on Made of...
Reader/reader blocking in reader/writer locks Abstract In writer-priority reader/writer locks, as soon as a single writer enters the acquisition...
over a year ago
10
over a year ago
Abstract In writer-priority reader/writer locks, as soon as a single writer enters the acquisition queue, all future accesses block behind any in-flight reads. Thus, if any readers hold the lock for extended periods of time, this can lead to extreme pauses and loss of throughput...
Posts on Made of...
What's with ML software and pickles? I have spent many years as an software engineer who was a total outsider to machine-learning, but...
a year ago
10
a year ago
I have spent many years as an software engineer who was a total outsider to machine-learning, but with some curiosity and occasional peripheral interactions with it. During this time, a recurring theme for me was horror (and, to be honest, disdain) every time I encountered the...
Posts on Made of...
New reptyr feature: TTY-stealing Ever since I wrote reptyr, I’ve been frustrated by a number of issues in reptyr that I fundamentally...
over a year ago
10
over a year ago
Ever since I wrote reptyr, I’ve been frustrated by a number of issues in reptyr that I fundamentally didn’t know how to solve within the reptyr model. Most annoyingly, reptyr fundamentally only worked on single processes, and could not attach processes with children, making it...
Posts on Made of...
Git in pictures In my previous blog post, I discussed how git is distinctive among version control system in the way...
over a year ago
10
over a year ago
In my previous blog post, I discussed how git is distinctive among version control system in the way in which it makes the backend model that is being used to store data the most important element of the tool, and that experts use it by having the complete model in their head,...
Posts on Made of...
Fun with the preprocessor: CONFIG_IA32_EMULATION hacks in Linux About two months ago, Linux saw CVE-2010-0307, which was a trival denial-of-service attack that...
over a year ago
10
over a year ago
About two months ago, Linux saw CVE-2010-0307, which was a trival denial-of-service attack that could crash essentially any 64-bit Linux machine with 32-bit compatibility enabled. LWN has an excellent writeup of the bug, which turns out to be a subtle error related to the details...
Posts on Made of...
A Brief Introduction to termios If you’re a regular user of the terminal on a UNIX system, there are probably a large number of...
over a year ago
10
over a year ago
If you’re a regular user of the terminal on a UNIX system, there are probably a large number of behaviors you take mostly for granted without really thinking about them. If you press ^C or ^Z it kills or stops the foreground program – unless it’s something like emacs or vim, in...
Posts on Made of...
Navigating the Linux Kernel In response to my query last time, ezyang asked for any tips or tricks I have for finding my way...
over a year ago
9
over a year ago
In response to my query last time, ezyang asked for any tips or tricks I have for finding my way around the Linux kernel. I’m not sure I have much in the way of systematic advice for tracking down the answers to questions about the Linux kernel, but thinking about what I do when...
Posts on Made of...
Lab Notebooking for the Software Engineer A few weeks ago, I wrote that software engineers should keep lab notebooks as they work, in addition...
over a year ago
9
over a year ago
A few weeks ago, I wrote that software engineers should keep lab notebooks as they work, in addition to just documenting things after the fact. Today, I’m going to share the techniques that I’ve found useful to try to get in the habit of lab-notebooking my work, even though I...
Posts on Made of...
DEF CON I’m sitting in the airport in Las Vegas on the way back from DEF CON 15. It’s the first time I’ve...
over a year ago
9
over a year ago
I’m sitting in the airport in Las Vegas on the way back from DEF CON 15. It’s the first time I’ve been at the con, and it wasn’t really what I expected. Frankly, I walked away feeling kinda underwhelmed. Very few of the talks were as technical as I was hoping – they were almost...
Posts on Made of...
Iron Blogger: Blogging for Beer So, you may have noticed that I suddenly started updating this blog for the first time in a while....
over a year ago
9
over a year ago
So, you may have noticed that I suddenly started updating this blog for the first time in a while. The reason is that I’ve recently started an ongoing event with a whole bunch of friends around here to encourage us to blog more. Like so many good ideas, it all started with a...
Posts on Made of...
Computers can be understood Introduction This post attempts to describe a mindset I’ve come to realize I bring to essentially...
over a year ago
9
over a year ago
Introduction This post attempts to describe a mindset I’ve come to realize I bring to essentially all of my work with software. I attempt to articulate this mindset, some of its implications and strengths, and some of the ways in which it’s lead me astray. Software can be...
Posts on Made of...
wpa_supplicant: GUI and wpa_action I’ve made two new interesting discoveries about wpa_supplicant since writing my last blog post on...
over a year ago
9
over a year ago
I’ve made two new interesting discoveries about wpa_supplicant since writing my last blog post on the subject. (Actually, I pretty much made both of them while reading documentation in order to write it, and have been lame about writing them up). Using wpa_gui It turns out that...
Posts on Made of...
Conkeror I’ve recently switched to Conkeror as my primary browser. It started life as a Firefox extension,...
over a year ago
9
over a year ago
I’ve recently switched to Conkeror as my primary browser. It started life as a Firefox extension, but nowadays it’s a standalone app built on top of Mozilla’s xulrunner, so it uses the Gecko rendering engine. What it is, is an emacs implemented in Javascript, for the web. This...
Posts on Made of...
How is duct tape like the force? I’m at Dragon*Con this weekend, my second time here now. I decided that if I was going to Dragon*Con...
over a year ago
9
over a year ago
I’m at Dragon*Con this weekend, my second time here now. I decided that if I was going to Dragon*Con again, I needed to do something in terms of costuming, and I wanted it to be something unique – I wasn’t going to come anywhere near as epic as some of the costumes people pull...
Posts on Made of...
Two Perspectives on the End-to-End Principle Back when I was an undergraduate, as part of a class called “Computer Systems Engineering”, we read...
over a year ago
9
over a year ago
Back when I was an undergraduate, as part of a class called “Computer Systems Engineering”, we read numerous classic papers of systems design. I enjoyed and learned a great deal from many of these papers, but one that paper that has stuck with me in particular was Saltzer et al’s...
Posts on Made of...
Followup to "A Very Subtle Bug" After my previous post got posted to reddit, there was a bunch of interesting discussion there about...
over a year ago
9
over a year ago
After my previous post got posted to reddit, there was a bunch of interesting discussion there about some details I’d handwaved over. This is a quick followup on some the investigation that various people carried out, and the conclusions they reached. In the reddit thread,...
Posts on Made of...
Write testable code by writing generic code Alex Gaynor recently asked this question in an IRC channel I hang out in (a channel which contains...
over a year ago
9
over a year ago
Alex Gaynor recently asked this question in an IRC channel I hang out in (a channel which contains several software engineers nearly as obsessed with software testing as I am): uhh, so I’m writing some code to handle an econnreset… how do I test this? This is a good question!...
Posts on Made of...
My Apollo Bibliography Over the last few years — perhaps not that unusually among the nerds I know — I’ve become...
over a year ago
9
over a year ago
Over the last few years — perhaps not that unusually among the nerds I know — I’ve become increasingly fascinated by the Apollo program (and early space program more generally), and been reading my way through a growing number of books and documentaries written about it. At a...
Posts on Made of...
Configuring dnsmasq with VMware Workstation I love VMware workstation. I keep VMs around for basically every version of every major Linux...
over a year ago
9
over a year ago
I love VMware workstation. I keep VMs around for basically every version of every major Linux distribution, and use them heavily for all kinds of kernel testing and development. This post is a quick writeup of my networking setup with VMware Workstation, using dnsmasq to assign...
Posts on Made of...
Using X forwarding with screen by proxying $DISPLAY If you’re reading this blog, I probably don’t have to explain why I love GNU screen. I can keep a...
over a year ago
9
over a year ago
If you’re reading this blog, I probably don’t have to explain why I love GNU screen. I can keep a long-running session going on a server somewhere, and log in and resume my session without losing any state. I also love X-forwarding. I love being able to log into a remote server...
Posts on Made of...
Building LLVM in 90 seconds using Amazon Lambda Last week, Frederic Cambus wrote about building LLVM quickly on some very large machines,...
over a year ago
9
over a year ago
Last week, Frederic Cambus wrote about building LLVM quickly on some very large machines, culminating in a 2m37s build on a 160-core ARM machine. I don’t have a giant ARM behemoth, but I have been working on a tool I call Llama, which lets you offload computational work –...
Posts on Made of...
Check Plus: An EDSL for writing unit tests in C Check is an excellent unit-testing framework for C code, used by a number of relatively well-known...
over a year ago
9
over a year ago
Check is an excellent unit-testing framework for C code, used by a number of relatively well-known projects. It includes features such as running all tests in separate address spaces (using fork(2)), which means that the test suite can properly report segfaults or similar crashes...
Posts on Made of...
Wordpress tricks: Disabling editing shortcuts One of the major reasons I can’t stand webapps is because I’m a serious emacs junkie, and I can’t...
over a year ago
9
over a year ago
One of the major reasons I can’t stand webapps is because I’m a serious emacs junkie, and I can’t edit text in anything that doesn’t have decent emacs keybindings. Fortunately, on Linux, at least, GTK provides basic emacs keybindings if you add gtk-key-theme-name = "Emacs" to...
Posts on Made of...
Systems that defy detailed understanding Last week, I wrote about the mindset that computer systems can be understood, and behaviors can be...
over a year ago
9
over a year ago
Last week, I wrote about the mindset that computer systems can be understood, and behaviors can be explained, if we’re willing to dig deep enough into the stack of abstractions our software is built atop. Some of the ensuing discussion on Twitter and elsewhere lead me to write...
Posts on Made of...
Test suites as classifiers Suppose we have some codebase we’re considering applying some patch to, and which has a robust and...
over a year ago
9
over a year ago
Suppose we have some codebase we’re considering applying some patch to, and which has a robust and maintained test suite. Considering the patch, we may ask, is this patch acceptable to apply and deploy. By this we mean to ask if the patch breaks any important functionality,...
Posts on Made of...
How I Write Tests The longer I spend as a software engineer, the more obsessive I get about testing. I fully subscribe...
over a year ago
9
over a year ago
The longer I spend as a software engineer, the more obsessive I get about testing. I fully subscribe to the definition of legacy code as “code without an automated test suite.” I’m convinced that the best thing you can do to encourage fast progress in a test suite is to design...
Posts on Made of...
Two kinds of testing While talking about thinking about tests and testing in software engineering recently, I’ve come to...
over a year ago
9
over a year ago
While talking about thinking about tests and testing in software engineering recently, I’ve come to the conclusion that there are (at least) two major ideas and goals that people have when they test or talk about testing. This post aims to outline what I see as these two schools,...
Posts on Made of...
The Window Manager I Want Since I first discovered ratpoison in 2005 or so, I've basically exclusively used tiling window...
over a year ago
9
over a year ago
Since I first discovered ratpoison in 2005 or so, I've basically exclusively used tiling window managers, going through, over the years, StumpWM, Ion 3, and finally XMonad. They've all had various strengths and weaknesses, but I've never been totally happy with any of them. This...
Posts on Made of...
Testing and feedback loops Testing and feedback loops This post tries to set out one mental model I have for thinking about...
over a year ago
9
over a year ago
Testing and feedback loops This post tries to set out one mental model I have for thinking about testing and the purpose testing serves in software engineering, and to explore some of the suggestions of this model. As mentioned in an earlier post, I think a lot about working in...
Posts on Made of...
Design for Testability When designing a new software project, one is often faced with a glut of choices about how to...
over a year ago
9
over a year ago
When designing a new software project, one is often faced with a glut of choices about how to structure it. What should the core abstractions be? How should they interact with each other? In this post, I want to argue for a design heuristic that I’ve found to be a useful guide to...
Posts on Made of...
Some notes on CVE-2010-3081 exploitability Most of you reading this blog probably remember CVE-2010-3081. The bug got an awful lot of publicity...
over a year ago
9
over a year ago
Most of you reading this blog probably remember CVE-2010-3081. The bug got an awful lot of publicity when it was discovered an announced, due to allowing local privilege escalation against virtually all 64-bit Linux kernels in common use at the time. While investigating...
Posts on Made of...
Getting carried away with hack value Recently, I’ve been working on some BarnOwl branches that move more of the core functionality of...
over a year ago
9
over a year ago
Recently, I’ve been working on some BarnOwl branches that move more of the core functionality of BarnOwl into perl code, instead of C (BarnOwl is written in an unholy mix of C and perl code that call each other back and forth obsessively). Moving code into perl has many...
Posts on Made of...
A Very Subtle Bug 6.033, MIT’s class on computer systems, has as one of its catchphrases, “Complex systems fail for...
over a year ago
9
over a year ago
6.033, MIT’s class on computer systems, has as one of its catchphrases, “Complex systems fail for complex reasons”. As a class about designing and building complex systems, it’s a reminder that failure modes are subtle and often involve strange interactions between multiple parts...
Posts on Made of...
Some musings on ORMs I’m pretty sure every developer who has ever worked with a modern database-backed application,...
over a year ago
8
over a year ago
I’m pretty sure every developer who has ever worked with a modern database-backed application, particularly a web-app, has a love/hate relationship with their ORM, or object-relational mapper. On the one hand, ORMs are vastly more pleasant to work with than code that constructs...
Posts on Made of...
Versioning dotfiles in git I’ve been looking for a good solution for versioning and synchronizing my dotfiles between machines...
over a year ago
8
over a year ago
I’ve been looking for a good solution for versioning and synchronizing my dotfiles between machines for some time. I experimented with keeping all of ~ in subversion for a while, but it never worked out well for me. I’ve finally settled on a solution that I like using git, and so...
Posts on Made of...
The architecture of declarative configuration management With the ongoing move towards “infrastructure-as-code” and similar notions, there’s been an ongoing...
over a year ago
8
over a year ago
With the ongoing move towards “infrastructure-as-code” and similar notions, there’s been an ongoing increase in the number and popularity of declarative configuration management tools. This post attempts to lay out my mental model of the conceptual architecture and internal...
Posts on Made of...
Measuring Capacity Through Utilization (This post is cross-posted from Honeycomb’s instrumentation series). One of my favorite concepts...
over a year ago
8
over a year ago
(This post is cross-posted from Honeycomb’s instrumentation series). One of my favorite concepts when thinking about instrumenting a system to understand its overall performance and capacity is what I call “time utilization”. By this I mean: If you look at the behavior of a...
Posts on Made of...
A week with the iPhone I’ve had a new iPhone for about a week now, so I figure it’s time to write up some thoughts about...
over a year ago
8
over a year ago
I’ve had a new iPhone for about a week now, so I figure it’s time to write up some thoughts about it. First, the little things. It is, in typical Apple fashion, an incredibly slick piece of work. Scrolling and zooming images or webpages is simple, easy, and, well, just fun to do...
Posts on Made of...
Distributed cloud builds for everyone CPU cycles are cheaper than they have ever been, and cloud computing has never been more ubiquitous....
over a year ago
8
over a year ago
CPU cycles are cheaper than they have ever been, and cloud computing has never been more ubiquitous. All the major cloud providers offer generous free tiers, and services like GitHub Actions offer free compute resources to open-source repositories. So why do so many developers...
Posts on Made of...
autocutsel As most of you probably know, X has several different mechanisms for copy-paste, used by different...
over a year ago
8
over a year ago
As most of you probably know, X has several different mechanisms for copy-paste, used by different applications in different ways. I know some people who use them deliberately, juggling two pieces of text in different clipboards at once, but for me, it’s always just been...
Posts on Made of...
On git and usability I’ve been helping a number of people get started working with git over the last couple of weeks, as...
over a year ago
8
over a year ago
I’ve been helping a number of people get started working with git over the last couple of weeks, as Ksplice has brought on some new interns, and we’ve had to get them up to speed on our internal git repositories. (As you might expect from a bunch of kernel hackers, we use git for...
Posts on Made of...
Reflections on software performance At this point in my career, I’ve worked on at least three projects where performance was a defining...
over a year ago
8
over a year ago
At this point in my career, I’ve worked on at least three projects where performance was a defining characteristic: Livegrep, Taktician, and Sorbet (I discussed sorbet in particular last time, and livegrep in an earlier post). I’ve also done a lot of other performance work on the...
Posts on Made of...
Exploiting misuse of Python's "pickle" If you program in Python, you’re probably familiar with the pickle serialization library, which...
over a year ago
8
over a year ago
If you program in Python, you’re probably familiar with the pickle serialization library, which provides for efficient binary serialization and loading of Python datatypes. Hopefully, you’re also familiar with the warning printed prominently near the start of pickle’s...
Posts on Made of...
Security doesn't respect abstraction boundaries The fundamental tool of any engineering discipline is the notion of abstraction. If we can build a...
over a year ago
8
over a year ago
The fundamental tool of any engineering discipline is the notion of abstraction. If we can build a set of useful, easily-described behaviors out of a complex system, we can build other systems on top of those pieces, without having to understand to worry about the full complexity...
Posts on Made of...
Three kinds of memory leaks So, you’ve got a program that’s using more and more over time as it runs. Probably you can...
over a year ago
8
over a year ago
So, you’ve got a program that’s using more and more over time as it runs. Probably you can immediately identify this as a likely symptom of a memory leak. But when we say “memory leak”, what do we actually mean? In my experience, apparent memory leaks divide into three broad...
Posts on Made of...
Some thoughts on Quora With the announcement this week that Quora had taken $11 million in VC at an $86 million valuation,...
over a year ago
8
over a year ago
With the announcement this week that Quora had taken $11 million in VC at an $86 million valuation, there’s been an awful lot of attention on Quora. I’ve had an account there and wanted to write up some of my initial thoughts. If you haven’t heard about Quora, it’s yet another...
Posts on Made of...
todo.pl ratmenu broder has been hacking on some better quicksilver integration for Hiveminder using todo.pl. I don’t...
over a year ago
8
over a year ago
broder has been hacking on some better quicksilver integration for Hiveminder using todo.pl. I don’t use a mac, but I don’t see why linux users shouldn’t get fun toys to. So I hacked up the following two-liner that uses todo.pl and ratmenu to pop up a list of tasks, and mark one...
Posts on Made of...
Graceful behavior at capacity Suppose we’ve got a service. We’ll gloss over the details for now, but let’s stipulate that it...
a year ago
8
a year ago
Suppose we’ve got a service. We’ll gloss over the details for now, but let’s stipulate that it accepts requests from the outside world, and takes some action in response. Maybe those requests are HTTP requests, or RPCs, or just incoming packets to be routed at the network layer....
Posts on Made of...
Dear Twitter: Stop screwing over your developers. I really like Twitter. I think it’s a great, fun, service, that helps enable interesting online...
over a year ago
8
over a year ago
I really like Twitter. I think it’s a great, fun, service, that helps enable interesting online communities, and is a surprisingly effective way to spread news and information to lots of people online. One of the things that I’ve loved about Twitter is their API, and how open and...
Posts on Made of...
BlackHat/DEFCON 2011 talk: Breaking out of KVM I’ve posted the final slides from my talk this year at DEFCON and Black Hat, on breaking out of the...
over a year ago
7
over a year ago
I’ve posted the final slides from my talk this year at DEFCON and Black Hat, on breaking out of the KVM Kernel Virtual Machine on Linux. Virtunoid: Breaking out of KVM from Nelson Elhage [Edited 2011-08-11] The code is now available. It should be fairly well-commented, and...
Posts on Made of...
New Blog Location I finally got fed up with Blogger, and am moving this blog to live on Wordpress hosted off of...
over a year ago
7
over a year ago
I finally got fed up with Blogger, and am moving this blog to live on Wordpress hosted off of scripts.mit.edu. In the process of converting everything over and setting up Wordpress I’ve decided I hate it, but hopefully I hate it less than I hate Blogger. We’ll see. I’ve also...
Posts on Made of...
Thoughts On Kubernetes I spent a while the last week porting livegrep.com from running directly AWS to running on...
over a year ago
7
over a year ago
I spent a while the last week porting livegrep.com from running directly AWS to running on Kubernetes on Google’s Cloud Platform (specifically, the google container engine, which provisions and manages the cluster for me). I left this experience profoundly enthusiastic about the...
Posts on Made of...
Regular Expression Search with Suffix Arrays Back in January of 2012, Russ Cox posted an excellent blog post detailing how Google Code Search had...
over a year ago
7
over a year ago
Back in January of 2012, Russ Cox posted an excellent blog post detailing how Google Code Search had worked, using a trigram index. By that point, I’d already implemented early versions of my own livegrep source-code search engine, using a different indexing approach that I...
Posts on Made of...
Some opinionated thoughts on SQL databases People who work with me tend to realize that I have Opinions about databases, and SQL databases in...
over a year ago
7
over a year ago
People who work with me tend to realize that I have Opinions about databases, and SQL databases in particular. Last week, I wrote about a Postgres debugging story and tweeted about AWS’ policy ban on internal use of SQL databases, and had occasion to discuss and debate some of...
Posts on Made of...
What MongoDB got Right MongoDB is perhaps the most-widely-mocked piece of software out there right now. While some of the...
over a year ago
7
over a year ago
MongoDB is perhaps the most-widely-mocked piece of software out there right now. While some of the mockery is out-of-date or rooted in misunderstandings, much of it is well-deserved, and it’s difficult to disagree that much of MongoDB’s engineering is incredibly simplistic,...
Posts on Made of...
Software Engineers should keep lab notebooks Software engineers, as a rule, suck at writing things down. Part of this is training – unlike...
over a year ago
7
over a year ago
Software engineers, as a rule, suck at writing things down. Part of this is training – unlike chemists and biologists who are trailed to obsessively document everything they do in their lab notebooks, computer scientists are taught to document the end results of their work, but...
Posts on Made of...
reptyr: Changing a process's controlling terminal reptyr (announced recently on this blog) takes a process that is currently running in one terminal,...
over a year ago
7
over a year ago
reptyr (announced recently on this blog) takes a process that is currently running in one terminal, and transplants it to a new terminal. reptyr comes from a proud family of similar hacks, and works in the same basic way: We use ptrace(2) to attach to a target process and force...
Posts on Made of...
Automounting sshfs For some time now, many of us around MIT have noticed just how awesome sshfs is. It gives a totally...
over a year ago
7
over a year ago
For some time now, many of us around MIT have noticed just how awesome sshfs is. It gives a totally lightweight way to access the remote filesystem of any machine you have ssh to, without requiring any extra setup on the host. I’ve been running for at least a year now with my...
Posts on Made of...
Property Testing Like AFL In my last last post, I argued that property-based testing and fuzzing are essentially the same...
over a year ago
7
over a year ago
In my last last post, I argued that property-based testing and fuzzing are essentially the same practice, or at least share a lot of commonality. In this followup post, I want to explore that idea a bit more: I’ll first detour into some of my frustrations and hesitations around...
Posts on Made of...
CVE-2007-4573: The Anatomy of a Kernel Exploit CVE-2007-4573 is two years old at this point, but it remains one of my favorite vulnerabilities. It...
over a year ago
7
over a year ago
CVE-2007-4573 is two years old at this point, but it remains one of my favorite vulnerabilities. It was a local privilege-escalation vulnerability on all x86_64 kernels prior to v2.6.22.7. It’s very simple to understand with a little bit of background, and the exploit is...
Posts on Made of...
Transformers for software engineers Ever since its introduction in the 2017 paper, Attention is All You Need, the Transformer model...
over a year ago
7
over a year ago
Ever since its introduction in the 2017 paper, Attention is All You Need, the Transformer model architecture has taken the deep-learning world by storm. Initially introduced for machine translation, it has become the tool of choice for a wide range of domains, including text,...
Posts on Made of...
Why the Sorbet typechecker is fast This is the second in an indefinite series of posts about things that I think went well in the...
over a year ago
7
over a year ago
This is the second in an indefinite series of posts about things that I think went well in the Sorbet project. The previous one covered our testing approach. Sorbet is fast. Numerous of our early users commented specifically on how fast it was, and how much they appreciated this...
Posts on Made of...
A Cursed Bug In my day job at Anthropic, we run relatively large distributed systems to train large language...
over a year ago
6
over a year ago
In my day job at Anthropic, we run relatively large distributed systems to train large language models. One of the joys of using a lot of computing resources, especially on somewhat niche software stacks, is that you spend a lot of time running into the long-tail of bugs which...
Posts on Made of...
A brief look at Linux's security record After the fuss of the last two weeks because of CVE-2010-3081 and CVE-2010-3301, I decided to take a...
over a year ago
6
over a year ago
After the fuss of the last two weeks because of CVE-2010-3081 and CVE-2010-3301, I decided to take a look at a handful of the high-profile privilege escalation vulnerabilities in Linux from the last few years. So, here's a summary of the ones I picked out. There are also a large...
Posts on Made of...
Why node.js is cool (it's not about performance) For the past N months, it seems like there is no new technology stack that is either hotter or more...
over a year ago
6
over a year ago
For the past N months, it seems like there is no new technology stack that is either hotter or more controversial than node.js. node.js is cancer! node.js cures cancer! node.js is bad ass rock star tech!. I myself have given node.js a lot of shit, often involving the phrase...
Posts on Made of...
amd64 and va_arg A while back, I was poking around LLVM bugs, and discovered, to my surprise, that LLVM doesn’t...
over a year ago
6
over a year ago
A while back, I was poking around LLVM bugs, and discovered, to my surprise, that LLVM doesn’t support the va_arg intrinsic, used by functions to accept multiple arguments, at all on amd64. It turns out that clang and llvm-gcc, the compilers that backend to LLVM, have their own...
Posts on Made of...
Efficiency trades off against resiliency What’s the “right” level of CPU utilization for a server? If you look at a monitoring dashboard from...
a year ago
6
a year ago
What’s the “right” level of CPU utilization for a server? If you look at a monitoring dashboard from a well-designed and well-run service, what CPU utilization should we hope to see, averaged over a day or two? It’s a very general question, and it’s not clear it should have a...
Posts on Made of...
Running Tensorflow on AWS GPUs I’ve been spending some time learning deep learning and tensorflow recently, and as part of that...
over a year ago
6
over a year ago
I’ve been spending some time learning deep learning and tensorflow recently, and as part of that project I wanted to be able to train models using GPUs on EC2. This post contains some notes on what it took to get that working. As many people have commented, the environment setup...
Posts on Made of...
A Brief Introduction to termios: Signaling and Job Control (This is part three of a multi-part introduction to termios and terminal emulation on UNIX. Read...
over a year ago
6
over a year ago
(This is part three of a multi-part introduction to termios and terminal emulation on UNIX. Read part 1 or part 2 if you’re new here) For my final entry on termios, I will be looking at job control in the shell (i.e. backgrounding and foreground jobs) and the very closely related...
Posts on Made of...
Why scons is cool I’ve recently started playing with scons a little for some small personal projects. It’s not...
over a year ago
6
over a year ago
I’ve recently started playing with scons a little for some small personal projects. It’s not perfect, but I’ve rapidly come to the conclusion that it’s a probably far better choice than make in many cases. The main exceptions would be cases where you need to integrate into legacy...
Posts on Made of...
Tracking down a memory leak in Ruby's EventMachine At Stripe, we rely heavily on ruby and EventMachine to power various internal and external services....
over a year ago
5
over a year ago
At Stripe, we rely heavily on ruby and EventMachine to power various internal and external services. Over the last several months, we’ve known that one such service suffered from a gradual memory leak, that would cause its memory usage to gradually balloon from a normal ~50MB to...
Posts on Made of...
Write yourself an strace in 70 lines of code Basically anyone who’s used Linux for any amount of time eventually comes to know and love the...
over a year ago
5
over a year ago
Basically anyone who’s used Linux for any amount of time eventually comes to know and love the strace command. strace is the system-call tracer, which traces the calls that a program makes into the kernel in order to interact with the outside world. If you’re not already familiar...
Posts on Made of...
Lightweight Linux Kernel Development with KVM I don’t do a ton of Linux kernel development these days, but I’ve done a fair bit in the past, and...
over a year ago
5
over a year ago
I don’t do a ton of Linux kernel development these days, but I’ve done a fair bit in the past, and picked up a number of useful techniques for doing kernel development in a relatively painless fashion. This blog post is a writeup of the tools and techniques I use when developing...
Posts on Made of...
Implementing a declarative mini-language in the C preprocessor Last time, I announced Check Plus, a declarative language for defining Check tests in C. This time,...
over a year ago
5
over a year ago
Last time, I announced Check Plus, a declarative language for defining Check tests in C. This time, I want to talk about the tricks I used to implement a declarative minilanguage using the C preprocessor (and some GCC extensions). The Problem We want to write some toplevel...