More from the singularity is nearer
“Why are you here on a Sunday?” “John’s in town,” I said. “And he knows I’m looking for him.” I’ve carried this case for five years. When Operant moved its compute out to Long Island—cheaper power, easier permits—it landed in my world by accident. Detective James Reese, Nassau County Police. Since then, every time I think I’ve got a straight line, the story bends. People call it “mind control.” That’s the wrong phrase. You hear that and you start hunting for sci-fi. What you should be hunting for is timing. There are the clean facts. It started with a private investigator caught at night inside Jane Street’s office. He was there to plant a device. Not a camera, not a mic. A flat plastic square the size of a drop ceiling tile, featureless, no lens, no obvious grill. If you tapped it with a knuckle it sounded dead, like dense foam. The FBI took the evidence, said as little as possible, and then Trump dissolved the Bureau and the chain of custody with it. The PI pled to B&E, did eighteen months, swore a friend offered him ten grand and a location. The friend never existed long enough for us to find. While the tile vanished, Operant didn’t. They grew. They put their name on the Ducks’ ballpark, donated to everyone they should, and pushed eight percent of Long Island’s power through their meter. Every time I asked questions, a lawyer answered them. I still have a job mostly because I don’t stick my questions in microphones. But the corporate espionage wasn’t the hook. The hook’s name was Tom Park. Young, gifted, on Operant’s “research” payroll. He died off the roof of their building. We asked for the CCTV. They delayed until the delay became its own story, and when the files came they were grainy enough you could convince yourself resolution had gone out of style. We couldn’t prove a cut. We couldn’t prove a lie. We could see a silhouette on the roof with a phone in his hand, see him put it away, and watch him walk forward like he’d decided to walk forward an hour ago. We pulled his phone records. The carrier said the device never left his parents’ house that night. The family’s router logs said the same thing—MAC associated all evening, steady signal, Netflix on the downstairs TV. At the time of death the rooftop access point didn’t record a roam. No one found a phone with the body. If you’re generous, you call that “inconsistency.” If you’ve been around long enough, you call it “choreography.” I didn’t see the tile again, but I kept a copy of the photos and I stared at the connector pads until I’d memorized the geometry. Four edge pads, power bus shape. Months later, a fire inspection at an Operant satellite site flagged “non-listed luminaires with integrated driver boards.” That’s code for “custom lights.” The brand on the sticker didn’t exist in any registry. It matched nothing you could buy. What does a ceiling tile do if it isn’t a ceiling tile? You can guess: a planar array under plastic, phaseable, a clock inside that doesn’t drift. You don’t need to read thoughts. You need to make the room keep time. We ran a small experiment in our squad room. Nothing that requires approval. We set up a tapping game on a laptop—left or right as quickly as you can when a cue appears. We added a desk lamp we could modulate in the last hundred milliseconds before the cue—no visible flicker, just PWM phase changes—and a piezo disc under the mouse pad that could make a vibration too soft to notice unless you were trying to notice. We told the script to wait until the model thought the subject was likely to pick left, then line up the lamp phase and the tick so “left” felt a hair earlier. The hit rate shifted eight points. The officers said it felt like the computer was “on it” that round. No one said they felt pushed. That’s the thing about timing: when it works, it feels like you were going to do it anyway. I went back to Tom. We subpoenaed what we could: badge swipes, elevator logs, building automation schedules for lights and HVAC. The elevator cabin he rode at 23:41 ran a “door nudge” cycle at floor 35—exact term in the manual. Not a stop, a shove. At the same minute the east conference rooms above ran a luminance ramp—35 to 50 percent and back down—logged as a “pattern test.” Two minutes later the air handlers kicked a “night purge,” unscheduled. The lobby mic’s spectrogram shows the change as a clean band sliding up. None of those facts make a person move. Together they draw a rhythm line through a building. We never found what Tom had in his hand, his “not a phone,” but a year after his death, one of their contractors quit and dumped an issue tracker on a public repo by accident. It was up for an hour before it vanished, but the internet is full of raccoons, and one of them sent me a ZIP. Half the issues were boring—install scripts, driver mismatches, bad GPIO pull-ups. The other half had words like “phase,” “latency,” “confidence gate,” “avoid visible artifacts,” “EEG-free,” and the tag “ROOM.” There was a set of comments on a bug titled “End Token Misfires.” The engineers were arguing about whether printing the full predicted sequence at the start of a session biased the subject into making it true. One person said that was the point. Another said if your only wins are the ones you can cause you aren’t measuring prediction anymore, you’re measuring control. The thread ends with a “resolved—won’t fix.” Mind control isn’t the right term. It makes people look for sci-fi. The right term is “nudge,” the one the elevator manuals use. You put your thumb on the timing. You don’t push the person; you lean on the moment. Tom stood on a roof with a clock in his pocket that belonged to the room, and a room that belonged to the company, and a company that had learned you can make a person look like a prediction if you take away all the moments where they would have surprised you. Sometimes I think the real trick isn’t the tile or the lights. It’s the bookkeeping. You arrange your systems so that there’s nothing to subpoena. The carrier shows a phone at home. The Wi-Fi shows a phone at home. The building shows a test pattern and a purge cycle and a polite door. Nothing is illegal in a log file. I keep a copy of that ZIP on a USB stick in my desk. There’s a folder called “SAFE_GATES” with a README someone wrote in plain English. “Do not schedule interventions if subject arousal > threshold. Do not schedule end token if subject mentions self-harm. Cooldown after consecutive errors.” Half the rules are commented out. The most recent commit message is just a shrug emoji. The worst part is how ordinary it all is. The elevator nudge. The lamp nudge. The HVAC tone. The not-a-phone. If you want to find the devil, you don’t go looking for horns. You go looking for clock edges. I told my sergeant I was taking the rest of the day. I stopped by a hardware store and bought a dimmer I knew I could open, and a roll of white tape. Back at the office I put a strip of tape on the lamp in interview room two, covering the LED that the supplier put there to indicate “smart mode.” We don’t use smart mode. We don’t use anything with a mode. When I left, the room looked like every other room. That’s the point. You only notice the timing when it slips. On the way home I drove past the Ducks ballpark. Operant’s name on the sign looked like every other naming deal. Families walking in, kids with foam fingers, warm light over the field. If you didn’t know to look, you’d think it was all just baseball.
Mom 12:37 – hey when are you getting home? Dave You set the disappearing message time to 3 hours hey you doing better lol yea i really didn't sleep much what's up u didn't set timeout for that yea is what Brian said true what did he say about how Tom worked at operant? yea why? you know that's where my dad worked and he kind of went crazy too do you know what he did there? not rly my mom gets real upset when i bring it up it was some math shit with magnets wanna come over and ask her lol I did not want to have a conversation with Dave’s mother. 12:55 – haha im good but im just chilling at home if you want to come by here The doorbell rang again. Resolving to be less of a pussy, I answered it. I was prepared to talk to the cops. Polite, short answers, step outside and lock the door, find out what they want. Not a pussy. Not a pussy. Not a pussy. It wasn’t the cops. It was my Mom’s friend Anne, and I told her she wasn’t here. It was always strange to me that that generation would just drop by. Like she didn’t text her first? She said she was in the neighborhood and had extra bagels she wanted to drop off. I thought about telling her that I hadn’t heard from my Mom since yesterday and that she didn’t reply to my text, but decided against it. I didn’t know the dynamic of my Mom’s friend group. Maybe she is out sleeping with Anne’s husband or something. I didn’t want to be a link in the chain of Anne finding out. I was vague but very polite. Anne left the bagels. I didn’t touch the bag. I went up the stairs to my Mom’s room. Did I mention how much I like true crime? It’s probably done bad things for me personality wise. I know that the people on there are out of the normal distribution of people, but those podcasts are one of my only exposures to the outside world. The world beyond this little slice of Brooklyn. So you kind of start thinking everyone is like that. I’d always just assumed my Dad was like, a Wall Street guy. Boring. Get money, fuck bitches. When I was little we had tons of money. We lived in a huge house in Cobble Hill. I flew first class to Europe when I was 7. We spent a week on a yacht in Monaco. My mom loved the luxury lifestyle, and would put up with a lot of my Dad’s eccentricities to keep it. When he left she didn’t seem that upset though. I think the money was still coming in from him, which was the main thing she cared about. It clearly wasn’t as much, we moved out to Sheepshead Bay and never went back to Europe. But she didn’t work and I always got good birthday presents, and she never said anything bad about my Dad, so I assume that’s where the money was coming from. The first drawer I opened had sex toys in it. I saw a vibrator and a butt plug before I quickly closed the drawer. The second drawer had socks. The third drawer had tons of scattered papers. My college rejections. Some essays from high school. A note written in crayon about how I wanted a Nintendo Switch for Christmas. I guess this was the “me” drawer. The fourth drawer was papers, but more organized. My parents marriage certificate. My mom’s birth certificate. My old passport. As far as I knew, they never got a divorce. He just left. Then, something I didn’t know. A document entitling one Jessica Baker to $10,000 per month, to be paid out on the first of every month by the Triangle Trust, for the rest of her natural life, or until the trust is dissolved. That was nice of my Dad. I went through the rest of the drawers, but didn’t find anything else interesting. I put everything back as carefully as I could. I considered that someone might dust for fingerprints. I wondered if I did anything illegal. I live here, right? I checked on the text message to my Mom and noticed that it hadn’t been delivered. This was really unlike her. Sometimes she’d go out drinking and meet a guy and stay out all night, but she’d always at least text me by the morning when she sobered up. 2:14 – are you okay? Not delivered. Maybe her phone died? Nah but it’s the afternoon she probably would have charged it by now. I tried calling. Straight to voicemail. I checked the Mercedes app to see where her car was. She’d let me take the car sometimes, so we were both on the app. It asked me to login. I copied the password from 1password. Incorrect Password Maybe she changed it? I tried to set her up on 1password but she didn’t get it. She’d just reset the passwords when she needed to login. Ugh, logging into stuff is the worst. I clicked the reset password and typed in the e-mail. There is no account with that e-mail Okay, that doesn’t make sense. My Mom and I shared an e-mail for this stuff, and she wouldn’t change it without telling me. I clicked forgot e-mail. It needed the VIN of the car. The title was in the fourth drawer with the other papers. I went and got it and typed in the VIN. The e-mail associated with your account is: skinner666@gmail.com A jolt of anxiety coarsed through my body. I’d never seen that e-mail address in my life.
Dave 2:41 – yoo i took a nap wanna play minecraft? actually i need your help with something what just come over kk i gotta shower Dave knew stuff about computers. Don’t let the being high all the time fool you, he was the smartest guy in our friend group. And he was the most chronically online. He showed up on his bike 20 minutes later. “Yo,” I yelled to him across the lawn. “Dude what’s up? You look rattled” “My mom is missing and she isn’t answering her phone. Then I tried to track her on the car app, but somebody changed the e-mail.” Dave came inside the house (I locked the door behind him) and opened his laptop at the kitchen table. I messaged him the VIN, and he tried the reset e-mail again but on the website. Same result. Same skinner666 “What did the e-mail used to be? Do you have access to that account?” “Yea,” I brought up the password to bakerfamily43@gmail.com on my 1Password and Dave logged in on his laptop. We saw like 15 e-mails notifying that account details were changed for various services. Ranging from 2 to 3 am last night. Most interesting was the first e-mail. It was from Google saying they blocked a login attempt to this account. From Atlanta, Georgia. Dave explained, “They probably tried to log in to the GMail, but when it didn’t work, they changed the e-mails on other services to an account they controlled” “From Atlanta? Who’s in Atlanta? They got hackers down there?” I was imagining…well never mind, you know what I was imagining. “It’s probably a VPN, hang on, I’ll check the IP.” Dave copy pasted some numbers from the Google e-mail into the terminal. “Yea, NordVPN exit node.” “What’s a VPN?” “No wonder you can’t find hookers on the Internet.” I’d asked Dave about this once. He was high. I didn’t think he remembered. “Even Brian knows what a VPN is.” “You think we should call the police,” I asked. But I already knew Dave’s answer. Dave hated the police more than all of us. Back before the suicide, his mother would take him to Black Lives Matter protests. And as dumb as it was, those were probably some of the best memories he had with his family. It was a day outside, there were people, there were food stalls. And his mother was happy. Or maybe angry? But not depressed. “Fuck 12,” he mumbled. It wasn’t just the politics. After Tom’s suicide, the cops harassed his family. Well it wasn’t the police, it was child protective services. It became standard practice to investigate all cases where young people who still lived at home killed themselves. Nothing ended up happening with the investigation, but to Dave, one government mooching pig was the same as the next, and none of them were on his side. “Okay, no police. I don’t like them either. You think my mom is okay?” Dave thought for a minute, “I think I know how to find her car” As the number of sensors on the Internet grew, the availability of data about the real world skyrocketed. Tons of people were willing to put cameras outside their house or dashcams on their car to earn a couple dollars. There were news articles predicting that this would end all crime. But of course this isn’t what happened. Ending crime was always more a question of will than a question of ability. And most people never made back the money they spent on the camera, the data wasn’t all that useful and not that many people bought it. But if you knew where to look, you could access it. “Do you have the license plate number of the car?” I didn’t know it, but I went back upstairs to get the title. I also grabbed the Triangle Trust document. Dave typed it into website billing itself as the world’s data marketplace. “Search millions of dashcams for a license plate” was one of the options, along with a bunch of other video search options, including “search millions of CCTV cameras for perky nipples.” It seemed like there was a whole Internet that I didn’t know about. $0.78 for the query. He typed in his credit card. I offered to pay him back. He thanked me for the bagel. “Three hits in the last 24 hours” It was $8 each to download the video clips. One of them wouldn’t download, it said “Dashcam Offline.” It still took his $8 for that clip. He downloaded the other two and dragged them into ChatGPT. No point in watching them manually. Clip 1 Timestamp (overlay): ~5:36:12 PM, Sat Jul 12, 2031 Location (inferred): Manhattan Bridge lower roadway, westbound into Manhattan. Overhead signage OCR fragments: “CANAL ST / CHINATOWN”; steel truss pattern matches the bridge; skyline and arch glimpses align. What’s visible: Mercedes (body/DRL signature consistent) in center lane, moderate traffic, dry pavement, dusk light. No notable tail vehicle persists across frames. Clip 2 Timestamp (overlay): ~11:29:47 PM, Sat Jul 12, 2031 Location (inferred): Sheepshead Bay corridor. Streetfront OCR: “EMMONS AVE,” “BAY DELI,” and a marina awning; sodium-vapor lighting; parked fishing boats visible for 2–3 frames. What’s visible: Mercedes eastbound along Emmons Ave, signals right at the next intersection and turns toward a residential side street (likely Knapp/Bragg area; exact street name unreadable due to glare). Traffic light cycle and storefront shutters consistent with late-night return. I pondered, “That doesn’t make sense. That second clip is right by our house, but I got home an hour after that and she wasn’t here. What’s the third clip?” “It won’t let me download it. Because of how this marketplace works, the video file isn’t uploaded to them until somebody buys it. If the user’s device is offline, we can’t download it.” “Do we know anything about it?” “Not really. Sometimes the user has a profile, but all he has is a username. It’s liducksfan” For once, I could actually be useful. “The Long Island Ducks! I went to a game once!” I wasn’t actually that useful, Dave had already Googled “liducks” and that was the obvious first hit. “So she was on Long Island?” “You’re the true crime guy. I’m just the tech guy.” I had a hunch. I think she came home around 11:30, something happened here, either she saw something or met someone, and she left again and went to Long Island, all before I got home. I looked around for James Reese’s business card. I couldn’t find it. However, in plain sight, there was a note from my mother on the refrigerator. I guess it had been there the whole time; wow I’m a bad detective. “Met a guy. Going to the Hamptons. Might be out of cell service.” and then Mom in a heart. She always signs her notes like that. I showed the note to Dave. He shrugged and asked if he could have one of the bagels that Anne left. Sure. Was this just me being paranoid, or is that the exact note my mother would leave if she didn’t want me to worry and that’s not at all what happened? Who logged into her accounts? Who was skinner666?
7 years earlier Jonathan sat there while his Mercedes S-Class with DRIVE PILOT drove him across the Brooklyn Bridge to Wall Street. He had been working at Operant Capital for 10 years. The idea was simple. Predict the market, trade based on those predictions, and get rich. The implementation however, was not as simple as the idea. The market doesn’t work like physics. The market moves based on the thoughts and feelings of billions of humans, and they were all trying to do this same prediction thing. That’s what made the economy. Everyone was basically doing this algorithm on some level. And even worse, this was a Red Queen’s race, where the tactics that worked last year didn’t work this year. Everyone was predicting everyone else predicting the market. Except everyone was not as smart as Jonathan. He was a child prodigy. He got his first IMO Gold when he was 14. He graduated from MIT in 3 years. And he wanted a job where he was surrounded by other people as smart as him. He’d been working on the RPS project for the last three years. As one of the more senior people there, he got to work on the more speculative research. Operant was blessed with basically an unlimited budget; the bounties of providing a valuable capital allocation service to the market. Jonathan believed in the mission. Well, mostly anyway. Make markets more efficient. They weren’t a high frequency trading shop skimming pennies off of each transaction by being fast. They weren’t doing arbitrage. They were a market intelligence service. They knew where capital should be allocated before the market did, and in the grand scheme of things, they charged so little for their service. Most of the tricks were standard. The same run of the mill models everyone in the industry was using. But they felt that their differentiator was always remembering that the market doesn’t obey physical laws. Technical analysis is complete bullshit. This is a simple betting game played against other agents. Which is the line of thinking that led to the RPS project. RPS stood for rock-paper-scissors. A game with a simple Nash equilibrium. One-third, one-third, one-third. Deviate at all from that strategy, and your opponent can exploit you. But if your opponent isn’t playing that strategy, you have to deviate to exploit them. Shortly after MRI machines were invented, people started using them to see if they could read minds. They’d put someone in an MRI and give them two buttons, say a red and a blue. Of course the machine knew which button you would press before you pressed it. But what was crazier is that the machine knew which button you would press before you were even aware of your choice. The obvious follow up question is, how much before? Seconds was very believable. But could you know as soon as the subject walked in the room? At the time there wasn’t really a way to explore this. Aside from the 55% prior on red, there wasn’t much more to say. Rock-paper-scissors was basically the same game. Humans do have unequal priors, preferring rock at 35.4%. But is it possible that someone was a scissors type guy? Maybe you could read their social media posts and tell. Maybe you could just tell by looking at their face. The goal of the RPS project was to become perfect at rock-paper-scissors. And the results were very promising. The computer had a win rate of 86%, with a first round win rate of 54%. Once you were playing multiple games it was easy, but Jonathan was particularly proud of the first round win rate. However, 54% wasn’t close to 100%. It was better than random, sure. But this showed diminishing returns. Maybe the data just wasn’t there. Maybe you had to look inside to spot a scissors type guy. The new research direction was outfitting the room with SQUIDs, very sensitive magnetometers capable of measuring fields in the femtotesla range. The rules for the project disallowed things that were obvious and bulky like MRIs. But anything that could be put in a normal room was fair game. It was Monday and the more junior people had been there working all weekend. Jonathan had taken his wife and son camping. When he got to the office, he was ambushed as soon as he stepped out of the elevator. “You have to try it,” said Tom with the energy of a kid seeing presents on Christmas morning. Tom was 17, also an IMO Gold winner, and probably the brightest guy on the team. He didn’t go to college, he came straight to work at Operant. Real LeBron energy. Jonathan stepped into the room. He played scissors. He lost. He played scissors again. He lost. He played rock. He lost. He played paper. He lost. He went back to scissors. He lost. He played scissors again. He lost. He stepped back to take a breath. He played rock. He lost. He played rock again. He lost. He walked out the room. Tom handed him a piece of paper. At the top it had the date and 10:09 AM, and had SSRPSSRRE on it. 10:09 was when he walked into the room, and those were all the moves he played. “What’s this E?” “That’s end. It’s when you’d walk out of the room.” “So are you saying…it predicted all of this before I even played one game?” Tom shrugged, “If it knows all the predictions are going to be correct, why does it need any feedback?”
More in programming
Although it looks really good, I have not yet tried the Jujutsu (jj) version control system, mainly because it’s not yet clearly superior to Magit. But I have been following jj discussions with great interest. One of the things that jj has not yet tackled is how to do better than git refs / branches / tags. As I underestand it, jj currently has something like Mercurial bookmarks, which are more like raw git ref plumbing than a high-level porcelain feature. In particular, jj lacks signed or annotated tags, and it doesn’t have branch names that always automatically refer to the tip. This is clearly a temporary state of affairs because jj is still incomplete and under development and these gaps are going to be filled. But the discussions have led me to think about how git’s branches are unsatisfactory, and what could be done to improve them. branch merge rebase squash fork cover letters previous branch workflow questions branch One of the huge improvements in git compared to Subversion was git’s support for merges. Subversion proudly advertised its support for lightweight branches, but a branch is not very useful if you can’t merge it: an un-mergeable branch is not a tool you can use to help with work-in-progress development. The point of this anecdote is to illustrate that rather than trying to make branches better, we should try to make merges better and branches will get better as a consequence. Let’s consider a few common workflows and how git makes them all unsatisfactory in various ways. Skip to cover letters and previous branch below where I eventually get to the point. merge A basic merge workflow is, create a feature branch hack, hack, review, hack, approve merge back to the trunk The main problem is when it comes to the merge, there may be conflicts due to concurrent work on the trunk. Git encourages you to resolve conflicts while creating the merge commit, which tends to bypass the normal review process. Git also gives you an ugly useless canned commit message for merges, that hides what you did to resolve the conflicts. If the feature branch is a linear record of the work then it can be cluttered with commits to address comments from reviewers and to fix mistakes. Some people like an accurate record of the history, but others prefer the repository to contain clean logical changes that will make sense in years to come, keeping the clutter in the code review system. rebase A rebase-oriented workflow deals with the problems of the merge workflow but introduces new problems. Primarily, rebasing is intended to produce a tidy logical commit history. And when a feature branch is rebased onto the trunk before it is merged, a simple fast-forward check makes it trivial to verify that the merge will be clean (whether it uses separate merge commit or directly fast-forwards the trunk). However, it’s hard to compare the state of the feature branch before and after the rebase. The current and previous tips of the branch (amongst other clutter) are recorded in the reflog of the person who did the rebase, but they can’t share their reflog. A force-push erases the previous branch from the server. Git forges sometimes make it possible to compare a branch before and after a rebase, but it’s usually very inconvenient, which makes it hard to see if review comments have been addressed. And a reviewer can’t fetch past versions of the branch from the server to review them locally. You can mitigate these problems by adding commits in --autosquash format, and delay rebasing until just before merge. However that reintroduces the problem of merge conflicts: if the autosquash doesn’t apply cleanly the branch should have another round of review to make sure the conflicts were resolved OK. squash When the trunk consists of a sequence of merge commits, the --first-parent log is very uninformative. A common way to make the history of the trunk more informative, and deal with the problems of cluttered feature branches and poor rebase support, is to squash the feature branch into a single commit on the trunk instead of mergeing. This encourages merge requests to be roughly the size of one commit, which is arguably a good thing. However, it can be uncomfortably confining for larger features, or cause extra busy-work co-ordinating changes across multiple merge requests. And squashed feature branches have the same merge conflict problem as rebase --autosquash. fork Feature branches can’t always be short-lived. In the past I have maintained local hacks that were used in production but were not (not yet?) suitable to submit upstream. I have tried keeping a stack of these local patches on a git branch that gets rebased onto each upstream release. With this setup the problem of reviewing successive versions of a merge request becomes the bigger problem of keeping track of how the stack of patches evolved over longer periods of time. cover letters Cover letters are common in the email patch workflow that predates git, and they are supported by git format-patch. Github and other forges have a webby version of the cover letter: the message that starts off a pull request or merge request. In git, cover letters are second-class citizens: they aren’t stored in the repository. But many of the problems I outlined above have neat solutions if cover letters become first-class citizens, with a Jujutsu twist. A first-class cover letter starts off as a prototype for a merge request, and becomes the eventual merge commit. Instead of unhelpful auto-generated merge commits, you get helpful and informative messages. No extra work is needed since we’re already writing cover letters. Good merge commit messages make good --first-parent logs. The cover letter subject line works as a branch name. No more need to invent filename-compatible branch names! Jujutsu doesn’t make you name branches, giving them random names instead. It shows the subject line of the topmost commit as a reminder of what the branch is for. If there’s an explicit cover letter the subject line will be a better summary of the branch as a whole. I often find the last commit on a branch is some post-feature cleanup, and that kind of commit has a subject line that is never a good summary of its feature branch. As a prototype for the merge commit, the cover letter can contain the resolution of all the merge conflicts in a way that can be shared and reviewed. In Jujutsu, where conflicts are first class, the cover letter commit can contain unresolved conflicts: you don’t have to clean them up when creating the merge, you can leave that job until later. If you can share a prototype of your merge commit, then it becomes possible for your collaborators to review any merge conflicts and how you resolved them. To distinguish a cover letter from a merge commit object, a cover letter object has a “target” header which is a special kind of parent header. A cover letter also has a normal parent commit header that refers to earlier commits in the feature branch. The target is what will become the first parent of the eventual merge commit. previous branch The other ingredient is to add a “previous branch” header, another special kind of parent commit header. The previous branch header refers to an older version of the cover letter and, transitively, an older version of the whole feature branch. Typically the previous branch header will match the last shared version of the branch, i.e. the commit hash of the server’s copy of the feature branch. The previous branch header isn’t changed during normal work on the feature branch. As the branch is revised and rebased, the commit hash of the cover letter will change fairly frequently. These changes are recorded in git’s reflog or jj’s oplog, but not in the “previous branch” chain. You can use the previous branch chain to examine diffs between versions of the feature branch as a whole. If commits have Gerrit-style or jj-style change-IDs then it’s fairly easy to find and compare previous versions of an individual commit. The previous branch header supports interdiff code review, or allows you to retain past iterations of a patch series. workflow Here are some sketchy notes on how these features might work in practice. One way to use cover letters is jj-style, where it’s convenient to edit commits that aren’t at the tip of a branch, and easy to reshuffle commits so that a branch has a deliberate narrative. When you create a new feature branch, it starts off as an empty cover letter with both target and parent pointing at the same commit. Alternatively, you might start a branch ad hoc, and later cap it with a cover letter. If this is a small change and rebase + fast-forward is allowed, you can edit the “cover letter” to contain the whole change. Otherwise, you can hack on the branch any which way. Shuffle the commits that should be part of the merge request so that they occur before the cover letter, and edit the cover letter to summarize the preceding commits. When you first push the branch, there’s (still) no need to give it a name: the server can see that this is (probably) going to be a new merge request because the top commit has a target branch and its change-ID doesn’t match an existing merge request. Also when you push, your client automatically creates a new instance of your cover letter, adding a “previous branch” header to indicate that the old version was shared. The commits on the branch that were pushed are now immutable; rebases and edits affect the new version of the branch. During review there will typically be multiple iterations of the branch to address feedback. The chain of previous branch headers allows reviewers to see how commits were changed to address feedback, interdiff style. The branch can be merged when the target header matches the current trunk and there are no conflicts left to resolve. When the time comes to merge the branch, there are several options: For a merge workflow, the cover letter is used to make a new commit on the trunk, changing the target header into the first parent commit, and dropping the previous branch header. Or, if you like to preserve more history, the previous branch chain can be retained. Or you can drop the cover letter and fast foward the branch on to the trunk. Or you can squash the branch on to the trunk, using the cover letter as the commit message. questions This is a fairly rough idea: I’m sure that some of the details won’t work in practice without a lot of careful work on compatibility and deployability. Do the new commit headers (“target” and “previous branch”) need to be headers? What are the compatibility issues with adding new headers that refer to other commits? How would a server handle a push of an unnamed branch? How could someone else pull a copy of it? How feasible is it to use cover letter subject lines instead of branch names? The previous branch header is doing a similar job to a remote tracking branch. Is there an opportunity to simplify how we keep a local cache of the server state? Despite all that, I think something along these lines could make branches / reviews / reworks / merges less awkward. How you merge should me a matter of your project’s preferred style, without interference from technical limitations that force you to trade off one annoyance against another. There remains a non-technical limitation: I have assumed that contributors are comfortable enough with version control to use a history-editing workflow effectively. I’ve lost all perspective on how hard this is for a newbie to learn; I expect (or hope?) jj makes it much easier than git rebase.
I’ve long been interested in new and different platforms. I ran Debian on an Alpha back in the late 1990s and was part of the Alpha port team; then I helped bootstrap Debian on amd64. I’ve got somewhere around 8 Raspberry Pi devices in active use right now, and the free NNCPNET Internet email service … Continue reading ARM is great, ARM is terrible (and so is RISC-V) →
In my first interview out of college I was asked the change counter problem: Given a set of coin denominations, find the minimum number of coins required to make change for a given number. IE for USA coinage and 37 cents, the minimum number is four (quarter, dime, 2 pennies). I implemented the simple greedy algorithm and immediately fell into the trap of the question: the greedy algorithm only works for "well-behaved" denominations. If the coin values were [10, 9, 1], then making 37 cents would take 10 coins in the greedy algorithm but only 4 coins optimally (10+9+9+9). The "smart" answer is to use a dynamic programming algorithm, which I didn't know how to do. So I failed the interview. But you only need dynamic programming if you're writing your own algorithm. It's really easy if you throw it into a constraint solver like MiniZinc and call it a day. int: total; array[int] of int: values = [10, 9, 1]; array[index_set(values)] of var 0..: coins; constraint sum (c in index_set(coins)) (coins[c] * values[c]) == total; solve minimize sum(coins); You can try this online here. It'll give you a prompt to put in total and then give you successively-better solutions: coins = [0, 0, 37]; ---------- coins = [0, 1, 28]; ---------- coins = [0, 2, 19]; ---------- coins = [0, 3, 10]; ---------- coins = [0, 4, 1]; ---------- coins = [1, 3, 0]; ---------- Lots of similar interview questions are this kind of mathematical optimization problem, where we have to find the maximum or minimum of a function corresponding to constraints. They're hard in programming languages because programming languages are too low-level. They are also exactly the problems that constraint solvers were designed to solve. Hard leetcode problems are easy constraint problems.1 Here I'm using MiniZinc, but you could just as easily use Z3 or OR-Tools or whatever your favorite generalized solver is. More examples This was a question in a different interview (which I thankfully passed): Given a list of stock prices through the day, find maximum profit you can get by buying one stock and selling one stock later. It's easy to do in O(n^2) time, or if you are clever, you can do it in O(n). Or you could be not clever at all and just write it as a constraint problem: array[int] of int: prices = [3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5, 8]; var int: buy; var int: sell; var int: profit = prices[sell] - prices[buy]; constraint sell > buy; constraint profit > 0; solve maximize profit; Reminder, link to trying it online here. While working at that job, one interview question we tested out was: Given a list, determine if three numbers in that list can be added or subtracted to give 0? This is a satisfaction problem, not a constraint problem: we don't need the "best answer", any answer will do. We eventually decided against it for being too tricky for the engineers we were targeting. But it's not tricky in a solver; include "globals.mzn"; array[int] of int: numbers = [3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5, 8]; array[index_set(numbers)] of var {0, -1, 1}: choices; constraint sum(n in index_set(numbers)) (numbers[n] * choices[n]) = 0; constraint count(choices, -1) + count(choices, 1) = 3; solve satisfy; Okay, one last one, a problem I saw last year at Chipy AlgoSIG. Basically they pick some leetcode problems and we all do them. I failed to solve this one: Given an array of integers heights representing the histogram's bar height where the width of each bar is 1, return the area of the largest rectangle in the histogram. The "proper" solution is a tricky thing involving tracking lots of bookkeeping states, which you can completely bypass by expressing it as constraints: array[int] of int: numbers = [2,1,5,6,2,3]; var 1..length(numbers): x; var 1..length(numbers): dx; var 1..: y; constraint x + dx <= length(numbers); constraint forall (i in x..(x+dx)) (y <= numbers[i]); var int: area = (dx+1)*y; solve maximize area; output ["(\(x)->\(x+dx))*\(y) = \(area)"] There's even a way to automatically visualize the solution (using vis_geost_2d), but I didn't feel like figuring it out in time for the newsletter. Is this better? Now if I actually brought these questions to an interview the interviewee could ruin my day by asking "what's the runtime complexity?" Constraint solvers runtimes are unpredictable and almost always than an ideal bespoke algorithm because they are more expressive, in what I refer to as the capability/tractability tradeoff. But even so, they'll do way better than a bad bespoke algorithm, and I'm not experienced enough in handwriting algorithms to consistently beat a solver. The real advantage of solvers, though, is how well they handle new constraints. Take the stock picking problem above. I can write an O(n²) algorithm in a few minutes and the O(n) algorithm if you give me some time to think. Now change the problem to Maximize the profit by buying and selling up to max_sales stocks, but you can only buy or sell one stock at a given time and you can only hold up to max_hold stocks at a time? That's a way harder problem to write even an inefficient algorithm for! While the constraint problem is only a tiny bit more complicated: include "globals.mzn"; int: max_sales = 3; int: max_hold = 2; array[int] of int: prices = [3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5, 8]; array [1..max_sales] of var int: buy; array [1..max_sales] of var int: sell; array [index_set(prices)] of var 0..max_hold: stocks_held; var int: profit = sum(s in 1..max_sales) (prices[sell[s]] - prices[buy[s]]); constraint forall (s in 1..max_sales) (sell[s] > buy[s]); constraint profit > 0; constraint forall(i in index_set(prices)) (stocks_held[i] = (count(s in 1..max_sales) (buy[s] <= i) - count(s in 1..max_sales) (sell[s] <= i))); constraint alldifferent(buy ++ sell); solve maximize profit; output ["buy at \(buy)\n", "sell at \(sell)\n", "for \(profit)"]; Most constraint solving examples online are puzzles, like Sudoku or "SEND + MORE = MONEY". Solving leetcode problems would be a more interesting demonstration. And you get more interesting opportunities to teach optimizations, like symmetry breaking. Because my dad will email me if I don't explain this: "leetcode" is slang for "tricky algorithmic interview questions that have little-to-no relevance in the actual job you're interviewing for." It's from leetcode.com. ↩
I’m something of a filesystem geek, I guess. I first wrote about ZFS on Linux 14 years ago, and even before I used ZFS, I had used ext2/3/4, jfs, reiserfs, xfs, and no doubt some others. I’ve also used btrfs. I last posted about it in 2014, when I noted it has some advantages over … Continue reading btrfs on a Raspberry Pi →