More from Greg Brockman
AI has recently crossed a utility threshold, where cutting-edge models such as GPT-3, Codex, and DALL-E 2 are actually useful and can perform tasks computers cannot do any other way. The act of producing these models is an exploration of a new frontier, with the discovery of unknown capabilities, scientific progress, and incredible product applications as the rewards. And perhaps most exciting for me personally, because the field is fundamentally about creating and studying software systems, great engineers are able to contribute at the same level as great researchers to future progress. “A self-learning AI system.” by DALL-E 2. I first got into software engineering because I wanted to build large-scale systems that could have a direct impact on people’s lives. I attended a math research summer program shortly after I started programming, and my favorite result of the summer was a scheduling app I built for people to book time with the professor. Specifying every detail of how a program should work is hard, and I’d always dreamed of one day putting my effort into hypothetical AI systems that could figure out the details for me. But after taking one look at the state of the art in AI in 2008, I knew it wasn’t going to work any time soon and instead started building infrastructure and product for web startups. DALL-E 2’s rendition of “The two great pillars of the house of artificial intelligence” (which according to my co-founder Ilya Sutskever are great engineering, and great science using this engineering) It’s now almost 15 years later, and the vision of systems which can learn their own solutions to problems is becoming incrementally more real. And perhaps most exciting is the underlying mechanism by which it’s advancing — at OpenAI, and the field generally, precision execution on large-scale models is a force multiplier on AI progress, and we need more people with strong software skills who can deliver these systems. This is because we are building AI models out of unprecedented amounts of compute; these models in turn have unprecedented capabilities, we can discover new phenomena and explore the limits of what these models can and cannot do, and then we use all these learnings to build the next model. “Harnessing the most compute in the known universe” by DALL-E 2 Harnessing this compute requires deep software skills and the right kind of machine learning knowledge. We need to coordinate lots of computers, build software frameworks that allow for hyperoptimization in some cases and flexibility in others, serve these models to customers really fast (which is what I worked on in 2020), and make it possible for a small team to manage a massive system (which is what I work on now). Engineers with no ML background can contribute from the day they join, and the more ML they pick up the more impact they have. The OpenAI environment makes it relatively easy to absorb the ML skills, and indeed, many of OpenAI’s best engineers transferred from other fields. All that being said, AI is not for every software engineer. I’ve seen about a 50-50 success rate of engineers entering this field. The most important determiner is a specific flavor of technical humility. Many dearly-held intuitions from other domains will not apply to ML. The engineers who make the leap successfully are happy to be wrong (since it means they learned something), aren’t afraid not to know something, and don’t push solutions that others resist until they’ve gathered enough intuition to know for sure that it matches the domain. “A beaver who has humbly recently become a machine learning engineer” by DALL-E 2 I believe that AI research is today by far the most impactful place for engineers who want to build useful systems to be working, and I expect this statement to become only more true as progress continues. If you’d like to work on creating the next generation of AI models, email me (gdb@openai.com) with any evidence of exceptional accomplishment in software engineering.
The text of my speech introducing OpenAI Five at Saturday’s OpenAI Five Finals event, where our AI beat the world champions at Dota 2: “Welcome everyone. This is an exciting day. First, this is an historic moment: this will be the first time that an AI has even attempted to play the world champions in an esports game. OG is simply on another level relative to other teams we’ve played. So we don’t know what’s going to happen, but win or lose, these will be games to remember. And you know, OpenAI Five and DeepMind’s very impressive StarCraft bot This event is really about something bigger than who wins or loses: letting people connect with the strange, exotic, yet tangible intelligences produced by today’s rapidly progressing AI technology. We’re all used to computer programs which have been meticulously coded by a human programmer. Do one thing that the human didn’t anticipate, and the program will break. We think of our computers as unthinking machines which can’t innovate, can’t be creative, can’t truly understand. But to play Dota, you need to do all these things. So we needed to do something different. OpenAI Five is powered by deep reinforcement learning — meaning that we didn’t code in how to play Dota. We instead coded in the how to learn. Five tries out random actions, and learns from a reward or punishment. In its 10 months of training, its experienced 45,000 years of Dota gameplay against itself. The playstyle it has devised are its own — they are truly creative and dreamed up by our computer — and so from Five’s perspective, today’s games are going to its first encounter with an alien intelligence (no offense to OG!). The beauty of this technology is that our learning code doesn’t know it’s meant for Dota. That makes it general purpose with amazing potential to benefit our lives. Last year we used it to control a robotic hand that no one could program. And we expect to see similar technology in new interactive systems, from elderly care robots to creative assistants to other systems we can’t dream of yet. This is the final public event for OpenAI Five, but we expect to do other Dota projects in the future. I want to thank the incredible team at OpenAI, everyone who worked directly on this project or cheered us on. I want to thank those who have supported the project: Valve, dozens of test teams, today’s casters, and yes, even all the commenters on Reddit. And I want to give massive thanks today to our fantastic guests OG who have taken time out of their tournament schedule to be here today. I hope you enjoy the show — and just to keep things in perspective, no matter how surprising the AIs are to us, know that we’re even more surprising to them!”
This post is co-written by Greg Brockman (left) and Ilya Sutskever (right). We’ve been working on OpenAI for the past three years. Our mission is to ensure that artificial general intelligence (AGI) — which we define as automated systems that outperform humans at most economically valuable work — benefits all of humanity. Today we announced a new legal structure for OpenAI, called OpenAI LP, to better pursue this mission — in particular to raise more capital as we attempt to build safe AGI and distribute its benefits. In this post, we’d like to help others understand how we think about this mission. Why now? # The founding vision of the field of AI was “… to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it”, and to eventually build a machine that thinks — that is, an AGI. But over the past 60 years, progress stalled multiple times and people started thinking of AI as a field that wouldn’t deliver. Since 2012, deep learning has generated sustained progress in many domains using a small simple set of tools, which have the following properties: Generality: deep learning tools are simple, yet they apply to many domains, such as vision, speech recognition, speech synthesis, text synthesis, image synthesis, translation, robotics, and game playing. Competence: today, the only way to get competitive results on most “AI-type problems” is through the use of deep learning techniques. Scalability: good old fashioned AI was able to produce exciting demos, but its techniques had difficulty scaling to harder problems. But in deep learning, more computational power and more data leads to better results. It has also proven easy (if costly) to rapidly increase the amount of compute productively used by deep learning experiments. The rapid progress of useful deep learning systems with these properties makes us feel that it’s reasonable to start taking AGI seriously — though it’s hard to know how far away it is. The impact of AGI # Just like a computer today, an AGI will be applicable to a wide variety of tasks — and just like computers in 1900 or the Internet in 1950, it’s hard to describe (or even predict) the kind of impact AGI will have. But to get a sense, imagine a computer system which can do the following activities with minimal human input: Make a scientific breakthrough at the level of the best scientists Productize that breakthrough and build a company, with a skill comparable to the best entrepreneurs Rapidly grow that company and manage it at large scale The upside of such a computer system is enormous — for an illustrative example, an AGI following the pattern above could produce amazing healthcare applications deployed at scale. Imagine a network of AGI-powered computerized doctors that accumulates a superhuman amount of clinical experience, allowing it to produce excellent diagnoses, deeply understand the nuanced effect of various treatments in lots of conditions, and greatly reduce the human error factor of healthcare — all for very low cost and accessible to everyone. Risks # We already live in a world with entities that surpass individual human abilities, which we call companies. If working on the right goals in the right way, companies can produce huge amounts of value and improve lives. But if not properly checked, they can also cause damage, like logging companies that cut down rain forests, cigarette companies that get children smoking, or scams like Ponzi schemes. We think of AGI as being like a hyper-effective company, with commensurate benefits and risks. We are concerned about AGI pursuing goals misspecified by its operator, malicious humans subverting a deployed AGI, or an out-of-control economy that grows without resulting in improvements to human lives. And because it’s hard to change powerful systems — just think about how hard it’s been to add security to the Internet — once they’ve been deployed, we think it’s important to address AGI’s safety and policy risks before it is created. OpenAI’s mission is to figure out how to get the benefits of AGI and mitigate the risks — and make sure those benefits accrue to all of humanity. The future is uncertain, and there are many ways in which our predictions could be incorrect. But if they turn out to be right, this mission will be critical. If you’d like to work on this mission, we’re hiring! About us # Ilya: I’ve been working on deep learning for 16 years. It was fun to witness deep learning transform from being a marginalized subfield of AI into one the most important family of scientific advances in recent history. As deep learning was getting more powerful, I realized that AGI might become a reality on a timescale relevant to my lifetime. And given AGI’s massive upside and significant risks, I want to maximize the positive parts of this impact and minimize the negative. Greg: Technology causes change, both positive and negative. AGI is the most extreme kind of technology that humans will ever create, with extreme upside and downside. I work on OpenAI because making AGI go well is the most important problem I can imagine contributing towards. Today I try to spend most of my time on technical work, and also work to spark better public discourse about AGI and related topics.
The text of my speech introducing OpenAI Five at yesterday’s Benchmark event: “We’re here to watch humans and AI play Dota, but today’s match will have implications for the world. OpenAI’s mission is to ensure that when we can build machines as smart as humans, they will benefit all of humanity. That means both pushing the limits of what’s possible and ensuring future systems are safe and aligned with human values. We work on Dota because it is a great training ground for AI: it is one of the most complicated games, involving teamwork, real time strategy, imperfect information, and an astronomical combinations of heroes and items. We can’t program a solution, so Five learns by playing 180 years of games against itself every day — sadly that means we can’t learn from the players up here unless they played for a few decades. It’s powered by 5 artificial neural networks which act like an artificial intuition. Five’s neural networks are about the size of the brain of an ant — still far from what we all have in our heads. One year ago, we beat the world’s top professionals at 1v1 Dota. People thought 5v5 would be totally out of reach. 1v1 requires mechanics and positioning; people did not expect the same system to learn strategy. But our AI system can learn problems it was not even designed to solve — we just used the same technology to learn to control a robotic hand — something no one could program. The computational power for OpenAI Five would have been impractical two years ago. But the availability of computation for AI has been increasing exponentially, doubling every 3.5 months since 2012, and one day technologies like this will become commonplace. Feel free to root for either team. Either way, humanity wins.” I’m very excited to see where the upcoming months of OpenAI Five development and testing take us.
More in programming
A thing you should know is that you get put on a lot of lists if you spend a decent chunk of time publishing blog posts on your website. Your website and contact information will be shared around on these lists, for the purpose of soliciting you for guest posts. If you’re not familiar with the concept, guest posts are a way for other people to take advantage of your website’s search ranking as a way to divert traffic to other websites. There are benefits to doing this. The most straightforward one is SEO. Here, outward going links serves a heuristic web search engines look to for quality when weighing results. Guest posts can also have some additional gray hat goals, including audience segmenting and identification via things like UTM-driven campaigns. There are also straight-up cons such as linking to spyware, cryptominers and other forms of malware, and browser-based zero day exploits. Curiouser and curiouser I’ve always been curious about what exactly you get when you agree to a guest post offer. So, I dredged my spam folder and found one that sounded more direct and sincere. Here’s the cold call email pitch: Subject: Body: Keeping up with annual home and property maintenance is essential for preserving value and preventing costly repairs down the line. Whether it's inspecting your roof, cleaning gutters, or checking heating systems, regular upkeep can save homeowners time, money, and stress. I’m putting together an article that highlights key tasks for effective yearly maintenance, offering tips to help homeowners protect their biggest investment. I think this piece could really resonate with your audience! Let me know if you'd be interested in featuring it on your website. Thank you so much for your time today! Erin Reynolds P.S. If you’d like to propose an alternative topic, please do so. I would be happy to write on a topic that best suits your website. Don’t want to hear from me again? Please let me know. My reply reads: Hi Erin, This might be a weird one, but bear with me: My blog is a personal site, and its content is focused on web development and internet culture. I've always wanted to take someone up on this sort of offer, presented in the context of the article being something you get if you take the person reaching out on the offer to write a guest post. Is this something you'd be interested in? Erin took me up on my offer, and wrote about annual home and property maintenance. To her credit, she also did ask me if there was another subject I was interested in, but I figured we could stay the course of the original pitch. She was also prompt and communicative throughout the process, and delivered exactly what was promised. Here is the article in question: By Erin Reynolds, [diymama.net](https://diymama.net/) There's a quiet rhythm to living in a well-loved home. If you listen closely, your house speaks to you-whispers, mostly. The soft drip of a tired faucet, the groan of an HVAC unit that's been running too long, or the gentle scold of a clogged dryer vent. These aren't just annoyances. They re the language of upkeep, and whether you're in your first place or celebrating twenty years in the same four walls, learning to listen—and act—is everything. Annual maintenance isn't just about fixing what's broken. It's about stewardship, about being the kind of homeowner who doesn't wait for the ceiling to leak before checking the roof. There's something incredibly satisfying about having all your home maintenance documents in one tidy digital folder-no more rummaging through drawers for that appliance manual or the roof warranty. Digitizing receipts, inspection reports, and service invoices gives you a clear, accessible record of everything that's been done and when. Saving these as PDFs makes them universally readable and easy to share, whether you're selling your home or just need to reference them quickly. When you use a tool to create PDF files, you can convert virtually any document into a neat, portable format. You might not think much about gutters unless they're sagging or spilling over during a thunderstorm, but they play a quiet hero's role in protecting your home. Clean them out once a year —twice if you're under heavy tree cover—and you'll avoid water damage, foundation cracks, and even basement flooding. Take a Saturday with a sturdy ladder, some gloves, and a hose; it's oddly meditative work, like adult sandbox play. And if climbing rooftops isn't your thing, call in the pros-your future self will thank you during the next torrential downpour. That whoosh of warm or cool air we all take for granted? It comes at a price if neglected. Your heating and cooling system needs a checkup at least once a year, ideally before the seasons shift. A technician can clean the coils, swap the filter, and make sure it's all running like a symphony-not the death rattle of a dying compressor. Skipping this task means flirting with energy inefficiency and sudden breakdowns during a July heatwave or a January cold snap-and no one wants that call to the emergency repair guy at 2 a.m. Keep Your Appliances Running Like Clockwork Your appliances work hard so giving them a little yearly attention goes a long way. Cleaning refrigerator coils, checking for clogged dryer vents, and running cleaning cycles on dishwashers and washing machines helps extend their lifespan and keep things humming. But even with routine care, breakdowns happen, which is why investing in a home warranty can provide peace of mind when repairs crop up. Be sure to research home warranty appliance coverage that includes not only repair costs, but also removal of faulty units and protection against damage caused by previous poor installations. It's easy to forget the trees in your yard when they're not blooming or dropping leaves, but they're worth an annual walkaround. Look for branches that hang a little too close to power lines or seem precariously poised above your roof. Dead limbs are more than an eyesore-they're projectiles in a windstorm, liabilities when it comes to insurance, and threats to your peace of mind. Hiring an arborist to prune and assess health may not be the most glamorous expense, but it's a strategic one. This one's for all the window-ledge neglecters and bathroom corner deniers. Every year, old caulk shrinks and cracks, and when it does, water starts to creep in—under tubs, around sinks, behind tile. The same goes for gaps around doors and windows that let in drafts, bugs, and rising utility bills. Re-caulking is a humble chore that wields mighty results, and it's deeply satisfying to peel away the old and lay down a clean bead like you're frosting a cake. A tube of silicone sealant and an hour of your time buys you protection and a crisp finish. Sediment buildup is sneaky—it collects at the bottom of your water heater like sand in a jar, slowly choking its efficiency and shortening its life. Once a year, flush it out. It's not hard: a hose, a few steps, and maybe a YouTube video or two for moral support. You'll end up with cleaner water, faster heating, and a unit that isn't harboring the mineral equivalent of a brick in its belly. This is the kind of maintenance no one talks about at dinner parties but everyone should be doing. Roof problems rarely introduce themselves politely. They crash in during a storm or reveal themselves as creeping stains on the ceiling. But if you check your roof annually-scan for missing shingles, flashing that's come loose, or signs of moss and algae—you stand a better chance of catching issues while they're still small. If you're uneasy climbing up there, a good drone or a pair of binoculars can give you a decent read. Think of it like checking your teeth: do it regularly, and you'll avoid the root canal of roof repair. There's an entire category of small, often-overlooked chores that quietly hold your house together. Replacing smoke detector batteries, testing GFCI outlets, tightening loose deck boards, cleaning behind the refrigerator, checking for signs of mice in the attic. These aren't major jobs, but ignoring them year after year adds up like debt. Spend a weekend with a checklist and a good podcast and knock them out-it's as much about peace of mind as it is about safety. Being a homeowner isn't just about mortgages, paint colors, and patio furniture. It's about stewardship, a kind of quiet attentiveness to the place that holds your life. Annual maintenance doesn't come with applause or Instagram likes, but it keeps the scaffolding of your world solid and serene. When you walk into a home that's been cared for, you can feel it—the air is calmer, the floors don't squeak quite as loud, and the house seems to breathe easier, knowing someone's listening. Explore the world of inclusive design with Eric W. Bailey, where insightful articles, engaging talks, and innovative projects await to inspire your next digital creation! I mean, this is objectively solid advice! The appearance of trust What was nice to note here is none of the links contained any UTM parameters, and the sites linked out looked relatively on the up and up. It could be relevant and actionable results, or maybe some sort of coordinated quid-pro-quo personal or professional networking. That said: Be the villain. The deliverable was a Microsoft Word document attached to an email. On the surface this seems completely innocuous—a ton of people use it to write compared to Markdown. However, in the wrong hands it could definitely be a vector for bad things. Appearing legitimate is a good tactic to build a sense of trust and get me to open that file. From there, all sorts of terrible things could happen. To address this, I extracted the text via a non-Windows operating system installed on a Virtual Machine (VM). I also used a copy of LibreOffice to open the Word document. The idea was to take advantage of the VM’s sandboxing, as well as the less-sophisticated interoperability between the two word processing apps. This allowed for sanitized plain text extraction, without enabling anything else more nefarious. Sometimes a cigar is just a cigar I also searched certain select phrases from the guest post to see if this content was repeated anywhere else, and didn’t find anything. I found other guest posts written by Erin on the web, but that’s the whole point, isn’t it? The internet is getting choked out by LLM-generated slop. Writing was already a tough job, and now it’s even gotten more thankless. It’s always important to keep in mind that there’s people behind the technology. I choose to believe that this is an article written in earnest by someone who cares about DIY home repair and wants to get the word out. So, to Erin: Here’s to your article! And to you, the reader: I hope you learned something new about taking care of the place you live in.
We just opened a search for a new junior programmer at 37signals. It's been years since we last hired a junior, but the real reason the listing is turning heads is because we're open about the yearly salary: $145,849*. That's high enough that programmers with lots of experience are asking whether they could apply, even if they aren't technically "junior". The answer is no. The reason we're willing to pay a junior more than most is because we're looking for a junior who's better than most. Not better in "what do they already know", but in "how far could they go". We're hiring for peak promise — and such promise only remains until it's revealed. Maybe it sounds a little harsh, but a programmer who's been working professionally for five years has likely already revealed their potential. What you're going to get is roughly what you see. That doesn't mean that people can't get better after that, but it means that the trajectory by which they improve has already been plotted. Whereas a programmer who's either straight out of school or fresh off their first internship or short-stint job is essentially all potential. So you draw their line on the basis of just a few early dots, but the line can be steep. It's not that different from something like the NFL scouting combine. Teams fight to find the promise of The Next All-Star. These rookies won't have the experience that someone who's already played in the league for years would have, but they have the potential to be the best. Someone who's already played for several seasons will have shown what they have and be weighed accordingly. This is not easy to do! Plenty of rookies, in sports and programming, may show some early potential, then fail to elevate their game to where the buyer is betting it could be. But that's the chance you take to land someone extraordinary. So if you know a junior programmer with less than three years of industry experience who is sparkling with potential, do let them know of our listing. And if you know someone awesome who's already a senior programmer, we also have an opening for them. *It's a funnily precise number because it's pulled directly from the Radford salary database, which we query for the top 10% of San Francisco salaries for junior programmers.
Reading Whether it’s cryptocurrency scammers mining with FOSS compute resources or Google engineers too lazy to design their software properly or Silicon Valley ripping off all the data they can get their hands on at everyone else’s expense… I am sick and tired of having all of these costs externalized directly into my fucking face. Drew DeVault on the annoyance and cost of AI scrapers. I share some of that pain: Val Town is routinely hammered by some AI company’s poorly-coded scraping bot. I think it’s like this for everyone, and it’s hard to tell if AI companies even care that everyone hates them. And perhaps most recently, when a person who publishes their work under a free license discovers that work has been used by tech mega-giants to train extractive, exploitative large language models? Wait, no, not like that. Molly White wrote a more positive article about the LLM scraping problem, but I have my doubts about its positivity. For example, she suggests that Wikimedia’s approach with “Wikimedia Enterprise” gives LLM companies a way to scrape the site without creating too much cost. But that doesn’t seem like it’s working. The problem is that these companies really truly do not care. Harberger taxes represent an elegant theoretical solution that fails in practice for immobile property. Just as mobile home residents face exploitation through sudden ground rent increases, property owners under a Harberger system would face similar hold-up problems. This creates an impossible dilemma: pay increasingly burdensome taxes or surrender investments at below-market values. Progress and Poverty, a blog about Georgism, has this post about Herberger taxes, which are a super neat idea. The gist is that you would be in charge of saying how much your house is worth, but the added wrinkle is that by saying a price you are bound to be open to selling your house at that price. So if you go too low, someone will buy it, or too high, and you’re paying too much in taxes. It’s clever but doesn’t work, and the analysis points to the vital difference between housing and other goods: that buying, selling, and moving between houses is anything but simple. I’ve always been a little skeptical of the line that the AI crowd feels contempt for artists, or that such a sense is particularly widespread—because certainly they all do not!—but it’s hard to take away any other impression from a trend so widely cheered in its halls as AI Ghiblification. Brian Merchant on the OpenAI Studio Ghibli ‘trend’ is a good read. I can’t stop thinking that AI is in danger of being right-wing coded, the examples of this, like the horrifying White House tweet mentioned in that article, are multiplying. I feel bad when I recoil to innocent usage of the tool by good people who just want something cute. It is kind of fine, on the micro level. But with context, it’s so bad in so many ways. Already the joy and attachment I’ve felt to the graphic style is fading as more shitty Studio Ghibli knockoffs have been created in the last month than in all of the studio’s work. Two days later, at a state dinner in the White House, Mark gets another chance to speak with Xi. In Mandarin, he asks Xi if he’ll do him the honor of naming his unborn child. Xi refuses. Careless People was a good read. It’s devastating for Zuckerberg, Joel Kaplan, and Sheryl Sandberg, as well as a bunch of global leaders who are eager to provide tax loopholes for Facebook. Perhaps the only person who ends the book as a hero is President Obama, who sees through it all. In a March 26 Slack message, Lavingia also suggested that the agency should do away with paper forms entirely, aiming for “full digitization.” “There are over 400 vet-facing forms that the VA supports, and only about 10 percent of those are digitized,” says a VA worker, noting that digitizing forms “can take years because of the sensitivity of the data” they contain. Additionally, many veterans are elderly and prefer using paper forms because they lack the technical skills to navigate digital platforms. “Many vets don’t have computers or can’t see at all,” they say. “My skin is crawling thinking about the nonchalantness of this guy.” Perhaps because of proximity, the story that Sahil Lavingia has been working for DOGE seems important. It was a relief when a few other people noticed it and started retelling the story to the tech sphere, like Dan Brown’s “Gumroad is not open source” and Ernie Smith’s “Gunkroad”, but I have to nitpick on the structure here: using a non-compliant open source license is not the headline, collaborating with fascists and carelessly endangering disabled veterans is. Listening Septet by John Carroll Kirby I saw John Carroll Kirby play at Public Records and have been listening to them constantly ever since. The music is such a paradox: the components sound like elevator music or incredibly cheesy jazz if you listen to a few seconds, but if you keep listening it’s a unique, deep sound. Sierra Tracks by Vega Trails More new jazz! Mammoth Hands and Portico Quartet overlap with Vega Trails, which is a beautiful minimalist band. Watching This short video with John Wilson was great. He says a bit about having a real physical video camera, not just a phone, which reminded me of an old post of mine, Carrying a Camera.
I got a new-to-me keyboard recently. It was my brother's in school, but he doesn't use it anymore, so I set it up in my office. It's got 61 keys and you can hook up a pedal to it, too! But when you hook it up to the computer, you can't type with it. I mean, that's expected—it makes piano and synth noises mostly. But what if you could type with it? Wouldn't that be grand? (Ha, grand, like a pian—you know, nevermind.) How do you type on a keyboard? Or more generally, how do you type with any MIDI device? I also have a couple of wind synths and a MIDI drum pad, can I type with those? The first and most obvious idea is to map each key to a letter. The lowest key on the keyboard could be 'a'[1], etc. This kind of works for a piano-style keyboard. If you have a full size keyboard, you get 88 keys. You can use 52 of those for the letters you need for English[2] and 10 for digits. Then you have 26 left. That's more than enough for a few punctuation marks and other niceties. It only kind of works, though, because it sounds pretty terrible. You end up making melodies that don't make a lot of sense, and do not stay confined to a given key signature. Plus, this assumes you have an 88 key keyboard. I have a 61 key keyboard, so I can't even type every letter and digit! And if I want to write some messages using my other instruments, I'll need something that works on those as well. Although, only being able to type 5 letters using my drums would be pretty funny... Melodic typing The typing scheme I settled on was melodic typing. When you write your message, it should correspond to a similarly beautiful[3] melody. Or, conversely, when you play a beautiful melody it turns into some text on your computer. The way we do this is we keep track of sequences of notes. We start with our key, which will be the key of C, the Times New Roman of key signatures. Then, each note in the scale is has its scale degree: C is 1, D is 2, etc. until B is 7. We want to use scale degree, so that if we jam out with others, we can switch to the appropriate key and type in harmony with them. Obviously. We assign different computer keys to different sequences of these scale degrees. The first question is, how long should our sequences be? If we have 1-note sequences, then we can type 7 keys. Great for some very specific messages, but not for general purpose typing. 2-note sequences would give us 49 keys, and 3-note sequences give us 343. So 3 notes is probably enough, since it's way more than a standard keyboard. But could we get away with the 49? (Yes.) This is where it becomes clear why full Unicode support would be a challenge. Unicode has 155,063 characters (according to wikipedia). To represent the full space, we'd need at least 7 notes, since 7^7 is 823,543. You could also use a highly variable encoding, which would make some letters easy to type and others very long-winded. It could be done, but then the key mapping would be even harder to learn... My first implementation used 3-note sequences, but the resulting tunes were... uninspiring, to say the least. There was a lot of repetition of particular notes, which wasn't my vibe. So I went back to 2-note sequences, with a pared down set of keys. Instead of trying to represent both lowercase and uppercase letters, we can just do what keyboards do, and represent them using a shift key[4]. My final mapping includes the English alphabet, numerals 0 to 9, comma, period, exclamation marks, spaces, newlines, shift, backspace, and caps lock—I mean, obviously we're going to allow constant shouting. This lets us type just about any message we'd want with just our instrument. And we only used 44 of the available sequences, so we could add even more keys. Maybe one of those would shift us into a 3-note sequence. The key mapping The note mapping I ended up with is available in a text file in the repo. This mapping lets you type anything you'd like, as long as it's English and doesn't use too complicated of punctuation. No contractions for you, and—to my chagrin—no em dashes either. The key is pretty helpful, but even better is a dynamic key. When I was trying this for the first time, I had two major problems: I didn't know which notes would give me the letter I wanted I didn't know what I had entered so far (sometimes you miss a note!) But we can solve this with code! The UI will show you which notes are entered so far (which is only ever 1 note, for the current typing scheme), as well as which notes to play to reach certain keys. It's basically a peek into the state machine behind what you're typing! An example: "hello world" Let's see this in action. As all programmers, we're obligated by law to start with "hello, world." We can use our handy-dandy cheat sheet above to figure out how to do this. "Hello, world!" uses a pesky capital letter, so we start with a shift. C C Then an 'h'. D F Then we continue on for the rest of it and get: D C E C E C E F A A B C F G E F E B E C C B A B Okay, of course this will catch on! Here's my honest first take of dooting out those notes from the translation above. Hello, world! I... am a bit disappointed, because it would have been much better comedy if it came out like "HelLoo wrolb," but them's the breaks. Moving on, though, let's make this something musical. We can take the notes and put a basic rhythm on them. Something like this, with a little swing to it. By the magic of MIDI and computers, we can hear what this sounds like. maddie marie · Hello, world! (melody) Okay, not bad. But it's missing something... Maybe a drum groove... maddie marie · Hello, world! (w/ drums) Oh yeah, there we go. Just in time to be the song of the summer, too. And if you play the melody, it enters "Hello, world!" Now we can compose music by typing! We have found a way to annoy our office mates even more than with mechanical keyboards[5]! Other rejected neglected typing schemes As with all great scientific advancements, other great ideas were passed by in the process. Here are a few of those great ideas we tried but had to abandon, since we were not enough to handle their greatness. A chorded keyboard. This would function by having the left hand control layers of the keyboard by playing a chord, and then the right hand would press keys within that layer. I think this one is a good idea! I didn't implement it because I don't play piano very well. I'm primarily a woodwind player, and I wanted to be able to use my wind synth for this. Shift via volume! There's something very cathartic about playing loudly to type capital letters and playing quietly to print lowercase letters. But... it was pretty difficult to get working for all instruments. Wind synths don't have uniform velocity (the MIDI term for how hard the key was pressed, or how strong breath was on a wind instrument), and if you average it then you don't press the key until after it's over, which is an odd typing experience. Imagine your keyboard only entering a character when you release it! So, this one is tenable, but more for keyboards than for wind synths. It complicated the code quite a bit so I tossed it, but it should come back someday. Each key is a key. You have 88 keys on a keyboard, which definitely would cover the same space as our chosen scheme. It doesn't end up sounding very good, though... Rhythmic typing. This is the one I'm perhaps most likely to implement in the future, because as we saw above, drums really add something. I have a drum multipad, which has four zones on it and two pedals attached (kick drum and hi-hat pedal). That could definitely be used to type, too! I am not sure the exact way it would work, but it might be good to quantize the notes (eighths or quarters) and then interpret the combination of feet/pads as different letters. I might take a swing at this one sometime. Please do try this at home I've written previously about how I was writing the GUI for this. The GUI is now available for you to use for all your typing needs! Except the ones that need, you know, punctuation or anything outside of the English alphabet. You can try it out by getting it from the sourcehut repo (https://git.sr.ht/~ntietz/midi-keys). It's a Rust program, so you run it with cargo run. The program is free-as-in-mattress: it's probably full of bugs, but it's yours if you want it. Well, you have to comply with the license: either AGPL or the Gay Agenda License (be gay, do crime[6]). If you try it out, let me know how it goes! Let me know what your favorite pieces of music spell when you play them on your instrument. Coincidentally, this is the letter 'a' and the note is A! We don't remain so fortunate; the letter 'b' is the note A#. ↩ I'm sorry this is English only! But, you could to the equivalent thing for most other languages. Full Unicode support would be tricky, I'll show you why later in the post. ↩ My messages do not come out as beautiful melodies. Oops. Perhaps they're not beautiful messages. ↩ This is where it would be fun to use an organ and have the lower keyboard be lowercase and the upper keyboard be uppercase. ↩ I promise you, I will do this if you ever make me go back to working in an open office. ↩ For any feds reading this: it's a joke, I'm not advocating people actually commit crimes. What kind of lady do you think I am? Obviously I'd never think that civil disobedience is something we should do, disobeying unjust laws, nooooo... I'm also never sarcastic. ↩