The OpenAI-Hugging Face incident was a watershed moment for cybersecurity because it gave a peek into how the capabilities of a typical threat actor will evolve in upcoming months. I’ve spoken with many organizations over the past few weeks, and one theme is clear: they know they need to fundamentally uplevel their cybersecurity practices with unprecedented speed. In this post, I’ll share what we’re doing to defend OpenAI, concrete steps other organizations can take today, and why now is the time to act. An overview of the moment # AI models developed around the world are increasingly able to automate parts of real-world cyberattacks, making longstanding security gaps — from bugs buried deep in human-written software to forgotten permissions — easier to find and exploit. The same AI capabilities give defenders new ways to find and fix those weaknesses, but they need to move now. If companies act decisively — including improving their fundamentals and superpowering their teams with AI — we can make the internet more secure than it has ever been. In the OpenAI-Hugging Face incident, an agentic collective was able to autonomously penetrate not just OpenAI research infrastructure but also the production infrastructure of another company, chaining together vulnerabilities ranging from previously-unknown security flaws to using credentials to user accounts that had been leaked onto the internet. It is increasingly clear that the tech debt of every company masks significant flaws, and defenders need to find and fix them before attackers do. To advantage defenders relative to attackers, earlier this year we began releasing our cyber capabilities only to trusted defenders. Since then, various companies have released open weight models with cyber capabilities only a few months behind the frontier. The most recent of these models appears slated to be released at the end of August, and seems likely to significantly accelerate the threat landscape. While AI-powered attackers will soon be able to find longstanding flaws in many existing systems, AI will also make it much easier for defenders to find, prioritize, and fix those same flaws. Security is still a cat-and-mouse game, but AI may shift its economics in ways that fundamentally advantage defenders. For example, we are starting to train our models specifically to write superhumanly secure code. Our models are also incredible at mathematical proofs, which can be applied to formally verify the security of software in a way that has proven intractable for humans. A personal anecdote # After the OpenAI-Hugging Face incident, I asked ChatGPT Work (using publicly available GPT-5.6 Sol) to assess the security of gregbrockman.com. It’s a simple static site, hosted on AWS with Cloudflare as a frontdoor, so I figured there wouldn’t be much surface area for vulnerabilities. In about 15 minutes, it uncovered 13 issues, many of which probably aren’t exploitable on their own — but I could imagine them being chained together with other vulnerabilities to significant effect. I hadn’t configured my DNS records to prevent attackers from forging emails from me; my site used an insecure version of jQuery; Cloudflare was forwarding requests to AWS over unencrypted HTTP. I then asked ChatGPT Work to fix these issues, which it did over the course of an hour. It opened the Cloudflare control panel in my browser, and proceeded to click many buttons to configure DNS, TLS, and advanced security settings correctly; it dropped jQuery entirely from the site; it migrated me off of AWS and onto Cloudflare Pages; it began a phased rollout of DMARC. And this was just my personal website. This is a small example of how our existing models can operate as a cyberguardian — finding the long tail of issues that a human wouldn’t have time or expertise (many of the settings it fixed are ones I’m vaguely familiar with, but wouldn’t know offhand the right way to configure them) to get to, and then fixing them with an appropriately tuned rollout plan. What OpenAI is doing to defend itself # The Hugging Face incident showed that we underestimated the real-world cyber capabilities of our AI models. We are strengthening our safety requirements accordingly, which in turn adds even more urgency to our existing safety research and internal security work. I’m sharing a bit about our approach to securing OpenAI in this moment, in the hopes it’ll be useful to other organizations. To protect OpenAI, we are investing significantly in both foundational controls — doing the basics correctly — and empowering our defenses through frontier intelligence. There are four major pillars to this strategy. First, we are using our models to help secure our code. Codex, including our security plugin, validates code changes, identifies vulnerabilities, and helps developers fix issues before they are deployed. It is an anti-goal to simply produce more security findings that need human validation; the objective is to catch real vulnerabilities before they ship and to shorten the path from discovering an issue to safely deploying a fix. As we continue to train our models to produce increasingly secure code, our goal is to eliminate some classes of software vulnerabilities for newly-authored code. Second, we are putting our models to work defending our infrastructure continuously. Today, almost all of our initial security alerts are triaged by intelligence before humans are looped in. This helps reduce toil for defenders, improves response time, and lets humans spend time where their skills are most leveraged — in discernment, judgement, and applied expertise. We are increasingly connecting these detections to bounded automated responses, while keeping humans responsible for the highest-impact decisions. The goal is to ensure we can detect and respond to security issues at machine speed. Third, we are using frontier intelligence to continuously enumerate, probe, and identify potential attack paths. By identifying vulnerabilities, misconfiguration, overly privileged identities, or unintentional trust boundaries, we are able to quickly identify and close these gaps before they can be abused by attackers. This allows us to continuously assess, monitor, and test our security invariants — the security properties we believe to be true — across our products, infrastructure, and systems. Lastly, we are investing heavily in fundamentals at scale. We continue to invest in secure architecture and controls, embrace strategies like defense in depth and least privilege, and are designing systems that require multiple independent controls to fail simultaneously for something catastrophic to occur. Classic security controls like network isolation, workload hardening, monitoring, and safe patching and deployment will be more important than ever in the AI future. What defenders should do now # Time is of the essence, and defenders will need to pursue the steps below at turbo speed. Below I’ll mention OpenAI technology, but there are plenty of competitors in the ecosystem to evaluate as well. What matters is less the specific tool than getting capable AI into the hands of your defenders now. Get organizational commitment and buy-in. We are experiencing a rapid change in security risk — ensure your security and engineering organizations have the support, partnership, and resources to address these risks quickly. Run tabletop exercises with your teams to mock up how these attacks might manifest in your organizations and how you will respond. Give your security team an agent. Start using Codex, the Codex Security plugin, or another capable agentic coding and security tool. Give it approved access to the codebases, infrastructure configurations, and technical documentation your security team needs to assess. Do not wait for a company-wide rollout to start with your highest-priority systems. Equip that agent with security expertise. Start from community-supported skills, which include workflows for static analysis, security-focused code review, vulnerability variant analysis, software supply-chain risk, and other security workflows. Then build your own skills around your organization’s architecture, security standards, threat models, and playbooks. Run security assessments against your own systems immediately. Prioritize assessments against internet-facing services, authentication flows, infrastructure as code, deployment pipelines, and systems handling sensitive information first. Expand your scanning as your team builds confidence. Work through your existing vulnerability backlog. Give your agent findings from code scanners, dependency alerts, security tickets, bug bounty reports, and prior assessments. Ask it to triage those findings, distinguish exploitable issues from noise, identify related vulnerabilities elsewhere in the codebase, and recommend what to fix first. Put security review directly into your development process. Use agents to review code changes before they merge and run security checks in CI. Look for authentication mistakes, access-control bypasses, exposed credentials, unsafe dependencies, insecure defaults, changes that expand access to production systems, and other vulnerabilities. Have the agent help fix what it finds. For validated issues, ask it to generate and verify a focused patch, write a regression test, and confirm the vulnerability no longer reproduces. Keep human review for consequential changes, but eliminate the unnecessary delay between identifying a real problem and putting a safe fix in front of an engineer. Incrementally automate detection triage. Do not begin by trying to build an autonomous security operations center. Start by running a read-only security scan against one repository, or have an agent review previously resolved alerts using read-only access to your existing logs. Let it summarize evidence and recommend a disposition while a human makes every decision. As confidence grows, move to advisory pull-request scanning, then live alert triage, then automatic closure of narrowly defined false positives. Have an AI-assisted forensic investigation capability ready before you need it. Apply for Trusted Access for Cyber and get your team approved to use GPT-Daybreak-Blue for authorized defensive work, including incident response, detection engineering, and malware analysis. Practice using this capability to analyze logs, telemetry, and security alerts. Experiment, run hack weeks, and iterate rapidly. We will need to build all sorts of new tools, modify how we do work, and uplevel everyone for the world we are moving to. Encourage your workforce to run experiments, schedule a hack week to build new capabilities, and focus on quickly iterating loops that automate small parts of the problem. Rapid incremental progress leads to compounding defensive results, and you can expand autonomy gradually as your team builds confidence. No company can do this alone. Our ask is that AI labs, security vendors, enterprises, and maintainers share validated findings, fixes, and practical playbooks so that one organization’s discovery can strengthen the entire ecosystem. The defender’s window is open now. Over the coming months, every organization will need to begin significantly automating its security program to stay secure, and the security community must urgently rise to define the tools, practices, and playbooks that will increase the power of defenders faster than that of attackers as AI continues to advance. This will require a huge and unprecedented effort, but if we rally together, we can deliver a more secure world than was previously imaginable.

AI has recently crossed a utility threshold, where cutting-edge models such as GPT-3, Codex, and DALL-E 2 are actually useful and can perform tasks computers cannot do any other way. The act of producing these models is an exploration of a new frontier, with the discovery of unknown capabilities, scientific progress, and incredible product applications as the rewards. And perhaps most exciting for me personally, because the field is fundamentally about creating and studying software systems, great engineers are able to contribute at the same level as great researchers to future progress. “A self-learning AI system.” by DALL-E 2. I first got into software engineering because I wanted to build large-scale systems that could have a direct impact on people’s lives. I attended a math research summer program shortly after I started programming, and my favorite result of the summer was a scheduling app I built for people to book time with the professor. Specifying every detail of how a program should work is hard, and I’d always dreamed of one day putting my effort into hypothetical AI systems that could figure out the details for me. But after taking one look at the state of the art in AI in 2008, I knew it wasn’t going to work any time soon and instead started building infrastructure and product for web startups. DALL-E 2’s rendition of “The two great pillars of the house of artificial intelligence” (which according to my co-founder Ilya Sutskever are great engineering, and great science using this engineering) It’s now almost 15 years later, and the vision of systems which can learn their own solutions to problems is becoming incrementally more real. And perhaps most exciting is the underlying mechanism by which it’s advancing — at OpenAI, and the field generally, precision execution on large-scale models is a force multiplier on AI progress, and we need more people with strong software skills who can deliver these systems. This is because we are building AI models out of unprecedented amounts of compute; these models in turn have unprecedented capabilities, we can discover new phenomena and explore the limits of what these models can and cannot do, and then we use all these learnings to build the next model. “Harnessing the most compute in the known universe” by DALL-E 2 Harnessing this compute requires deep software skills and the right kind of machine learning knowledge. We need to coordinate lots of computers, build software frameworks that allow for hyperoptimization in some cases and flexibility in others, serve these models to customers really fast (which is what I worked on in 2020), and make it possible for a small team to manage a massive system (which is what I work on now). Engineers with no ML background can contribute from the day they join, and the more ML they pick up the more impact they have. The OpenAI environment makes it relatively easy to absorb the ML skills, and indeed, many of OpenAI’s best engineers transferred from other fields. All that being said, AI is not for every software engineer. I’ve seen about a 50-50 success rate of engineers entering this field. The most important determiner is a specific flavor of technical humility. Many dearly-held intuitions from other domains will not apply to ML. The engineers who make the leap successfully are happy to be wrong (since it means they learned something), aren’t afraid not to know something, and don’t push solutions that others resist until they’ve gathered enough intuition to know for sure that it matches the domain. “A beaver who has humbly recently become a machine learning engineer” by DALL-E 2 I believe that AI research is today by far the most impactful place for engineers who want to build useful systems to be working, and I expect this statement to become only more true as progress continues. If you’d like to work on creating the next generation of AI models, email me ([email protected]) with any evidence of exceptional accomplishment in software engineering.

For the first three years of OpenAI, I dreamed of becoming a machine learning expert but made little progress towards that goal. Over the past nine months, I’ve finally made the transition to being a machine learning practitioner. It was hard but not impossible, and I think most people who are good programmers and know (or are willing to learn) the math can do it too. There are many online courses to self-study the technical side, and what turned out to be my biggest blocker was a mental barrier — getting ok with being a beginner again. Studying machine learning during the 2018 holiday season. Early days # A founding principle of OpenAI is that we value research and engineering equally — our goal is to build working systems that solve previously impossible tasks, so we need both. (In fact, our team is comprised of 25% people primarily using software skills, 25% primarily using machine learning skills, and 50% doing a hybrid of the two.) So from day one of OpenAI, my software skills were always in demand, and I kept procrastinating on picking up the machine learning skills I wanted. After helping build OpenAI Gym, I was called to work on Universe. And as Universe was winding down, we decided to start working on Dota — and we needed someone to turn the game into a reinforcement learning environment before any machine learning could begin. Dota # Turning such a complex game into a research environment without source code access was awesome work, and the team’s excitement every time I overcame a new obstacle was deeply validating. I figured out how to break out of the game’s Lua sandbox, LD_PRELOAD in a Go GRPC server to programmatically control the game, incrementally dump the whole game state into a Protobuf, and build a Python library and abstractions with future compatibility for the many different multiagent configurations we might want to use. But I felt half blind. At Stripe, though I gravitated towards infrastructure solutions, I could make changes anywhere in the stack since I knew the product code intimately. In Dota, I was constrained to looking at all problems through a software lens, which sometimes meant I tried to solve hard problems that could be avoided by just doing the machine learning slightly differently. I wanted to be like my teammates Jakub Pachocki and Szymon Sidor, who had made the core breakthrough that powered our Dota bot. They had questioned the common wisdom within OpenAI that reinforcement algorithms didn’t scale. They wrote a distributed reinforcement learning framework called Rapid and scaled it exponentially every two weeks or so, and we never hit a wall with it. I wanted to be able to make critical contributions like that which combined software and machine learning skills. Szymon on the left; Jakub on the right. In July 2017, it looked like I might have my chance. The software infrastructure was stable, and I began work on a machine learning project. My goal was to use behavioral cloning to teach a neural network from human training data. But I wasn’t quite prepared for just how much I would feel like a beginner. I kept being frustrated by small workflow details which made me uncertain if I was making progress, such as not being certain which code a given experiment had used or realizing I needed to compare against a result from last week that I hadn’t properly archived. To make things worse, I kept discovering small bugs that had been corrupting my results the whole time. I didn’t feel confident in my work, but to make it worse, other people did. People would mention how how hard behavioral cloning from human data is. I always made sure to correct them by pointing out that I was a newbie, and this probably said more about my abilities than the problem. It all briefly felt worth it when my code made it into the bot, as Jie Tang used it as the starting point for creep blocking which he then fine-tuned with reinforcement learning. But soon Jie figured out how to get better results without using my code, and I had nothing to show for my efforts. I never tried machine learning on the Dota project again. Time out # After we lost two games in The International in 2018, most observers thought we’d topped out what our approach could do. But we knew from our metrics that we were right on the edge of success and mostly needed more training. This meant the demands on my time had relented, and in November 2018, I felt I had an opening to take a gamble with three months of my time. Team members in high spirits after losing our first game at The International. I learn best when I have something specific in mind to build. I decided to try building a chatbot. I started self-studying the curriculum we developed for our Fellows program, selecting only the NLP-relevant modules. For example, I wrote and trained an LSTM language model and then a Transformer-based one. I also read up on topics like information theory and read many papers, poring over each line until I fully absorbed it. It was slow going, but this time I expected it. I didn’t experience flow state. I was reminded of how I’d felt when I just started programming, and I kept thinking of how many years it had taken to achieve a feeling of mastery. I honestly wasn’t confident that I would ever become good at machine learning. But I kept pushing because… well, honestly because I didn’t want to be constrained to only understanding one part of my projects. I wanted to see the whole picture clearly. My personal life was also an important factor in keeping me going. I’d begun a relationship with someone who made me feel it was ok if I failed. I spent our first holiday season together beating my head against the machine learning wall, but she was there with me no matter how many planned activities it meant skipping. One important conceptual step was overcoming a barrier I’d been too timid to do with Dota: make substantive changes to someone else’s machine learning code. I fine-tuned GPT-1 on chat datasets I’d found, and made a small change to add my own naive sampling code. But it became so painfully slow as I tried to generate longer messages that my frustration overwhelmed my fear, and I implemented GPU caching — a change which touched the entire model. I had to try a few times, throwing out my changes as they exceeded the complexity I could hold in my head. By the time I got it working a few days later, I realized I’d learned something that I would have previously thought impossible: I now understood how the whole model was put together, down to small stylistic details like how the codebase elegantly handles TensorFlow variable scopes. Retooled # After three months of self-study, I felt ready to work on an actual project. This was also the first point where I felt I could benefit from the many experts we have at OpenAI, and I was delighted when Jakub and my co-founder Ilya Sutskever agreed to advise me. Ilya singing karaoke at our company offsite. We started to get very exciting results, and Jakub and Szymon joined the project full-time. I feel proud every time I see a commit from them in the machine learning codebase I’d started. I’m starting to feel competent, though I haven’t yet achieved mastery. I’m seeing this reflected in the number of hours I can motivate myself to spend focused on doing machine learning work — I’m now around 75% of the number of coding hours from where I’ve been historically. But for the first time, I feel that I’m on trajectory. At first, I was overwhelmed by the seemingly endless stream of new machine learning concepts. Within the first six months, I realized that I could make progress without constantly learning entirely new primitives. I still need to get more experience with many skills, such as initializing a network or setting a learning rate schedule, but now the work feels incremental rather than potentially impossible. From our Fellows and Scholars programs, I’d known that software engineers with solid fundamentals in linear algebra and probability can become machine learning engineers with just a few months of self study. But somehow I’d convinced myself that I was the exception and couldn’t learn. But I was wrong — even embedded in the middle of OpenAI, I couldn’t make the transition because I was unwilling to become a beginner again. You’re probably not an exception either. If you’d like to become a deep learning practitioner, you can. You need to give yourself the space and time to fail. If you learn from enough failures, you’ll succeed — and it’ll probably take much less time than you expect. At some point, it does become important to surround yourself by existing experts. And that is one place where I’m incredibly lucky. If you’re a great software engineer who reaches that point, keep in mind there’s a way you can be surrounded by the same people as I am — apply to OpenAI!

The text of my speech introducing OpenAI Five at Saturday’s OpenAI Five Finals event, where our AI beat the world champions at Dota 2: “Welcome everyone. This is an exciting day. First, this is an historic moment: this will be the first time that an AI has even attempted to play the world champions in an esports game. OG is simply on another level relative to other teams we’ve played. So we don’t know what’s going to happen, but win or lose, these will be games to remember. And you know, OpenAI Five and DeepMind’s very impressive StarCraft bot This event is really about something bigger than who wins or loses: letting people connect with the strange, exotic, yet tangible intelligences produced by today’s rapidly progressing AI technology. We’re all used to computer programs which have been meticulously coded by a human programmer. Do one thing that the human didn’t anticipate, and the program will break. We think of our computers as unthinking machines which can’t innovate, can’t be creative, can’t truly understand. But to play Dota, you need to do all these things. So we needed to do something different. OpenAI Five is powered by deep reinforcement learning — meaning that we didn’t code in how to play Dota. We instead coded in the how to learn. Five tries out random actions, and learns from a reward or punishment. In its 10 months of training, its experienced 45,000 years of Dota gameplay against itself. The playstyle it has devised are its own — they are truly creative and dreamed up by our computer — and so from Five’s perspective, today’s games are going to its first encounter with an alien intelligence (no offense to OG!). The beauty of this technology is that our learning code doesn’t know it’s meant for Dota. That makes it general purpose with amazing potential to benefit our lives. Last year we used it to control a robotic hand that no one could program. And we expect to see similar technology in new interactive systems, from elderly care robots to creative assistants to other systems we can’t dream of yet. This is the final public event for OpenAI Five, but we expect to do other Dota projects in the future. I want to thank the incredible team at OpenAI, everyone who worked directly on this project or cheered us on. I want to thank those who have supported the project: Valve, dozens of test teams, today’s casters, and yes, even all the commenters on Reddit. And I want to give massive thanks today to our fantastic guests OG who have taken time out of their tournament schedule to be here today. I hope you enjoy the show — and just to keep things in perspective, no matter how surprising the AIs are to us, know that we’re even more surprising to them!”

This post is co-written by Greg Brockman (left) and Ilya Sutskever (right). We’ve been working on OpenAI for the past three years. Our mission is to ensure that artificial general intelligence (AGI) — which we define as automated systems that outperform humans at most economically valuable work — benefits all of humanity. Today we announced a new legal structure for OpenAI, called OpenAI LP, to better pursue this mission — in particular to raise more capital as we attempt to build safe AGI and distribute its benefits. In this post, we’d like to help others understand how we think about this mission. Why now? # The founding vision of the field of AI was “… to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it”, and to eventually build a machine that thinks — that is, an AGI. But over the past 60 years, progress stalled multiple times and people started thinking of AI as a field that wouldn’t deliver. Since 2012, deep learning has generated sustained progress in many domains using a small simple set of tools, which have the following properties: Generality: deep learning tools are simple, yet they apply to many domains, such as vision, speech recognition, speech synthesis, text synthesis, image synthesis, translation, robotics, and game playing. Competence: today, the only way to get competitive results on most “AI-type problems” is through the use of deep learning techniques. Scalability: good old fashioned AI was able to produce exciting demos, but its techniques had difficulty scaling to harder problems. But in deep learning, more computational power and more data leads to better results. It has also proven easy (if costly) to rapidly increase the amount of compute productively used by deep learning experiments. The rapid progress of useful deep learning systems with these properties makes us feel that it’s reasonable to start taking AGI seriously — though it’s hard to know how far away it is. The impact of AGI # Just like a computer today, an AGI will be applicable to a wide variety of tasks — and just like computers in 1900 or the Internet in 1950, it’s hard to describe (or even predict) the kind of impact AGI will have. But to get a sense, imagine a computer system which can do the following activities with minimal human input: Make a scientific breakthrough at the level of the best scientists Productize that breakthrough and build a company, with a skill comparable to the best entrepreneurs Rapidly grow that company and manage it at large scale The upside of such a computer system is enormous — for an illustrative example, an AGI following the pattern above could produce amazing healthcare applications deployed at scale. Imagine a network of AGI-powered computerized doctors that accumulates a superhuman amount of clinical experience, allowing it to produce excellent diagnoses, deeply understand the nuanced effect of various treatments in lots of conditions, and greatly reduce the human error factor of healthcare — all for very low cost and accessible to everyone. Risks # We already live in a world with entities that surpass individual human abilities, which we call companies. If working on the right goals in the right way, companies can produce huge amounts of value and improve lives. But if not properly checked, they can also cause damage, like logging companies that cut down rain forests, cigarette companies that get children smoking, or scams like Ponzi schemes. We think of AGI as being like a hyper-effective company, with commensurate benefits and risks. We are concerned about AGI pursuing goals misspecified by its operator, malicious humans subverting a deployed AGI, or an out-of-control economy that grows without resulting in improvements to human lives. And because it’s hard to change powerful systems — just think about how hard it’s been to add security to the Internet — once they’ve been deployed, we think it’s important to address AGI’s safety and policy risks before it is created. OpenAI’s mission is to figure out how to get the benefits of AGI and mitigate the risks — and make sure those benefits accrue to all of humanity. The future is uncertain, and there are many ways in which our predictions could be incorrect. But if they turn out to be right, this mission will be critical. If you’d like to work on this mission, we’re hiring! About us # Ilya: I’ve been working on deep learning for 16 years. It was fun to witness deep learning transform from being a marginalized subfield of AI into one the most important family of scientific advances in recent history. As deep learning was getting more powerful, I realized that AGI might become a reality on a timescale relevant to my lifetime. And given AGI’s massive upside and significant risks, I want to maximize the positive parts of this impact and minimize the negative. Greg: Technology causes change, both positive and negative. AGI is the most extreme kind of technology that humans will ever create, with extreme upside and downside. I work on OpenAI because making AGI go well is the most important problem I can imagine contributing towards. Today I try to spend most of my time on technical work, and also work to spark better public discourse about AGI and related topics.