Full Width [alt+shift+f] Shortcuts [alt+shift+k]
Sign Up [alt+shift+s] Log In [alt+shift+l]
1
This is an adaptation of an internal doc I wrote for Anthropic. I’ve been noticing recently that often, a big blocker to teams staying effective as they grow is trust. “Alice doesn’t trust Bob” makes Alice sound like the bad guy, but it’s often completely appropriate for people not to trust each other in some areas: One might have an active reason to expect someone to be bad at something. For example, recently I didn’t fully trust two of my managers to set their teams’ roadmaps… because they’d joined about a week ago and had barely gotten their laptops working. (Two months later, they’re doing great!) One might just not have data. For example, I haven’t seen most of my direct reports deal with an underperforming team member yet, and this is a common blind spot for many managers, so I shouldn’t assume that they will reliably be effective at this without support. In general, if Alice is Bob’s manager and is an authority on, say, prioritizing research directions, Bob is probably actively trying to build a good mental “Alice simulator” so that he can prioritize autonomously without checking in all the time. But his simulator might not be good yet, or Alice might not have verified that it’s good enough. Trust comes from common knowledge of shared mental models, and that takes investment from both sides to build. If low trust is sometimes appropriate, what’s the problem? It’s that trust is what lets collaboration scale. If I have a colleague I don’t trust to (say) make good software design decisions, I’ll have to review their designs much more carefully and ask them to make more thorough plans in advance. If I have a report that I don’t fully trust to handle underperforming team members, I’ll have to manage them more granularly, digging into the details to understand what’s going on and forming my own views about what should happen, and checking on the situation repeatedly to make sure it’s heading in the right direction. That’s a lot more work both for me, but also for...
13th Jul 2024

Stay updated

Get a weekly newsletter with the top 5 articles worth reading every week.

More from benkuhn.net

You should try contra dancing

A popular icebreaker in San Francisco these days is “How would you spend your life if AGI meant nobody needed to work?” For me, I think a surprisingly big part of the answer is a dorky-sounding kind of folk dance called contra dancing. I started trying to answer that question by thinking: what are the things I do atelically—because I enjoy them for their own sake, not in pursuit of some longer-term goal? For me, a lot of that has to do with things that are (1) physically joyful, and (2) help me feel connected to other people. My pitch for contra dancing is that among physically joyful and connecting activities, it’s one of the ones where it’s easiest to get to the point where it’s fun, and therefore one of the best to start out with. Many other forms of dance require weeks of lessons before you’re even encouraged to do it socially at all. By contrast, the format of contra dancing means that you can go to a half-hour beginner lesson before your first dance, and there’s a good chance you will be having a lot of fun by the end of the evening. What’s more, the fact that it’s so easy to start having fun shapes its community and priorities, such that it has among the best vibes of any activity I do. I first encountered contra in middle school. My mom ran an intentional community, and we had a housemate move in named Bree. Bree was obsessed with this thing called “contra dancing,” and every week she would try to convince me and our other housemates to come with her. Which I absolutely refused. I was an extremely uncool middle schooler, and that meant I was terrified of doing anything that even whiffed of uncoolness, lest I give my classmates another reason to judge me. I was also deeply awkward; I’d never even been to a school dance and barely talked to girls. But Bree was persistent, and one of our other housemates got hooked, and eventually I was intrigued. By then, though, I was dug in as a contra hater, and I was scared that if I admitted I had changed my mind, my housemates would make fun of me. So I kept refusing. Finally, my housemate Thomas, who was even more of a skeptic than I was, told Bree that he would only go if I went, presumably so that she would stop bugging him and bug me instead. Aha—my golden opportunity! I could pass off my change of heart as a prank on Thomas. “Okay, sure! It’s a deal!” Bree and I showed up at the Concord Scout House one Thursday evening and jumped into the next dance. It was an unusually complex square dance, and I got confused—in fact, so confused that my entire square ended up giving up halfway through.1 I was mortified. Not only was I doing a deeply uncool activity, I was the least cool person there, ruining it for all the slightly less uncool people with my incompetence and lameness. Except that the other dancers didn’t seem to see it that way. Instead of resenting me or making fun of me, they said things like “Gosh, that one was really tricky!” and “Sorry, that’s really not what it’s usually like!” and “Please don’t give up, try a few more dances! Want to dance with me?” I tried some more dances, I didn’t cause any more collapses, and by the end of the night I was overflowing with exhilaration and joy. I went back the next week, and the week after that. I went to the twelve-hour Snow Ball and then the weekend-long Flurry Festival. I got to feel cool for the first time in my life when people started being excited to dance with me, and even cooler when some of them asked me on dates. It was a huge part of my life for a while, and although I’m now dancing at maybe a quarter of my peak intensity, it’s still one of my most joyful and treasured things I do. This happens to about one in ten people I take contra dancing: something clicks and it becomes a big part of their life. So what exactly is contra dancing? When people ask me this I often feel stuck, because describing the mechanics of a contra dance doesn’t really capture its essence or why it’s fun, but without understanding the mechanics, the essence doesn’t really make sense. So I’ll try both. But you have to promise to read through to the essence part, and not just bounce off if the mechanics sound silly. Mechanically: you show up to the dance hall. You ask someone to dance, or they ask you. You join a long line (“set”) of other couples and take hands in a four-person group with one adjacent (“neighbor”) couple. A caller stands at the top of the dance hall and does a “walk-through” of the dance you’re about to do, calling out a series of figures (“circle left three places,” “do-si-do your neighbor,” etc.) At the end, you’ve moved up or down the line, and you’re ready to do the same thing with some new neighbors coming at you. Next to the caller is a live band. (They probably sound a bit like this.) The pianist plays four potatoes and the fiddler launches into a reel. The caller calls out the figures for the first few times through the dance, then drops out as the dancers get into flow. It looks a bit like this video, although I don’t know how much this will transmit the vibe, since contra is a lot more fun to do than it is to watch: Bonus: you can see high school me in this dance! I'm in the dark red T-shirt at the very right of the frame at the beginning. To me, the essence of contra dancing is about flow, joy, and community. You can’t really tell from watching, but a lot of what makes for good contra dancing is learning how to trade momentum back and forth (“share weight”) with other dancers so that each figure flows seamlessly into the next one. When you get this right, it feels amazing. Subtle changes in how your partner holds your hand cue you into the next figure. The dance becomes effortless. You can turn your brain off and just do what the giant dancing super-organism invites you to do. It’s also physically joyful. The core figure of contra dancing is called the swing, in which you and your partner take a waltz-like hold and pivot around a shared center. Done right, it feels like a cross between flying and a world-class hug. Even outside of the swing, the shared weight, smooth momentum, and twirling “flourishes” will make you feel weightless, graceful and deeply connected to the music and the other dancers. Lots of kinds of dance feel flowy and exhilarating when done well. What makes contra different is how easy it is to get there. Most forms of dance require weeks of lessons before they get fun. In contra, all you need to do is show up for one beginner lesson half an hour before the dance starts.2 Of course, there’s plenty to learn after that—improving your swing technique, learning how to share weight, and where to put flourishes for the best joy and flow—but all of that is for getting you from a 8/10 to an 11/10 on the fun scale; getting to an 8 is pretty straightforward. So contra makes it easy to have a baseline level of fun dancing with anyone. And because it’s a group dance, you end up feeling connected to your entire set, not just your partner. I think those two things are behind my other favorite part of contra dancing, which is how friendly and welcoming the community is. Many kinds of dance are a bit snooty—the experienced dancers look down on newbies and try to avoid dancing with them. By contrast, my experience at my first dance, where I caused my square to crash and burn and was met with apologies and “I hope you’ll give it another shot,” is typical for contra. A lot of contras explicitly ask experienced dancers to dance with newcomers. It’s just hard to be snooty when you’re having so much fun! You want to share it with everyone.3 As draft reader Chloe put it: “I think in many ways, your story of your first dance is the heart of it—you can’t do it wrong. Or even if you do, it somehow leads to more love.” In my opinion, this makes contra dancing one of the best entry points to dance: Each dance is walked through, so you don’t have to worry about deciding what to do. You can get to an 8/10 fun level, where practice becomes rewarding instead of a chore, really quickly. The community is incredibly welcoming, which makes it a lot less stressful to be a beginner. While it’s structured enough to be beginner-friendly, there’s also enough room for improvisation and technique to give it a lot of depth. This makes it a great on-ramp to other dances and movement practices. When I started contra dancing I had a lot of fear and blocks about improvised dancing (for example, I didn’t feel comfortable dancing at standard American high school/wedding style dances), but recently I tried out fusion and contact improv and had fun at both. It’s pretty common for people to use contra as a gateway into more difficult forms of dance. Most importantly, though, it’s just incredibly joyful on its own! It’s the only activity I know of where I’ve had multiple friends mention that every time they go, their cheeks hurt from smiling so much. Beyond just fun, multiple draft readers independently commented that contra dance is a precise opposite of many things that bother them about… the rest of life: Much of my life happens on a screen in motionless silence; contra is as three-dimensional, movement- and sound-filled as you can get. The default mood of a lot of spaces is a sort of ironic detachment.4 The default mood of contra dancing is earnest joy. In everyday life it’s easy to end up atomized and isolated. At a contra, everyone does the same steps and makes the dance together; it’s deeply communal. In a lot of social contexts (especially in San Francisco), I often feel like I’m being “sized up” for how interesting or important I am. At most contras, people are just happy I’m there. Many things I do (work, choral music, blogging…) reward excellence above almost everything. Contra welcomes imperfection: we’d rather have an amateur live band than recorded professionals, and don’t believe in more than half an hour of organized teaching. This makes it a grounding and nourishing antitode to a lot of everyday stressors. When I was so drawn to it in middle school, I don’t think it was just about the fun: on some deep level it was very good for me. If I’ve gotten you excited enough to try contra, here’s what you should do: Go to try contra dot com to find dances near you. If there are multiple dances, I recommend trying to figure out which one is the largest (larger dances tend to have more energy and better vibes), although that might be tricky to figure out from the websites. Make a note of where and when the dance is, including the beginner lesson. Remember to show up beforehand for that! Dress code wise: contra dancing is generally light cardio; wear something you are happy to be active in.5 Most people dress pretty casually; many women (and some men!) wear skirts, especially skirts that do fun things when you twirl. Look at the dance site to see if they say anything about footwear; some dances request that you wear soft-soled shoes with clean soles to avoid damaging floors. Bring a water bottle! There are fountains at most dances but it helps to be able to hydrate more quickly since the breaks between dances are short. If you go to a dance and like it, the easiest way to go from 8/10 fun to 9.5/10 is to find someone with a really good swing and ask them how you can improve your technique. (If you’re not sure, look for someone who’s fast and smooth.) They will be tickled that you asked!6 If you have a friend who contra dances, try going with them! They will be thrilled to bring you and it’ll be even more fun for both of you.7 Have fun! I hope you smile so much your cheeks hurt. Thanks to Jeff Kaufman, Alex Allain, Chloe Lubinski, and Jessie Brown for commenting on a draft of this post. Further reading/listening/other consumption: My playlist of favorite contra dance music (because every dance has live music, contra has a thriving music scene that punches way above its weight!) Jeff Kaufman, Contra Dance as a Model For Post-AI Culture try contra dot com This is at least a 1-in-10,000 event; I’ve never seen it happen again in over 20 years of regular dancing. In fact, you can probably get away without even that! Another benefit of this is that contra is much more intergenerational than most forms of dance. I have dance friends in their twenties and seventies! At least for me, when I overuse ironic detachment it has subtle but far-reaching effects on how I see and engage with the world. It’s an easy way to amuse people, but ends up masking or numbing out a lot of more “real” responses. I’ve found it really interesting and a bit psychoactive to play what I call “the ironic detachment game,” in which I am banned from responding to anything in an ironically detached way. I personally sweat a lot and usually bring multiple shirts to change into, although most other dancers I know don’t need to do this. Teaching a swing is a separate skill from just having a good swing, so you may want to ask a few different people. I recently started a mission to get my friends into contra dancing (hence also this post)! It’s going great and it’s made it way more fun for me!

24th Apr 2026 1 votes
Advice for time management as a manager

This post was adapted from an internal doc I wrote at Wave. Welcome to being a manager! Your time-management problem just got a lot harder. As an IC, you can often get away with a very simple time-management strategy: Decide what your one most important thing is. Work on it until it’s done. GOTO 1 As a team lead, this isn’t going to work, because much more of your work is interrupt-driven or gets blocked for long periods of time. One-on-ones! Code reviews! Design reviews! Prioritization meetings! Project check-ins! All of these are subject to an external schedule rather than being the type of thing that you can push on in a single focused block until it’s done. Being a team lead means three big changes for your time management: You no longer have a single most important thing. You’ll have to learn how to juggle competing priorities. Your most important responsibility is for your team’s output, not your personal output. That means that individual engineering work goes last in your list of potential most important things. You’ll need to start spending some of your time on a manager’s schedule: There are two types of schedule, which I’ll call the manager’s schedule and the maker’s schedule. The manager’s schedule is for bosses. It’s embodied in the traditional appointment book, with each day cut into one hour intervals. You can block off several hours for a single task if you need to, but by default you change what you’re doing every hour. Most powerful people are on the manager’s schedule. It’s the schedule of command. But there’s another way of using time that’s common among people who make things, like programmers and writers. They generally prefer to use time in units of half a day at least. You can’t write or program well in units of an hour. That’s barely enough time to get started. Here’s some advice on how to cope with those changes. Have accurate expectations of yourself Your responsibilities to your team will take time, and even more importantly, attention. That means you’ll be a lot less productive on IC work than you have been in the past—especially at first while you’re finding your legs. Additionally, your time will be less predictable week-to-week as you might have to spend an unknown amount of time responding to “inbound” work. For the first few months, you should treat any individual engineering work that you get done as a bonus. Even after that, you should expect to have something like 10-20% less individual output per engineer you manage, depending on how experienced you are, they are, etc., and with substantial week-to-week variance. To mitigate this, my rule for myself has been to make sure that my IC work is important but not urgent—i.e. that nobody will be sad and no plans will be derailed if I end up having to spend the next week firefighting instead of pushing it forward. For honing my intuitions about how much I can actually expect to accomplish, I’ve found time tracking very useful (see How time tracking helped me be a better manager and I apparently got 50% better at my job last month). Prioritize ruthlessly A corollary of the above is that it becomes very important for you to prioritize what to work on, both on an hour-to-hour cadence and on a larger timescale. It’s not possible to write down a full algorithm for prioritizing in a blog post—that’s why they pay us the big bucks—but here are some heuristics for which things are most worth prioritizing: Deadlines where something bad happens if you miss them. (e.g. performance improvements in advance of a holiday rush.) Note that the badness of missing deadlines varies wildly. Make a habit of asking “what is the reason for this deadline?” for any deadline-driven project. Work that increases your or your team’s future bandwidth. This can include hiring, addressing tech / process debt, automating toil, mentoring people, reducing pager burden, etc. A useful way of thinking about it is to rank this kind of work based on the “payback period,” or how long it takes before the time saved by the improvement exceeds the time invested in making it. One-on-ones. Try very hard not to cancel these, unless you’re on vacation—the other 39.5 hours a week they’re focused on what you need from them, so please don’t disdain the 0.5 hours you spend focused on what they need from you. If you cancel too many, expect your reports to feel less safe bringing up tricky things, and to have more issues “blow up” because they didn’t get addressed early. Unemploy your future self One of the most important types of “work that increases your or your team’s future bandwidth” is delegating things. This is something entire books have been written about, but here’s how to avoid a few common delegation pitfalls for new team leads: Negotiate how hands-on to be. Effective delegation means finding the right balance between micromanaging, and throwing your report to the wolves. The stereotype is that new managers often micromanage, but at Wave, I’ve noticed that new managers often err on the side of undermanaging, or being too hands-off, perhaps out of a desire to signal that they trust their reports. If you’re unsure, it’s good to have an explicit conversation with the person you’re delegating to about how much support they want. E.g. “do you want to do the design for this feature yourself, have me do the high-level and fill in the details, or have me write the entire design doc?” Calibrate to your team members’ task-relevant maturity, or how capable they are of independently doing a particular type of task. A senior engineer should be able to design most features independently (with review), but give the same design task to a junior engineer and they’ll probably flail around and make no progress. You should be maintaining a mental map of each of your team members’ strengths and weaknesses—and updating it over time as you help them improve. Delegate ahead of future growth. Your team’s workload is going to increase over time, so even if you don’t feel like you have too much work to do right now, you probably will in the future, unless you currently feel underutilized. You should aim for a workload where in the steady state, you feel like you have some slack capacity. Delegate “stretch projects” to help your team level up. Getting enough slack might require you to delegate work that no one else on your team can currently do. Take your mental strengths-and-weaknesses map, ask yourself what the most important growth directions for each of those team members are, and figure out how to give them work that stretches them in that direction. Note that these delegations will probably require more frequent monitoring, since your reports will have less task-relevant maturity on their stretch projects! A five-step “help, I’m overwhelmed” checklist Despite your best efforts to follow the above advice, there will probably come a time when you feel very stressed about the amount of work on your plate. When that time comes, here’s what to do: Schedule time with your manager, for the soonest slot you can, to triage your todo list. (If the primary stakeholder for your scariest todos is your PM, schedule with them instead.) Make a list of everything that’s on your plate currently. Yes, everything, even that code review that’s been sitting in your backlog for the last 3 months. At the meeting you scheduled in step 1, figure out how to delegate everything in that list you can delegate, then stack-rank the remainder. Realistically (see Have accurate expectations of yourself) decide how far down the list you’re going to get. Remember to leave yourself some slack for whatever comes up! For things below the cutoff, decide that you’re not going to do them, and notify everyone who cares that you probably won’t get to it. Carve out focused time If you’re not careful, it’s easy to fill your entire calendar with meetings, Slack, etc. and have no time for deep work. With careful planning, you can avoid this by “batching” all your distractions to particular times of day. There are lots of tactical tips for doing this; I catalogued some that work for me in Tools for keeping focused. One tech-lead-specific one that I’ll add is batching meetings: I schedule all my meetings back-to-back on Tuesdays and Thursdays to leave the rest of the week as free as possible for deep work. Appendix: further reading Rest in Motion How time tracking helped me be a better manager Time management: the leadership meta-problem. Tools for keeping focused Attention is your scarcest resource I apparently got 50% better at my job last month The Top Idea in Your Mind Maker’s Schedule, Manager’s Schedule My weekly review habit

1st Apr 2025 1 votes
How I've run major projects

My few most productive individual weeks at Anthropic have all been “crisis project management:” coordinating major, time-sensitive implementation or debugging efforts. In a company like Anthropic, excellent project management is an extremely high-leverage skill, and not just during crises: our work has tons of moving parts with complex, non-obvious interdependencies and hard schedule constraints, which means organizing them is a huge job, and can save weeks of delays if done right. Although a lot of the examples here come from crisis projects, most of the principles here are also the way I try to run any project, just more-so. I think excellent project management is also rarer than it needs to be. During the crisis projects I didn’t feel like I was doing anything particularly impressive; mostly it felt like I was putting in a lot of work but doing things that felt relatively straightforward. On the other hand, I often see other people miss chances to do those things, maybe for lack of having seen a good playbook. So here’s an attempt to describe my playbook for when I’m being intense about project management. (I’ve described what I did as “coordinating” above, but that’s underselling it a bit; it mattered a lot for this playbook that I had enough technical context, and organizational trust, to autonomously make most prioritization decisions about the project. Sometimes we instead try to have the trusted decisionmakers not be highly involved in managing execution, and instead farm that out to a lower-context or less-trusted project manager to save the trusted decisionmaker time, but IMO this is usually a false economy for projects where it’s critical that they be executed well.) Focus For each of the crisis management projects I completely cleared my schedule to focus on them, and ended up spending 6+ hours a day organizing them. This is a bit unintuitive because I’m used to thinking of information processing as basically a free action. After all, you’re “just” moving info from place to place, not doing real work like coding, right? But if you add it all up—running meetings, pinging for updates, digesting Slack threads, pinging for updates again, thinking about what’s next, pinging for updates a third time, etc.—it’s surprisingly time-intensive. Even more importantly than freeing up time, clearing my schedule made sure the project was the top idea in my mind. If I don’t do that, it’s easy for me to let projects “go on autopilot,” where I keep them running but don’t proactively make time to think through things like whether we should change goals, add or drop priorities, or do other “non-obvious” things. For non-crisis projects, it’s often not tenable (or the right prioritization) to spend 6+ hours a day project-managing; but it’s still the case that you can improve execution a lot if you focus and make them a top priority, e.g. by carving out dedicated time every day to check statuses, contemplate priorities, broadcast updates, and so on. Maintain a detailed plan for victory A specific tool that I’ve found critical for staying oriented and updating quickly is a detailed plan for victory, i.e., a list of steps, as concrete as possible, that end with the goal being achieved. The plan is important because whether or not we’re achieving the plan is the best way to figure out how well or badly things are going. Knowing how well or badly things are going is important because it tells me when to start asking for more support, cutting scope, escalating problems, and otherwise sounding more alarms. One of the most common megaproject failure modes is to not freak out soon enough, and having a concrete plan is the best antidote. As a both positive and negative example of this, during a recent sprint to release a new implementation of a model, we took a detailed accounting of all the work we thought we had to do to launch. On the plus side, this made it clear three months before launch that things were going to be very tight, and this enabled us to ask for help from another team, who loaned us someone who sped up the project a fair amount. On the minus side, we also massively underestimated a few components of the project, and because of this, we still ended up very crunched at the end. As the above example shows, having a plan can’t completely save you if you underestimate how long all the steps in the plan will take. But it certainly helps! My sense is that the main things that would have helped even more in the above case were: We were inexperienced at estimating tasks, especially tasks related to new model implementations (which most people on the team were too new to have done before), and we were too cowardly to add the requisite amount of “slop” to our plan. We didn’t check in frequently enough against the plan once we made it, or sound the alarm early enough when we went off-plan. Run a fast OODA loop OODA stands for “observe, orient, decide, act”—in other words, the process by which you update your plans and behavior based on new information. Most of the large projects I’ve worked on have been characterized by incomplete information: Our cluster’s networking is bad, but we don’t understand why. We have a correctness bug but we don’t know where it is. We need to rewrite the system but we’re not totally sure what the rewrite should look like. In fact, I’d make a stronger claim: usually getting complete information was the hard part of the project, and took up a substantial fraction of the overall critical-path timeline. For example, let’s take a recent project to kick off a training run. The critical path probably looked something like: Chips for the training run are delivered We run some tests We discover one aspect of performance is unexpectedly poor We escalate the problem with our compute partner Compute partner staffs a large debugging effort We realize we had given our compute partner an outdated benchmark that is causing them to target the wrong improvements Compute partner switches benchmark and prioritizes different improvements We share our benchmarks with compute partner so they can run the exact same code as us Compute partner rolls out improvements We test the improvements Performance is still poor and we tell them that Repeat steps 8-10 until eventually it’s good enough Practically all of these steps are about information-processing, not writing code! Even the step where the compute partner debugged the problems on their side was itself constrained by information processing speed, since there were tens of people working on the debugging effort and coordinating / sharing info between them was difficult. Overall, the project timeline was strongly constrained by how quickly information could round-trip from our compute partner’s large-scale debugging effort, through their tech lead, me, and Anthropic’s large-scale debugging effort. This pattern generalizes to most projects I’ve been a part of, and as a result, one of my most productive project management habits is to try to run the fastest OODA loop that I can. A few specific things that I’ve found help: Spend time on it: running OODA loops takes time, and is one of the primary reasons that, as mentioned above, I usually spend 6+ hours a day on running a megaproject if it’s in crisis mode. Communicate uncomfortably much: For the training run debugging, to reduce the round-trip time between orgs as much as possible, I had multiple daily calls with my counterpart at our compute partner (9am and 6pm). For the model implementation effort, I was basically constantly bouncing between different groups of debuggers, asking for updates and processing them. Track and prioritize the biggest open questions: For most big projects I’ve maintained a living doc with a ranked list of all my biggest open questions about the project. Resolving or de-risking these uncertainties basically turns into the project’s priority list. Step back and reorient frequently: Other than asking for updates, the main thing I spend time on was reorienting—looking at our list of priorities, asking myself whether they should still be the top priorities, then looking at what people were working on, and making sure those things were attacking the top priorities. I probably reviewed the project’s priorities multiple times a day as well, although I often didn’t make changes as a result. (Note that it is possible to change what people are working on too often, since switching tasks is costly. Parallelizing work on the top few priorities, as mentioned above, helps with this, since if you decide that priority #3 is now #1, but there are 2 people working on each, then nobody has to switch tasks. The thing that kills you is when no one is working on the new priority #1.) Overcommunicate It’s not just enough for me personally to be running a fast OODA loop—in a large group, everyone needs to be autonomously making frequent, high-quality, local prioritization decisions, without needing a round-trip through me. To get there, they need to be ambiently aware of: what else is going on around them, so they can coordinate and update on new info quickly (“oh, we’re planning to kick off the next derisking run in three days, so I have to have my new RL environment ready and tested by then”) how their goal fits into the overall project, so they can make correct decisions about the details of their approach (“we’re trying to scale up as much as possible right now, so this direction isn’t valuable to pursue since it could never provide the scale of data we need”) I’ve usually found that to create the right level of ambient awareness, I have to repeat the same things way more often than I intuitively expect. This is roughly the same “communicate uncomfortably much” principle above, but applied to broadcasts and not just 1:1 conversations with people. For example, although the first team I managed at Anthropic started with a daily asynchronous standup, we found that synchronous meetings were much more effective for creating common knowledge and reorienting, so we moved to a twice-weekly synchronous standup, which probably qualified as “uncomfortably much” synchronous communication for Anthropic at the time. Break off subprojects Once a project gets over maybe 10 people, I can’t track everything myself in enough detail to project-manage the entire thing myself. At this point, it becomes critical to delegate. Here I mean delegating the project management, not just the execution (that’s what I’d be delegating to the first 10 people). This is the point where I need other people to help split up the work, monitor and communicate progress, escalate blockers, etc. A few things I try to keep in mind when delegating project management: The ideal unit of delegation is a crisp, simple, high-level goal, with limited overlap with other workstreams. (This is as opposed to, e.g., a list of tasks like “see if X helps.“) Good examples: “get X training technique working over Y networking protocol at Z throughput,” “get identical evals between model implementations A and B.” Bad examples: “follow this 10-step checklist that we hope results in training working,” “try these 3 techniques for debugging the loss eval.” The best project-managers are often not the strongest technical ICs. Instead the most important traits are that they’re highly organized and great at staying laser focused on end goals, perhaps to the point of being annoying about it. IC depth helps and I’ll never say no to it, but it’s not what I’d optimize for. People running subprojects are probably also doing a lot of the same stuff I do, in particular e.g. spending a lot of time on it. That means they’ll take a substantial hit to their IC productivity. This is expected, and is often worth it. “Direction is more important than magnitude”—it’s usually better to have a lower-velocity project that works on the right things, than a higher-velocity one that’s pointed at the wrong goal. One of my favorite things to make delegation easier is to keep goals simple—if they can fit in a Slack message while still crisply describing a path to the desired end state, then the people working on the goal will be much more able to prioritize autonomously, and point their work at the real end goal rather than doing something that turns out to be useless for some reason they didn’t think about. “Keep goals simple” doesn’t have to mean “do less”—the best way to keep goals simple is to find the latent structure that enables a clean recursive decomposition into subgoals. This often requires a deceptive amount of work—both cognitive and hands-on-keyboard—to identify the right intermediate goals, but I’ve found that it pays off immensely by clarifying what’s important to work on. Have fun Some of my favorite memories of Anthropic are of helping out with these big projects. While they can be intense, it’s also really inspiring to see how our team comes together, and the feeling of being part of a big team of truly excellent people cooking something ambitious together can be really magical! So I try to enjoy the chaos :) Appendix: my project DRI starter kit Here’s the internal doc I share with folks on my team who are getting into being responsible for large projects. So you’re the DRI of a project (or part of one). Concretely, what do you do to “be DRI”? This doc is my suggested “starter kit” answer to that question. The habits and rituals described here aren’t perfect for every situation, but they’re lightweight and broadly helpful. I suggest you use them as a starting point for iteration: try them out, then adjust as necessary. This is an SL init; the RL is your job :) Goals of this playbook The goal is to help you do your job as DRI— Make your project go quickly: Participants deeply understand the root goal and can autonomously choose the most important next things to work on People have “situational awareness” of what other people are working on, learn about relevant updates quickly, and coordinate quickly when needed People get quick feedback on their work If things aren’t going fast enough, you (the DRI) can notice and course-correct quickly “Play well with others:” Observers can figure out where to go to follow along Adjacent or intersecting people/projects don’t miss important updates or get caught by surprise People notice quickly if the project is behind or off-track, and can identify opportunities to help —without adding too much overhead: <1 hour of setup to make a working doc, schedule a weekly meeting, etc. 30 min/week of meetings 15-30 min/week to write an update (Note: being DRI will still unavoidably add some overhead—e.g. you’ll have to track what other people are doing, delegate work, unblock people, set and communicate goals, etc. The goal is specifically for the process/paperwork to be minimal.) Weekly meeting You should schedule at least one 30-minute weekly meeting with everyone working on the project. The goal of this meeting is to (1) be a backstop for any coordination that needs to happen and didn’t happen asynchronously; (2) be an efficient way to create common knowledge of goals, updates, etc.; (3) help you track whether things are going well. Starter-kit agenda: [5m] DRI reviews major updates from last week and sets goals for next week [10m] Silent write and comment on discussion topics [10m] Synchronous discussion of most important things not addressed during silent write Signs that more meetings might help (e.g. a second weekly standup): you have a very tight deadline and can’t afford to lose time people aren’t working on the most important thing people need feedback frequently people step on each others’ toes or miss opportunities to help each other out if you just like hanging out with each other :) Landing page / working doc It’s really helpful for discoverability and wayfinding to have a single “master doc” with all the most important info about a project. As you loop more people in, they can read the doc to get up to speed. And anyone who thinks “I wonder how X is going” can stop by there to find out. Create a doc for your workstream with: A go/ link in the name (if a subproject, maybe use go/project/subproject) → This makes it easier to find quickly (search is kinda rough) A clear description of a concrete top level goal and how it fits into broader goals → This is critical info for participants, so they can autonomously prioritize the most important things; and for observers, so that they know what outcome to expect. Staffing: A list of people working on the project, your name as the DRI, and a link to the slack channel that’s being used for discussion Links: A short list of relevant links at the top (work trackers, the project’s Slack channel, major design docs, etc.). If needed, a longer “docs / see also” section later links to relevant docs. → It’s really easy to lose track of relevant docs otherwise! A roadmap section with intermediate goals and target dates → See the section on plans; these will help people understand what the overall shape of the project is expected to be. A section for “running notes” containing meeting notes from your weekly meetings (and any other ad-hoc meetings) and broadcast updates → This really helps observers and new-joiners get up to speed! I like maintaining a list of important open questions / uncertainties/ risks and updating it over time. This helps me stay focused on removing risk from the project as quickly as possible. If it’s part of a larger project, your doc should be nested within the larger project’s working doc. If this ends up being too much for one doc, you can fork these out into sub-docs (esp. running notes and updates). Plan / roadmap / milestones In your working doc, include a section with some intermediate goals and dates by which you hope to accomplish them. → This is helpful mostly for noticing you’re off track or behind without getting frog-boiled. → Or noticing when you need to make a direction change because the intermediate goals don’t seem good anymore. You might feel some pressure to add false certainty or precision, but avoid this and be honest about your uncertainty instead. For a lot of research projects it’s hard to plan more than a couple weeks ahead. You can make the milestones fuzzier / more aspirational beyond that, or just drop them. I often find it helpful to phrase milestones in probabilitis and distributions (e.g. “my 90% confidence interval for this date is X-Y” or “I think there’s a 75% chance this technique works”) Who’s working on what You should have something somewhere that describes what people are working on. The minimum viable version of this is a list of what people are working on in your working doc. If you end up with a large set of tasks and a big backlog, maybe use a checklist and/or move to a subdoc. Stack rank your work list. It’s really important for people to understand priorities! If there’s more different people/TODOs, I suggest using some app to make a kanban board with “backlog” / “up next” / “in progress” / “done” columns. This is probably most helpful for more deterministic/plannable projects where there’s a clear backlog + set of future tasks, and a lot of things you need to remember to do. If you have an external task tracker, link it in the wiki section of the working doc. Slack norms Have conversations about the project in a Slack channel (not DMs). Reference the channel in your working doc. Link the working doc in the Slack channel bookmarks. Cross-post notebook posts and experiment write-ups into the channel so observers don’t have to follow tons of notebook channels. Do not use DMs. These make it hard to make info discoverable or share it further. If people send you important stuff in a DM, ask them to put it in the project channel. If you need confidentiality, make a private channel. Avoid centithreads. Most ≥10-message Slack threads would be better as a ~5-minute Tuple. (This is hard to do with people who are in tons and tons of meetings like execs. But you should try to do it for others.) If you end up with a centithread, assume nobody will read it; post a summary back to the channel afterwards. Bias towards fewer, larger, noisier channels. The right time to create a channel is when discussion is either not happening, or getting lost. → Too many slack channels makes it harder to manage membership, decide where to put things, or find where discussion is happening. Channel organization and membership matters. Invest in routing conversations to the right place and curating the channel “architecture.” Weekly broadcast updates Once a week, probably either just before or just after your weekly meeting, write up a brief update for a broader audience with: The overall vibe What’s changed since last update What’s coming up next When writing these updates, optimize for signal to noise ratio. Err towards concision No “we worked on X”—tell me “we accomplished Y” or “we learned Z” Remember your audience (= people not familiar with the project) State things crisply and concretely (“X improves eval Y by Z points,” not “we got X working”) Leave out anything that’s not actionable—you don’t need to be exhaustive Post the update in your project Slack channel, and cross-post it to other relevant channels (e.g. a larger “megaproject” channel) if necessary. If your project is part of a larger megaproject, these updates might feed into something broader like a weekly meeting of DRIs or an aggregated status update. Retrospectives Every so often, step back and ask “how could the last X weeks have gone better?” Frequency depends on how much there is going on—every 2 weeks is good if there’s a lot, maybe every 4-8 weeks for smaller projects Suggested meeting format Friday afternoon [13 min] Async brainstorm 2 lists of items: “what went well” / “what we could improve” [2 min] Dedupe topics and emoji vote by putting :heavy_plus_sign: next to ones you agree with Sort “what we could improve” by highest votes [10 min] Synchronous discussion of top points (either highest voted or flagged by DRI); figure out action items Thanks to Kelley Rivoire for many thoughtful comments on a draft!

16th Mar 2025 1 votes
How I build and run behavioral interviews

This is an adaptation of an internal doc I wrote for Wave. I used to think that behavioral interviews were basically useless, because it was too easy for candidates to bullshit them and too hard for me to tell what was a good answer. I’d end up grading every candidate as an “okay, I guess” because I was never sure what bar I should hold them to. I still think most behavioral interviews are like that, but after grinding out way too many of them, I now think it’s possible to escape that trap. Here are my tips and tricks for doing so! Confidence level: doing this stuff worked better than not doing it, but I still feel like I could be a lot better at behavioral interviews, so please suggest improvements and/or do your own thing :) Before the interview Budget 2+ hours to build That’s how long I usually take to design and prepare a new type of interview. If I spend a couple hours thinking about what questions and follow-ups to ask, I’m much more likely to get a strong signal about which candidates performed well. It might sounds ridiculous to spend 2 hours building a 1-hour interview that you’ll only give 4 times. But it’s worth it! Your most limited resource is time with candidates, so if you can spend more of your own time to use candidates’ time better, that’s worth it. Think ahead about follow-ups and rubric I spend most of those 2 hours trying to answer the following question: “what answers to these questions would distinguish a great candidate from a mediocre one, and how can I dig for that?” I find that if I wait until after the interview to evaluate candidates, I rarely have conviction about them, and fall back to grading them a “weak hire” or “weak no-hire.” To avoid this, write yourself a rubric of all the things you care about assessing, and what follow-up questions you’ll ask to assess those things. This will help you deliver the interview consistently, but most importantly, you’ll ask much better follow-up questions if you’ve thought about them beforehand. See the appendix for an example rubric. Focus on a small number of skills I usually focus on 1-3 related skills or traits. To get a strong signal from a behavioral interview question I usually need around 15 minutes, which only leaves time to discuss a small number of scenarios. For example, for a head of technical recruiting, I decided to focus my interview on the cluster of related traits of being great at communication, representing our culture to candidates, and holding a high bar for job candidate experience. You should coordinate with the rest of the folks on your interview loop to make sure that, collectively, you cover all the most important traits for the role. During the interview Kicking off My formula for kicking off a behavioral question is “Tell me about a recent time when [X situation happened]. Just give me some brief high-level context on the situation, what the problem was,1 and how you addressed it. You can keep it high-level and I’ll ask follow-up questions afterward.” I usually ask for a recent time to avoid having them pick the one time that paints them in the best possible light. The second sentence (context/problem/solution) is important for helping the candidate keep their initial answer focused—otherwise, they are more likely to ramble for a long time and leave less time for you to… Dig into details Almost everyone will answer the initial behavioral interview prompt with something that sounds vaguely like it makes sense, even if they don’t actually usually behave in the ways you’re looking for. To figure out whether they’re real or BSing you, the best way is to get them to tell you a lot of details about the situation—the more you get them to tell you, the harder it will be to BS all the details. General follow-ups you can use to get more detail: Ask for a timeline—how quickly people operate can be very informative. (Example: I asked someone how they dealt with an underperforming direct report and they gave a compelling story, but when I asked for the timeline, it seemed that weeks had elapsed between noticing the problem and doing anything about it.) “And then what happened?” / “What was the outcome?” (Example: I asked this to a tech recruiter for the “underperforming report” question and they admitted they had to fire the person, which they hadn’t previously mentioned—that’s a yellow flag on honesty.) Ask how big of an effect something had and how they know. (Example: I had a head of technical recruiting tell me “I did X and our outbound response rate improved;” when I asked how much, he said from 11% to 15%, but the sample size was small enough that that could have been random chance!) “Is there anything you wish you’d done differently?” (Sometimes people respond to this with non-actionable takeaways like “I wish I’d thought of that idea earlier” but having no plan or mechanism that could possibly cause them to think about the idea earlier the next time.) Evaluating candidates Make yourself a rubric One of the worst mistakes you can make in a behavioral interview is to wing it: to ask whatever follow-up questions pop into your head, and then at the end try to answer the question, “did I like this person?” If you do that, you’re much more likely to be a “weak yes” or “weak no” on every candidate, and to miss asking the follow-up questions that could have given you stronger signal. Instead, you should know what you’re looking for, and what directions to probe in, before you start the interview. The best way to do this is to build a scoring rubric, where you decide what you’re going to look for and what a good vs. bad answer looks like. See the appendix for an example. General things to watch out for Of course, most of your rubric should be based on the details of what traits you’re trying to evaluate! But here are some failure modes that are common to most behavioral interviews: Vague platitudes: some people have a tendency to fall back on vague generalities in behavioral interviews. “In recruiting, it’s all about communication!” “No org structure is perfect!” If they don’t follow this up with a more specific, precise or nuanced claim, they may not be a strong first-principles thinker. Communication bandwidth: if you find that you’re struggling to understand what the person is saying or get on the same page as them, this is a bad sign about your ability to discuss nuanced topics in the future if you work together. Self-improvement mindset: if the person responds to “what would you do differently” with “nothing,” or with non-actionable vague platitudes, it’s a sign they may not be great at figuring out how to get better at things over time. Being embarrassingly honest: if probing for more details causes you to learn that the thing went less well than the original impression you got, the candidate probably is trying to “spin” this at least a little bit. High standards: if they say there’s nothing they wish they’d done differently, this may also be lack of embarrassing honesty, or not holding themselves to a high standard. (Personally, even for any project that went exceptionally well I can think of lots of individual things I could have done better!) Scapegoating: if you ask about solving a problem, do they take responsibility for contributing to the problem? it’s common for people to imply/say that problems were all caused by other people and solved by them (eg “this hiring manager wanted to do it their way, but I knew they were wrong, but couldn’t convince them…”). Sometimes this is true, but usually problems aren’t a single person’s fault! Appendix: example rubric and follow-ups Here’s an example rubric and set of follow-up questions for a Head of Technical Recruiting. Question: “tell me about a time when your report wasn’t doing a good job.” moving quickly to detect and address the issue ask for a timeline of events  bad answer = lots of slop in time between “when problem started” / “when you discovered” / “when you addressed” setting clear expectations with their report and being embarrassingly honest ask what the conversation with their report was like making their reports feel psychologically safe ask how they thought their report felt after the tough convo bad answer = not sure, or saying things in a non-supportive / non-generous way being effective at discovering the root problem ask a mini postmortem / “five whys”  bad answer = not having deep understanding of root dynamics, only symptoms do they understand whether what they did worked ask for concrete metrics on how things were going before and after they intervened bad answer = not having metrics, having metrics that moved only a small amount (and not realizing this is a failure), etc. learning and updating over time ask them what they could have done differently next time bad answer = vague platitudes or “nothing” For behaviors that involve addressing problems—reword “problem” as something else as appropriate.

25th Feb 2024 1 votes

More in startups

Divorced from reality

Fundamentals will matter

6 days ago 2 votes
Tuesday

My breathing was a constant problem.

6 days ago 2 votes
Industry roundup #13

When "buy, don't build" stops applying to dev tools

a week ago 1 votes
AI, tools and transformation

It’s very tempting to imagine that AI turns everyone into a tool-builder - now everyone can just ask the model to make the software they need, and apps as we know them are dead. I think that misunderstands how most people think and where software actually comes from, and more importantly, it isn’t a path to change how companies actually work.

a week ago 2 votes
Paralyzed by the Perfect Resume

Those with the most to lose are often the least likely to take big swings

a week ago 2 votes
📚 BoredReading

You seem to be enjoying this.

Join free to unlock everything.

Create free account

Already have an account? Sign in