Full Width [alt+shift+f] Shortcuts [alt+shift+k]
Sign Up [alt+shift+s] Log In [alt+shift+l]
1

Learnings from a week of building with local LLMs

from Alex Strick van Linschoten [alt+shift+b] in AI

I took the past week off to work on a little side project. More on that at some point, but at its heart it’s an extension of what I worked on with my translation package tinbox. (The new project uses translated sources to bootstrap a knowledge database.) Building in an environment which has less pressure / deadlines gives you space to experiment, so I both tried out a bunch of new tools and also experimented with different ways of using my tried-and-tested development tools/processes. Along the way, there were a bunch of small insights which occurred to me so I thought I’d write them down. As usual with this blog, I’m mainly writing for my future self but I think there might be parts that are useful for others! Apologies for the somewhat rushed nature of these observations; better I get the blog finished and published than not at all! 🤖 Local Models During this project, I experimented with several local models, which continue to impress me with their evolving capabilities. The recent launch of gemma3 was particularly timely - I found myself regularly using the 27B version, which performed admirably across various tasks. There are three or four models I keep returning to. mistral-small stands out as an exceptional model that’s been relatively recently updated and seems a bit underrated / underappreciated. The original mistral model continues to hold up remarkably well, particularly for structured extraction tasks and general writing needs like summarization. One important realization when working with real-world use cases: benchmarks can be deceptive. While helpful as general indicators, each model has its own strengths and quirks. Many newer models are heavily optimized for structured data extraction, but their performance ultimately depends on whether their training documents align with your specific use case. It’s crucial to test models against your actual requirements rather than relying solely on published benchmarks. For robust results with local models, I’ve...
15th Mar 2025

Stay updated

Get a weekly newsletter with the top 5 articles worth reading every week.

More from Alex Strick van Linschoten

Testing out instrumenting LLM tracing for litellm with Braintrust and Langfuse

I previously tried (and failed) to setup LLM tracing for hinbox using Arize Phoenix and litellm. Since this is sort of a priority for being able to follow along with the Hamel / Shreya evals course with my practical application, I’ll take another stab using a tool with which I’m familiar: Braintrust. Let’s start simple and then if it works the way we want we can set things up for hinbox as well. Simple Braintrust tracing with litellm callbacks Callbacks are listed in the litellm docs as the way to do tracing with Braintrust. So we can do something like this: import litellm litellm.callbacks = ["braintrust"] completion_response = litellm.completion( model="openrouter/google/gemma-3n-e4b-it:free", messages=[ { "content": "What's the capital of China? Just give me the name.", "role": "user", } ], metadata={ # "project_id": "1235-a70e-4571-abcd-234235", "project_name": "hinbox", }, ) print(completion_response.choices[0].message.content) You can pass in a project_id or a project_name and the traces will be routed there. Here’s what it looks like in the Braintrust dashboard: Our first trace logged in Braintrust Note how you can’t see which model was used for the LLM call, nor any cost estimates. The docs mention that you can pass metadata into Braintrust using the metadata property: “braintrust_* - any metadata field starting with braintrust_ will be passed as metadata to the logging request” (link) This seems a bit rudimentary, however. If we take a look at the full tracing documentation on the Braintrust docs we can see that they seem to recommend wrapping the OpenAI client object instead: import os from braintrust import init_logger, traced, wrap_openai from openai import OpenAI logger = init_logger(project="hinbox") client = wrap_openai(OpenAI(api_key=os.environ["OPENAI_API_KEY"])) # @traced automatically logs the input (args) and output (return value) # of this function to a span. To ensure the span is named `answer_question`, # you should name the function `answer_question`. @traced def answer_question(body: str) -> str: prompt = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": body}, ] result = client.chat.completions.create( model="gpt-3.5-turbo", messages=prompt, ) return result.choices[0].message.content def main(): input_text = "What's the capital of China? Just give me the name." result = answer_question(input_text) print(result) if __name__ == "__main__": main() This indeed does label the span as answer_question but it doesn’t do much else. Even the model name isn’t captured here. Instrumenting a series of calls to handle ‘deeply nested code’ (as their docs puts it) even didn’t log the things it was supposed to: import os import random from braintrust import current_span, init_logger, start_span, traced, wrap_openai from openai import OpenAI logger = init_logger(project="hinbox") client = wrap_openai(OpenAI(api_key=os.environ["OPENAI_API_KEY"])) @traced def run_llm(input): model = "gpt-4o" if random.random() > 0.5 else "gpt-4o-mini" result = client.chat.completions.create( model=model, messages=[{"role": "user", "content": input}] ) current_span().log(metadata={"randomModel": model}) return result.choices[0].message.content @traced def some_logic(input): return run_llm("You are a magical wizard. Answer the following question: " + input) def simple_handler(input_text: str): with start_span() as span: output = some_logic(input_text) span.log(input=input_text, output=output, metadata=dict(user_id="test_user")) print(output) if __name__ == "__main__": question = "What's the capital of China? Just give me the name." simple_handler(question) This is adapted from the example they pasted in their docs as their one isn’t even a functional code example on its own. It is seeming increasingly clear that Braintrust isn’t going to be the right choice, at least as long as I want to keep using litellm. I know that Langfuse has a very nice integration with litellm, so I think I’ll pivot over to that now. Basic tracing with Langfuse and litellm Simple tracing is easy: import litellm litellm.callbacks = ["langfuse"] def query_llm(prompt: str): completion_response = litellm.completion( model="openrouter/google/gemma-3n-e4b-it:free", messages=[ { "content": "What's the capital of China? Just give me the name.", "role": "user", } ], ) return completion_response.choices[0].message.content def my_llm_application(): query1 = query_llm("What's the capital of China? Just give me the name.") query2 = query_llm("What's the capital of Japan? Just give me the name.") return (query1, query2) print(my_llm_application()) We specify langfuse for the callback and each llm call is logged as a separate trace + span. Here you can see what this looks like in the dashboard: Basic trace and span in Langfuse dashboard The litellm docs include information on how to specify custom metadata and grouping instructions for Langfuse. Notably, we can specify (as of June 2025, at least!) things like a session_id, tags, a trace_name and/or trace_id as well as custom trace metadata and so on. So we can get most of what we want to specify in the following way: import litellm litellm.callbacks = ["langfuse"] def query_llm(prompt: str, trace_id: str): completion_response = litellm.completion( model="openrouter/google/gemma-3n-e4b-it:free", messages=[ { "content": "What's the capital of China? Just give me the name.", "role": "user", } ], metadata={ "trace_id": trace_id, "trace_name": "my_llm_application", "project": "hinbox", }, ) return completion_response.choices[0].message.content def my_llm_application(): query1 = query_llm( "What's the capital of China? Just give me the name.", "my_llm_application_run_789", ) query2 = query_llm( "What's the capital of Japan? Just give me the name.", "my_llm_application_run_789", ) return (query1, query2) if __name__ == "__main__": print(my_llm_application()) This looks like this in the Langfuse dashboard: Spans grouped into traces This is honestly most of what I’m looking for in terms of my tracing. If I were to use a non-OpenRouter model, moreover, I’d also get full costs in the Langfuse dashboard, e.g.: LLM costs in Langfuse dashboard As such, I can monitor costs from within OpenRouter and have the option to keep track of costs in Langfuse by passing custom metadata should I wish. I’ll make a separate blog where I actually go into how I set up + instrumented hinbox for this kind of tracing while continuing to use litellm.

3rd Jun 2025 • 1 votes
How to think about evals

Today was the first session of Hamel + Shreya’s course, “AI Evals for Engineers and PMs”. The first session was all about mental models for thinking about the topic as a whole, mixed in with some teasers of practical examples and advice. I’ll try to keep up with blogging about what I learn as we go. Most of the actual content will go up online at some point in the future, I’m assuming, so not much point writing up super detailed notes. (There is also a book coming, which I assume will be great, and about which you can learn more here.) So in general I’ll try to be doing the following as I blog along: highlight things I found interesting or inspiring based on the formal ‘lectures’ anything that comes up while doing the practical ‘homework’ (there are some optional exercises assigned to ground everything) contextualise or situate things that come up in my own experience having worked on a few LLM-driven projects Today, fresh out of the first class, I wanted to write about the mental model of the ‘three gulfs’ that they propose, the improvement loop that they suggest is how to measurably improve your applications, and also prompting through the lens of evals. Finally I’ll round off with a bit about what I’ll be exploring this week. The Three Gulfs: Specification, Generalization and Comprehension So there’s this image that they shared in the book chapter preview discussion that came up again during the lesson today: The three gulfs of LLM application development (They’ve shared it already in the YouTube discussion + I see it on Twitter being shared so I think I’m not sharing something I ought not to!) The course is very practically focused, especially so for application developers, so this diagram is in that context. The diagram offers up a way of thinking about LLM application development that pinpoints the places where you might do your work, and it’s also a way of thinking through things systematically, too. I was especially interested in the differentiation between the gulf of specification and the gulf of generalisation, since these can often feel similar, but actually the way to get out of them is actually slightly different. I’ll go into a bit more detail below, but basically with the gulf of specification you might want to be working on your prompts + how specific you are, whereas with the generalisation gulf you might need things like splitting up your tasks or making sure your system is outputting things in a structured way, etc etc. Note also that the world of tooling also doesn’t help you in a specific or targeted way to focus on one aspect of this diagram. Too often the tools try to cover the whole picture and probably also muddy the water by eliding the differences between the different tasks and challenges of each island or the gulf in between. All this is pretty abstract, so let’s go through them one by one. The Gulf of Comprehension in Practice This was seen as sort of the starting point for thinking through LLM Application improvement. At this point your big problem is that you’re trying to understand the data that comes your way from your users. You’re trying to understand the inputs to your application (what your users are typing, assuming that text is the medium of communication / input) and you’re trying to understand what the application or LLM is outputting. The challenge comes because you can’t read every single log or morsel of data. You have to filter things down somehow! If this were something more like traditional ML you’d have statistics to help boil down your data, but mostly we’re talking about unstructured text data so it’s much more unwieldy. This challenge means that people often get stuck at this point. This is where POC applications live, breathe and eventually die. You have enough sense that things are ‘kinda’ working, but you don’t really know what the failure modes are, so you don’t know how to improve it. You’ve tried out one or two things in a halfway systematic way, but really you have no idea what’s working well and what’s not. On Tools vs Process Hamel made the good point that it’s probably not so useful to think about tools too much when thinking at this stage. Generally speaking what’s going on is most often actually a process problem and trying to go straight to ‘what tool do I need’ is probably avoiding the real issue. The Gulf of Specification in Practice This is the place where you are trying to translate intent into precise instructions that the LLM will follow. You’re trying to be explicit and specific in the hope that the LLM will do what you want it to do, and not do the things that you don’t want it to do. The obvious manifestation of this is people writing bad prompts. It might seem that it’s also present when you try to have an LLM solve one problem when it’s either unsuited for that task or the task needs to be broken up and so on, but that’s the sister gulf of generalisation. Here, we’re focused on how to improve the specificity of your prompts. When you split things up and highlight the fact that prompts are something that you’ll need to work on and to improve, it becomes clear that it’s something you wouldn’t want to outsource or to skimp time on. Really the prompt writing is the thing that you (at certain moments, and where it’s identified as the thing needing focus / improvement) want to be working on in partnership with domain experts. For small applications, you might be the same person as the domain expert! For bigger projects, you might be working with domain experts. Just be aware that often the domain expert might not necessarily be detached enough to be able to figure out what needs the focus, or where the weaknesses of a prompt are. That’s what the iterative process / error analysis and everything else that’ll be taught in the course is for (see below and see future posts). Another point Hamel made was about why prompts are actually so important: “you have to express your project taste somewhere”. Given that your application might be fully / mostly driven by LLMs, the prompt is actually a really crucial place to express this taste and as such might be thought of as your ‘moat’. I know just from having experienced a variety of LLM-driven applications, it’s quite easy to tell the ones where the product team gave their prompts and their specification some real love. It’s the difference between POC junk that will die a slow and lonely death and something that delights and solves real user problems. Gulf of Generalisation Shreya didn’t really get into the details around the generalisation gulf in practical terms in this lesson, but I think this one can be a sort of place of comfort for the technically-minded to make refuge in. It’s one where there’s a ton of tools and technologies and techniques to play with, and vendors also live in this space and try to claim that their particular product or special sauce is the thing to help you and so on. The Improvement Loop for LLM Applications We also got a high-level overview of the loop that allows you to iteratively improve an LLM application: The analyse, measure and improve loop; adapted from an image used in the course There’s a lot to unpack in all these different stages, and we didn’t really get into the details in the session today but you can see how this offers a really powerful way of thinking through what it means to iteratively improve an LLM application. Learning how to implement this in a practical way will be the main thing I want to get good at by the end of this course. The process is made up of a bunch of techniques, but in my experience companies or use cases that struggle with improving what they built also lack the scaffold of this loop to orient themselves. Prompting through the lens of evals As we explored above, prompting is sort of the table stakes of improving your LLM application. In order to get good at prompting, it can help to appreciate what they are good at and what they struggle with. So, as Shreya put it, “leverage their strengths and anticipate their weaknesses” (when prompting). At this point Shreya got into some points around what kinds of things went into a good prompt but I think I’ll write a separate blog on that and I don’t want to just regurgitate what we listened to. Today was more of a high-level introduction, and in any case it was much more about the outer-loop process instead of the inner loop (where tooling + specific techniques play more of a role.) A slide from a talk I gave about the inner loop vs the outer loop of GenAI development So it’s great that the course gets into the weeds (esp in the course materials, which include the draft of the book Hamel & Shreya are writing) but I think the really useful thing they’re doing is situating the tactical improvements and techniques within the strategic patterns and workflows that teams and individuals should be doing to work on these LLM applications. At a high level, what are we talking about: how to tease out failure scenarios for these applications and their behaviours conversely, how to understand exactly which domains it does well for Things I want to think about more There was a ton of really rich discussion around prompting in the Discord. I’m interested in exploring more: cross-provider prompting decisions (i.e. how prompting an OpenAI model differs from what you do with a Llama model or whatever) prompts that work with reasoning models vs non-reasoning models the tradeoffs of whether you put your instructions in system prompts vs user instructions In general there’s been a bunch of noise recently about so-called ‘leaked’ system prompts from a bunch of LLM API providers and I’ve mainly been struck by just how detailed they are. I consider myself pretty good at improving and iterating on prompts, but I’ll admit I’m not writing these multi-thousand word tomes. I’d like to explore which scenarios it makes sense to do so, and how to calculate at what point it makes sense from a cost or latency perspective to do so. As I’m sure you can detect, I’m really enthusiastic about the lesson to come and will work in the meanwhile on some of the readings that have been set as well as the homework task of writing a system prompt for a LLM-powered recipe recommendation application!

19th May 2025 • 1 votes
First impressions of the new Gemini Deep Research (with 2.5 Pro)

Google released an updated iteration of their Deep Research tool that uses the new 2.5 Pro model. This was taken from a post originally made on Twitter, so please excuse the terseness. First impressions: a bit too eager to jump into a deep research task even when I just ask a clarifying question quite verbose, just like the OpenAI version. Not sure why both play this up a lot. It looks impressive but in practice I think we need more entry points into this. The ‘Executive Summary’ and other concluding headers are nice touches but I feel maybe there should be some more adherence to user requests for short reports. (I get that as UI it’s maybe weird to think for 10 mins and then spit out a very concise version, but it might actually be more useful.) I continue to be annoyed about how these Gemini DR reports handle footnotes (i.e. as endnotes whacked on at the end of the report). Almost a deal-breaker IMO. It’s almost like GDR tries to show how scholarly and serious it is by giving you these walls of prose (vs OpenAI DR which throws in a lot more bullet points). Not sure one is better than the other but would appreciate a bit more flexibility! The portability of these reports has always been not great. Yes you can export them to Google Docs but markdown (+ other options) would have been much better. In practice, this means that whenever I use GDR the report stays stuck there and I’m far less likely to share it with anyone, whereas the OpenAI DR reports I drop parts/all into a Github Gist etc. These reports have been getting better and better, all things considered. I’ve been following along and using GDR from the early days (even pre-OpenAI DR) and this latest version is the best version of it so far (as you’d hope!) (It’s also a little bit annoying that GDR has removed any way to use the older versions of GDR with Gemini Pro 2.0 and 1.5 etc. Makes it harder to actually compare these things.) Please let’s get an ipad version of the Gemini iPad app soon, too? Feels a bit regressive to have to use GDR on the web interface always. For serious research (as opposed to simply generating a nice report on some area where you don’t know much about already), all these tools remain hamstrung by the quality of the sources. In areas where I am (or very recently used to be) a leading scholar / researcher, the difference between what I’d expect (in terms of taste / discernment for picking out these sources) is especially egregious. Make the models better, yes, but have better filters + retrieval. So yeah, these tools are getting good! Kudos to the teams who are implementing this stuff. Hard to make it perform reproducibly well on so many open-ended uses. But more work to be done! IMO the really great implementations of this ‘deep research’ pattern will all be in-house where you can have control over: source selection (i.e. high-quality inputs only, not just some random things on the internet) how long it spends thinking about a particular area / loop of the research (or decides to backtrack and dig deeper etc) output types / templates / length different modalities of Q&A (sometimes you want reports, other times you want a quick question answered, other times you want visual guides etc etc.) different models for different kinds of tasks possibly you have little sub-research agents / processes which will go off and work on some hypothesis, possibly involving actual datasets / analysis of tabular data etc, something clearly missing from the current versions we have A few other things: GDR’s ‘clarification step’ (which I’ve heard them discuss on podcasts etc) is not as good or useful as the OpenAI DR clarification questions. In practice, because it’s buried under a concealment button that you have to click etc, and where the entire UI seems to be screaming at you to ‘Start Research’, you basically never update or amend the research plan. And when you do, it’s really not clear what’s changed because you don’t get some feedback or diff that your comments were understood; you just get an entire new research plan (again buried under the concealment button) Going forward we’re probably going to want / need ways of navigating the layers to this research. A global overview report will have subsections that (should you wish) can be expanded into their own more detailed or granular reports. This is how research works, after all. Not just endless new reports all trailed one after another pointed in the same direction. The other thing that I think we’re really going to need to work on is research taste. Like the LLMs that power them, GDR and OpenAI DR offer a level of research taste developed to the mean. (I know people are thinking about this since it came up on Dwarkesh’s podcast with the AI 2027 guys, but they were focused on scientific research.) I think there’s not a single answer for this which is, again, why I see the end result as people bringing these things in-house where they get to develop and refine what makes their particular flavour of research unique. (In the human-generated research world this is very much the case, where certain institutions (or even particular authors) are known for how deep they go, or what kinds of sources they prefer, or how they choose to feature or highlight the primary sources they access, and so on.) There are many possible variations of how this manifest, and I hope that we’re headed into a world where all the AI ‘deep researchers’ will be unique and quirky in all the best senses of that word.

8th Apr 2025 • 1 votes
Dataset Engineering: The Art and Science of Data Preparation

Finally back on track and reading the next chapter of Chip Huyen’s book, ‘AI Engineering’. Here are my notes on the chapter. Overview and Core Philosophy “Data will be mostly just toil, tears and sweat.” This is how we start the chapter :) This candid assessment frames dataset engineering as a discipline that requires both technical sophistication and pragmatic persistence. While the chapter’s placement might have been suitable earlier in the book, its position allows it to build effectively on previously established concepts. Data Curation: The Foundation Data curation addresses various use cases including fine-tuning, pre-training, and training from scratch, with specific considerations for chain of thought reasoning and tool use. The process addresses three fundamental aspects: Data Quality: The equivalent of ingredient quality in cooking Data Coverage: Analogous to having the right mix of ingredients Data Quantity: Determining the optimal volume of ingredients Quality Criteria Data quality encompasses multiple dimensions: Relevance to task requirements Consistency in format and structure Sufficient uniqueness Regulatory compliance (especially critical in regulated industries) Coverage Considerations Coverage involves strategic decisions about data proportions: Large language models often utilize significant code data (up to 50%) in training, which appears to enhance logical reasoning capabilities beyond just coding Language distribution can be surprisingly efficient (even 1% representation of a language can enable meaningful capabilities) Training proportions may vary across different stages of the training process Quantity and Optimization A key phenomenon discussed is ossification, where extensive pre-training can effectively freeze model weights, potentially hampering fine-tuning adaptability. This effect is particularly pronounced in smaller models. Key quantity considerations include: Task complexity correlation with data requirements Base model performance implications Model size considerations (OpenAI notes that with ~100 examples, more advanced models show superior fine-tuning performance) Potential for using lower quality or less relevant data for initial fine-tuning to reduce high-quality data requirements Recognition of performance plateaus where additional data yields diminishing returns Data Acquisition Process The chapter provides a detailed example workflow for creating an instruction-response dataset: Initial dataset identification (~10,000 examples) Low-quality instruction removal (reducing to ~9,000) Low-quality response filtering (removing 3,000) Manual response writing for remaining high-quality instructions Topic gap identification and template creation (100 templates) AI synthesis of 2,000 new instructions Manual annotation of synthetic instructions Final result: 11,000 high-quality examples Data Augmentation and Synthesis Synthesis Objectives Increasing data quantity Expanding coverage Enhancing quality Addressing privacy concerns Enabling model distillation Notable Research: An Anthropic paper (2022) found that language model-generated datasets can match or exceed human-written ones in quality for certain tasks. Note that some teams actually prefer AI-generated preference data due to human fatigue and inconsistency factors. Synthesis Applications The chapter distinguishes between pre-training and post-training synthesis: Synthetic data appears more frequently in post-training Pre-training limitation: AI can reshape existing knowledge but struggles to synthesize new knowledge LLaMA 3 Synthesis Pipeline A comprehensive workflow example: AI generation of problem descriptions Solution generation in multiple programming languages Unit test generation Error correction Cross-language translation with test verification Conversation and documentation generation with back-translation verification This pipeline generated 2.7 million synthetic coding examples for LLaMA 3.1’s supervised fine-tuning. Model Collapse Considerations The chapter addresses the risk of model collapse in synthetic data usage: Potential loss of training signal through repeated synthetic data use Current research suggests proper implementation can avoid collapse Importance of quality control in synthetic data generation Model Distillation Notable example: BuzzFeed’s fine-tuning of Flan T5 using LoRa and OpenAI’s text-davinci-003 generated examples, achieving 80% inference cost reduction. Data Processing Best Practices Expert Tip: “Manual inspection of data has probably the highest value to prestige ratio of any activity in machine learning.” - Greg Brockman, OpenAI co-founder Processing Guidelines The chapter emphasizes efficiency optimization: Order optimization (e.g., deduplication before cleaning if computationally advantageous) Trial run validation before full dataset processing Data preservation (avoid in-place modifications) Original data retention for: Alternative processing needs Team requirements Error recovery Technical Processing Approaches Deduplication strategies include: Pairwise comparison Hashing methods Dimensionality reduction techniques Multiple libraries are referenced (page 400) for implementation. Data Cleaning and Formatting HTML tag removal for signal enhancement Careful prompt template formatting, crucial for: Fine-tuning operations Instruction tuning Model performance optimization Data Inspection The chapter emphasizes the importance of manual data inspection: Utilize various data exploration tools Dedicate time to direct data examination (recommended: 15 minutes of direct observation) Consider this step non-optional in the process

4th Feb 2025 • 1 votes

More in AI

20 years of blogging

20 years ago to this day, I started my new technical blog. I'd had a personal one for some time, but most of the stuff that happened there had already migrated to social media. I wanted somewhere to post the things that would only be of interest to my techier friends, and maybe of some use to anyone else that came across it. Things have changed a bit, but they've stayed the same in some ways. Earlier this year I finished my 48-part (!) LLM from scratch series, but it had predecessors -- the third post I made here was part one of a 13-part series where I worked through setting up automated backups for my homelab NAS, and then later on I spent a few months building a reverse proxy in C, not to mention the eight-month, ten-post series on fine-tuning LLMs. It does seem like there's some value to putting all of this online. I rarely check my logs and referrers, but people do still seem to be arriving here having searched for stuff that I've written about, and every now and then there's a kind "thanks for this!" comment left on an old post where I explained something I'd just learned. I wrote a bit more about the blog's history last year, and came to the conclusion that the best posts -- both for me to write, and as far as I could tell for other people to read -- were the ones where I wrote up a project that I'd been working on. I blogged about that in more detail, and it seemed to strike a chord on Hacker News; I think it still holds. Recently, Sean Goedecke suggested that the best blog posts are those where you write about things that you're in the process of learning, and I agree. You learn more than you would if you didn't write it up, and because things are fresh in your mind, you may well write more clearly about it than an actual expert in the subject might have done. Anyway, I'm going to carry on blogging. I'm working on my biggest LLM-related project so far (dozens of training runs, so my study aircon is getting quite the workout even with the weather turning colder), and look forward to posting about it later on this month. I hope it's at least as fun to read as it is to write!

5 hours ago • 1 votes
The missing quadrant in responses to AI

Announcing STABL: Societal Transformation and AI Bottlenecks Lab

9 hours ago • 1 votes
Fable 5.2 and Jagged AGI

General, often expert, and sometimes jagged

2 days ago • 1 votes
BAIR at CoRL 2026

The Conference on Robot Learning (CoRL) takes place November 9–12, 2026, in Austin, Texas. BAIR faculty, postdocs and students will be presenting research at various sessions, including an Early Career Keynote from Professor Negar Mehr. See below for the full schedule of sessions with BAIR members, including papers co-authored with collaborators at other universities and companies. h2 { border-bottom: 2px solid #003262; padding-bottom: 0.3rem; margin-bottom: 0.2rem; } .corl-day-sub { font-size: 0.8rem; color: #666; margin: 0 0 1em; } .corl-day-sub span + span::before { content: "·"; margin: 0 0.5em; color: #aaa; } .corl-session { display: grid; grid-template-columns: 7.5rem 1fr; gap: 1rem; padding: 1rem 0; border-top: 1px solid #eee; } .corl-session:first-of-type { border-top: 0; } .corl-session.corl-featured { background: #fffbf0; border-left: 3px solid #FDB515; padding-left: 0.8rem; } .corl-time { font-weight: 700; font-size: 0.9rem; line-height: 1.3; } .corl-time small { display: block; font-weight: 400; font-size: 0.7rem; color: #777; text-transform: uppercase; letter-spacing: 0.03em; } .corl-tag { display: inline-block; font-size: 0.7rem; font-weight: 700; text-transform: uppercase; letter-spacing: 0.04em; padding: 0.1rem 0.45rem; border-radius: 3px; background: #eef2f7; color: #003262; } .corl-tag.k-keynote { background: #FDB515; color: #1a1a1a; } .corl-tag.k-oral { background: #003262; color: #fff; } .corl-tag.k-spotlight { background: #3b7ea1; color: #fff; } .corl-tag.k-poster { background: #eef2f7; color: #003262; } .corl-tag.k-lbr { background: #e7f3ec; color: #145a32; } .corl-tag.k-workshop { background: #f1ecf7; color: #4a2a75; } .corl-session h3 { margin: 0.35rem 0 0.15rem; font-size: 1.05rem; } .corl-venue { font-size: 0.8rem; color: #666; margin: 0 0 0.6em; } .corl-chair { font-weight: 700; color: #333; } .corl-paper, .corl-appearance { margin: 0 0 0.9em; } .corl-paper h4, .corl-appearance h4 { font-size: 0.92rem; margin: 0 0 0.2em; line-height: 1.35; font-family: 'Lato', 'Helvetica Neue', Helvetica, sans-serif; font-weight: 700; } .corl-board { display: inline-block; font-size: 0.72rem; color: #555; border: 1px solid #ddd; border-radius: 3px; padding: 0 0.35rem; margin-bottom: 0.25em; } .corl-authors { font-size: 0.82rem; margin: 0 0 0.2em; line-height: 1.45; } .corl-also, .corl-sources, .corl-program, .corl-see { font-size: 0.75rem; color: #666; margin: 0 0 0.15em; } .corl-see { font-family: 'Lato', 'Helvetica Neue', Helvetica, sans-serif; font-style: italic; } .corl-talk { font-style: italic; margin: 0 0 0.2em; } .corl-appearance p { font-size: 0.85rem; margin: 0 0 0.2em; } .corl-organizers { margin: 0; padding-left: 1.1rem; } .corl-organizers li { margin-bottom: 0.5em; font-size: 0.88rem; } .corl-organizers p { margin: 0; font-size: 0.82rem; color: #444; } .corl-about { margin-top: 2.5em; border-top: 2px solid #003262; padding-top: 0.5em; font-size: 0.85rem; } .corl-about-links { font-family: 'Lato', 'Helvetica Neue', Helvetica, sans-serif; display: flex; flex-wrap: wrap; gap: 0.4rem 1.2rem; } @media (max-width: 40em) { .corl-stats { grid-template-columns: repeat(2, 1fr); } .corl-session { grid-template-columns: 1fr; gap: 0.3rem; } .corl-time small { display: inline; margin-left: 0.4em; } .corl-time { font-size: 1rem; } .corl-venue, .corl-also, .corl-sources, .corl-program, .corl-board, .corl-tag, .corl-time small, .corl-day-sub, .corl-stat small, .corl-nav-note, .corl-see { font-size: 0.8rem; } .corl-authors { font-size: 0.9rem; } } @media print { .corl-tools, .corl-status, .corl-nav, .corl-empty { display: none !important; } } November 9–12, 2026Austin, TexasAll times CST · UTC−6 33Main-conference papers 4Late-breaking results (LBR) posters 10Orals & spotlights3 orals · 7 spotlights 4Days of researchNovember 9–12 MonNov 9 TueNov 10 WedNov 11 ThuNov 12 Main conference Nov 9–11 · Workshops Nov 12 Times shown for oral and spotlight papers are session windows. Individual presentation order is not specified. Papers appear in both their talk and poster sessions. Paper inclusion: At least one author is affiliated with UC Berkeley; joint affiliations qualify. Faculty talks, session-chair roles and workshop participation are also included. Late-breaking results: Four late-breaking posters with Berkeley authors are listed so far. This list may grow as more late-breaking posters are confirmed. Find a paper or person Session typeAll sessionsKeynoteOralsSpotlightsPostersLate-breaking resultsWorkshops Reset Showing all sessionsUpdated November 8, 2026 No matching sessions. Try a surname, a shorter paper title, or reset the filters. Monday, November 9 Main conference10 papers + 3 LBR posters 9:00–9:30 AMCST Spotlight · S1 Generalist policies, memory and transfer Hilton Austin MEM: Multi-Scale Embodied Memory for Vision Language Action Models Marcel Torne, Karl Pertsch, Homer Walke, Kyle Vedder, Suraj Nair, Brian Ichter, Allen Z. Ren, Haohuan Wang, Jiaming Tang, Kyle Stachowicz, Karan Dhabalia, Michael Robert Equi, Quan Vuong, Jost Tobias Springenberg, Sergey Levine, Chelsea Finn, Danny Driess Also at Poster Session I · 5:00–6:30 PM · Poster 42 · JW Marriott Paper ↗ Diagnosing Compositional Generalization in Sequential Robot Tasks Yixiao Wang, Cheng-En Wu, Lingfeng Sun, Pengcheng Wang, Xiang Ji, Boyuan Liang, Guojian Zhan, Masayoshi Tomizuka Also at Poster Session I · 5:00–6:30 PM · Poster 115 · JW Marriott Paper ↗ 12:00–1:30 PMCST · LUNCH BREAK Late-breaking results · Posters Agentic Robotics Griffin · Level 2 · JW Marriott Austin Griffin board 119 · LBR #15 Scaling Online Exploration for Robot Agents Runhan Huang, Xiangcheng Zhang, Haoyu Chen, Yitong Li, Yen-Jen Wang, Bohan Lyu, Haotian Ye, Chuye Hong, Yixuan Wang, Chloe Su, Haoyu Xiong, Yangzhen Wu, Enze Xie, Travis Manderson, Stefano Ermon, Pieter Abbeel, Antonio Torralba, Dawn Song, Weirui Ye, Yilun Du Griffin board 120 · LBR #19 Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents Yen-Jen Wang, Haozhe Jiang, Shuying Deng, Haoru Xue, Weirui Ye, Rocky Duan, Nika Haghtalab, S. Shankar Sastry, Pieter Abbeel, Haozhi Qi Also at Agentic Robotics workshop · Thursday, November 12 · 4:15–5:55 PM Griffin board 131 · LBR #54 Flip Anything with Agentic Robotics Self-Learning Shuangyu Xie, Kaiyuan Chen, Ken Goldberg GitHub ↗ Late-breaking results program ↗ 2:00–2:30 PMCST Keynote Early Career Keynote II Hilton Austin Session chair: Ken Goldberg Professor Negar Mehr How Can Robots Learn to Interact and Coordinate with Others? Session details ↗ 5:00–6:30 PMCST Posters Poster Session I JW Marriott Poster 42 MEM: Multi-Scale Embodied Memory for Vision Language Action Models Marcel Torne, Karl Pertsch, Homer Walke, Kyle Vedder, Suraj Nair, Brian Ichter, Allen Z. Ren, Haohuan Wang, Jiaming Tang, Kyle Stachowicz, Karan Dhabalia, Michael Robert Equi, Quan Vuong, Jost Tobias Springenberg, Sergey Levine, Chelsea Finn, Danny Driess Authors listed under Monday, Spotlight S1 Paper ↗ Poster 57 Large Video Planner Enables Generalizable Robot Control Boyuan Chen, Tianyuan Zhang, Haoran Geng, Caiyi Zhang, Peihao Li, Kiwhan Song, William T. Freeman, Jitendra Malik, Pieter Abbeel, Russ Tedrake, Vincent Sitzmann, Yilun Du Paper ↗ Poster 68 How to Peel with a Knife: Aligning Fine-Grained Manipulation with Human Preference Toru Lin, Shuying Deng, Zhao-Heng Yin, Pieter Abbeel, Jitendra Malik Paper ↗ Poster 97 HiRE: Hindsight Reward Editing for Policy Finetuning Haoyi Niu, Zhengtao Han, Yufeng Ji, Zhongyu Li, Koushil Sreenath Paper ↗ Poster 113 Scalable Behavior Cloning with Open Data, Training, and Evaluation Arthur Allshire, Himanshu Gaurav Singh, Ritvik Singh, Adam Rashid, Hongsuk Choi, David McAllister, Justin Yu, Yiyuan Chen, Huang Huang, Pieter Abbeel, Xi Chen, Rocky Duan, Phillip Isola, Jitendra Malik, Fred Shentu, Guanya Shi, Philipp Wu, Angjoo Kanazawa Paper ↗ Poster 115 Diagnosing Compositional Generalization in Sequential Robot Tasks Yixiao Wang, Cheng-En Wu, Lingfeng Sun, Pengcheng Wang, Xiang Ji, Boyuan Liang, Guojian Zhan, Masayoshi Tomizuka Authors listed under Monday, Spotlight S1 Paper ↗ Poster 142 AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edge Noriaki Hirose, Catherine Glossop, Dhruv Shah, Sergey Levine Paper ↗ Poster 159 GLST: Global-Local Spatio-Temporal Memory Decomposition Framework for Online Chained-Goal Navigation Yousef Liang, Ryan Ardalan, Yarden Goraly, Chenhao Zhu, Jonas Frey, Claire Tomlin, Marco Pavone, S. Shankar Sastry Paper ↗ Poster 202 ENPIRE: Agentic Robot Policy Self-Improvement in the Real World Wenli Xiao, Jia Xie, Tonghe Zhang, Haotian Lin, Letian Fu, Haoru Xue, Jalen Lu, Yi Yang, Cunxi Dai, Zi Wang, Jimmy Wu, Guanzhi Wang, S. Shankar Sastry, Ken Goldberg, Linxi Fan, Yuke Zhu, Guanya Shi Paper ↗ Poster 225 Energy-Based Policy Orchestration Weirui Ye, Yen-Jen Wang, Elvis Hsieh, Zhiyang Dou, Shibo Zhao, Pieter Abbeel, Yilun Du, Leslie Pack Kaelbling, Tomás Lozano-Pérez Paper ↗ Tuesday, November 10 Main conference11 papers + 1 LBR poster 9:00–9:30 AMCST Oral · O4 Reinforcement learning and motion reasoning Hilton Austin Flow Policy Gradients for Robot Control Brent Yi, Hongsuk Choi, Himanshu Gaurav Singh, Xiaoyu Huang, Takara E. Truong, Carmelo Sferrazza, Yi Ma, Rocky Duan, Pieter Abbeel, Guanya Shi, Karen Liu, Angjoo Kanazawa Also at Poster Session II · 5:00–6:30 PM · Poster 63 · JW Marriott Paper ↗ 10:30–11:00 AMCST Oral · O5 Dexterous learning, grasping and assembly Hilton Austin EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data Ruijie Zheng, Dantong Niu, Yuqi Xie, Jing Wang, Mengda Xu, Yunfan Jiang, Fernando Castañeda, Fengyuan Hu, You Liang Tan, Letian Fu, Trevor Darrell, Furong Huang, Yuke Zhu, Danfei Xu, Linxi Fan Also at Poster Session II · 5:00–6:30 PM · Poster 54 · JW Marriott Paper ↗ 11:30 AM–12:00 PMCST Oral · O7 Humanoid data, control and skills Hilton Austin Session chair: Koushil Sreenath SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model Qiayuan Liao, Yiyang Shao, Junfeng Long, Kevin Zakka, Roberto Horowitz, Pieter Abbeel, Koushil Sreenath Also at Poster Session II · 5:00–6:30 PM · Poster 217 · JW Marriott Paper ↗ 12:00–1:30 PMCST · LUNCH BREAK Late-breaking results · Posters Real2Sim2Real Griffin · Level 2 · JW Marriott Austin Griffin board 122 · LBR #26 RoboSimStudio: An Interactive Toolkit for Rapid Real2Sim of Robot Manipulation Tasks Kaiyuan Chen, Shuangyu Xie, Uksang Yoo, Xinyue Zhu, Cael Magner, Alvin Zhu, Daehwa Kim, Eric Heiden, Erwin Coumans, Miles Macklin, Spencer Huang, Ken Goldberg Late-breaking results program ↗ 5:00–6:30 PMCST Posters Poster Session II JW Marriott Poster 7 DiscreteRTC: Discrete Diffusion Policies are Natural Asynchronous Executors Pengcheng Wang, Kaiwen Hong, Chensheng Peng, Katherine Rose Driggs-Campbell, Masayoshi Tomizuka, Chenfeng Xu, Chen Tang Paper ↗ Poster 39 Improving Robotic Generalist Policies via Flow Reversal Steering Andy Tang, William Chen, Andrew Wagenmaker, Chelsea Finn, Sergey Levine Paper ↗ Poster 54 EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data Ruijie Zheng, Dantong Niu, Yuqi Xie, Jing Wang, Mengda Xu, Yunfan Jiang, Fernando Castañeda, Fengyuan Hu, You Liang Tan, Letian Fu, Trevor Darrell, Furong Huang, Yuke Zhu, Danfei Xu, Linxi Fan Authors listed under Tuesday, Oral O5 Paper ↗ Poster 63 Flow Policy Gradients for Robot Control Brent Yi, Hongsuk Choi, Himanshu Gaurav Singh, Xiaoyu Huang, Takara E. Truong, Carmelo Sferrazza, Yi Ma, Rocky Duan, Pieter Abbeel, Guanya Shi, Karen Liu, Angjoo Kanazawa Authors listed under Tuesday, Oral O4 Paper ↗ Poster 77 Why Does Action Chunking Improve Behavioral Cloning Performance in Robotic Control? Filippo Lazzati, Kyle Stachowicz, William Chen, Alberto Maria Metelli, Andrew Wagenmaker, Sergey Levine Paper ↗ Poster 142 FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation Chengbo Yuan, Zicheng Zhang, Mingjie Zhou, Wendi Chen, Yi Wang, Zhuoyang Liu, Dantong Niu, Shuo Wang, Hui Zhang, Wenkang Zhang, Yingdong Hu, Yuanqing Gong, Wanli Xing, Chuan Wen, Cewu Lu, Kaifeng Zhang, Yang Gao Paper ↗ Poster 153 PTLD: Sim-to-Real Privileged Tactile Latent Distillation for Dexterous Manipulation Rosy Chen, Mustafa Mukadam, Michael Kaess, Tingfan Wu, Francois Robert Hogan, Jitendra Malik, Akash Sharma Paper ↗ Poster 184 Counterfactual Video Generation Enables Scalable Humanoid Loco-Manipulation Zihan Wang, Zhen Wu, Pieter Abbeel, Rocky Duan, Jitendra Malik, Carmelo Sferrazza, Karen Liu, Guanya Shi, Angjoo Kanazawa Paper ↗ Poster 189 VLK: Learning Humanoid Loco-Manipulation from Synthetic Interactions in Reconstructed Scenes Yen-Jen Wang, Jiaman Li, Sirui Chen, Takara E. Truong, Pei Xu, Pieter Abbeel, Rocky Duan, Koushil Sreenath, Angjoo Kanazawa, Carmelo Sferrazza, Guanya Shi, Karen Liu Paper ↗ Poster 195 Social Humanoid: Socially Responsive Human–Humanoid Interaction Vongani Hlavutelo Maluleke, James Ni, Kie Horiuchi, Lea Wilken, Jiahui Lei, Evonne Ng, Roei Herzig, Jitendra Malik, Trevor Darrell, Angjoo Kanazawa Poster 217 SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model Qiayuan Liao, Yiyang Shao, Junfeng Long, Kevin Zakka, Roberto Horowitz, Pieter Abbeel, Koushil Sreenath Authors listed under Tuesday, Oral O7 Paper ↗ Wednesday, November 11 Main conference12 papers 8:30–9:00 AMCST Spotlight · S6 Optimization, agentic planning and evaluation Hilton Austin Session chair: Negar Mehr Playful Agentic Robot Learning Junyi Zhang, Jiaxin Ge, Hanjun Yoo, Letian Fu, Zihan Yang, Yaowei Liu, Raj Saravanan, Shaofeng Yin, Justin Yu, Dantong Niu, Zirui Wang, Roei Herzig, Ken Goldberg, Yutong Bai, David M. Chan, Ion Stoica, Angjoo Kanazawa, Jiahui Lei, Haiwen Feng, Trevor Darrell Also at Poster Session III · 10:30 AM–12:00 PM · Poster 99 · JW Marriott Paper ↗ GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks Kaiyuan Chen, Shuangyu Xie, Letian Fu, Justin Yu, William Pacini, Sandeep Bajamahal, Hudson Kim, Jaimyn Drake, Daehwa Kim, Haoru Xue, Jonathan Francis, Christian Juette, Peter Schaldenbrand, Muhammet Yunus Seker, Ruwan Wickramarachchi, Uksang Yoo, Guanzhi Wang, Adithyavairavan Murali, Balakumar Sundaralingam, S. Shankar Sastry, Yuke Zhu, Linxi Fan, Spencer Huang, Ken Goldberg Also at Poster Session III · 10:30 AM–12:00 PM · Poster 100 · JW Marriott Paper ↗ 9:00–9:30 AMCST Spotlight · S7 Humanoid locomotion and whole-body control Hilton Austin LadderMan: Learning Humanoid Perceptive Ladder Climbing Siheng Zhao, Yuanhang Zhang, Ziqi Lu, Pieter Abbeel, Rocky Duan, Koushil Sreenath, Yue Wang, Karen Liu, Guanya Shi Also at Poster Session III · 10:30 AM–12:00 PM · Poster 195 · JW Marriott Paper ↗ ResSafe: Learning Safety Filtering with Residual Reinforcement Learning for Humanoids Gechen Qu, Tong Zhang, Bike Zhang, Yen-Jen Wang, Koushil Sreenath, Claire Tomlin, Jason Jangho Choi Also at Poster Session III · 10:30 AM–12:00 PM · Poster 198 · JW Marriott Paper ↗ TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model Anqi Li, Yuxin Chen, Zhaobo Li, Zhuo Cao, Junli Ren, Masayoshi Tomizuka, Dhruv Shah Also at Poster Session III · 10:30 AM–12:00 PM · Poster 200 · JW Marriott Paper ↗ 10:30 AM–12:00 PMCST Posters Poster Session III JW Marriott Poster 51 Do as I Do: Dexterous Manipulation Data from Everyday Human Videos Bhawna Paliwal, Haritheja Etukuru, William Liang, Pieter Abbeel, Nur Muhammad Mahi Shafiullah, Jitendra Malik Paper ↗ Poster 68 WARP-RM: A Warp-Augmented Relative Progress Reward Model for Data Curation Justin Yu, Andrew Goldberg, Kavish Kondap, Karim El-Refai, Ethan Ransing, Qianzhong Chen, Mac Schwager, Fred Shentu, Philipp Wu, Ken Goldberg Paper ↗ Poster 99 Playful Agentic Robot Learning Junyi Zhang, Jiaxin Ge, Hanjun Yoo, Letian Fu, Zihan Yang, Yaowei Liu, Raj Saravanan, Shaofeng Yin, Justin Yu, Dantong Niu, Zirui Wang, Roei Herzig, Ken Goldberg, Yutong Bai, David M. Chan, Ion Stoica, Angjoo Kanazawa, Jiahui Lei, Haiwen Feng, Trevor Darrell Authors listed under Wednesday, Spotlight S6 Paper ↗ Poster 100 GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks Kaiyuan Chen, Shuangyu Xie, Letian Fu, Justin Yu, William Pacini, Sandeep Bajamahal, Hudson Kim, Jaimyn Drake, Daehwa Kim, Haoru Xue, Jonathan Francis, Christian Juette, Peter Schaldenbrand, Muhammet Yunus Seker, Ruwan Wickramarachchi, Uksang Yoo, Guanzhi Wang, Adithyavairavan Murali, Balakumar Sundaralingam, S. Shankar Sastry, Yuke Zhu, Linxi Fan, Spencer Huang, Ken Goldberg Authors listed under Wednesday, Spotlight S6 Paper ↗ Poster 135 T-Rex: Tactile-Reactive Dexterous Manipulation Dantong Niu, Zhuoyang Liu, Zekai Wang, Boning Shao, Zhao-Heng Yin, Anirudh Pai, Yuvan Sharma, Stefano Saravalle, Ruijie Zheng, Jing Wang, Ryan Punamiya, Mengda Xu, Yuqi Xie, Yunfan Jiang, Letian Fu, Konstantinos Kallidromitis, Matteo Gioia, Junyi Zhang, Jiaxin Ge, Haiwen Feng, Fabio Galasso, Wei Zhan, David M. Chan, Yutong Bai, Roei Herzig, Jiahui Lei, Fei-Fei Li, Ken Goldberg, Jitendra Malik, Pieter Abbeel, Yuke Zhu, Danfei Xu, Linxi Fan, Trevor Darrell Paper ↗ Poster 148 Tactile Curiosity Drives Robot Interaction Klemens Iten, Alexander Proshkin, Bhavya Sukhija, Stelian Coros, Andreas Krause, Pieter Abbeel, Carmelo Sferrazza Paper ↗ Poster 159 RPL: Learning Robust Humanoid Perceptive Locomotion on Challenging Terrains Yuanhang Zhang, Younggyo Seo, Juyue Chen, Yifu Yuan, Koushil Sreenath, Pieter Abbeel, Carmelo Sferrazza, Karen Liu, Rocky Duan, Guanya Shi Paper ↗ Poster 175 TEXEDO: Test Time Scaling for Controller-aware Language-conditioned Humanoid Motion Generation Jianuo Cao, Yuxin Chen, Yuzhen Song, Masayoshi Tomizuka, Chenran Li, Thomas Tian Paper ↗ Poster 178 CLIFT: Turning Gemini Robotics On-Device into Humanoid Specialists via Non-Invasive Closed-Loop Iterative Fine-Tuning Yuxin Chen, Hari Srikanth, Nathan Jew, Menglin Wu, Pengcheng Wang, Junli Ren, Masayoshi Tomizuka, Peng Xu, Jinyu Xie, Thomas Tian Paper ↗ Poster 195 LadderMan: Learning Humanoid Perceptive Ladder Climbing Siheng Zhao, Yuanhang Zhang, Ziqi Lu, Pieter Abbeel, Rocky Duan, Koushil Sreenath, Yue Wang, Karen Liu, Guanya Shi Authors listed under Wednesday, Spotlight S7 Paper ↗ Poster 198 ResSafe: Learning Safety Filtering with Residual Reinforcement Learning for Humanoids Gechen Qu, Tong Zhang, Bike Zhang, Yen-Jen Wang, Koushil Sreenath, Claire Tomlin, Jason Jangho Choi Authors listed under Wednesday, Spotlight S7 Paper ↗ Poster 200 TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model Anqi Li, Yuxin Chen, Zhaobo Li, Zhuo Cao, Junli Ren, Masayoshi Tomizuka, Dhruv Shah Authors listed under Wednesday, Spotlight S7 Paper ↗ Thursday, November 12 WorkshopsTalks, papers and organizing roles 8:40–9:50 AMCST Workshop Roadmap to Sample-Efficient Real-World Reinforcement Learning (R2RL) · Keynote block JW Marriott · Room 213 Zhiyuan “Paul” Zhou Sergey Levine’s lab. Speaks during the keynote block. Session details ↗ 9:40–10:15 AMCST Workshop Bimanual Dexterous Tool Use · Spotlights JW Marriott · Rooms 303–304 T-Rex: Tactile-Reactive Dexterous Manipulation Dantong Niu, Zhuoyang Liu, Zekai Wang, Boning Shao, Zhao-Heng Yin, Anirudh Pai, Yuvan Sharma, Stefano Saravalle, Ruijie Zheng, Jing Wang, Ryan Punamiya, Mengda Xu, Yuqi Xie, Yunfan Jiang, Letian Fu, Konstantinos Kallidromitis, Matteo Gioia, Junyi Zhang, Jiaxin Ge, Haiwen Feng, Fabio Galasso, Wei Zhan, David M. Chan, Yutong Bai, Roei Herzig, Jiahui Lei, Fei-Fei Li, Ken Goldberg, Jitendra Malik, Pieter Abbeel, Yuke Zhu, Danfei Xu, Linxi Fan, Trevor Darrell Authors listed under Wednesday, Poster Session III Paper ↗ Do as I Do: Dexterous Manipulation Data from Everyday Human Videos Bhawna Paliwal, Haritheja Etukuru, William Liang, Pieter Abbeel, Nur Muhammad Mahi Shafiullah, Jitendra Malik Authors listed under Wednesday, Poster Session III Paper ↗ Workshop program ↗ 10:15–10:40 AMCST Workshop Bimanual Dexterous Tool Use · Posters & demos JW Marriott · Rooms 303–304 T-Rex: Tactile-Reactive Dexterous Manipulation Dantong Niu, Zhuoyang Liu, Zekai Wang, Boning Shao, Zhao-Heng Yin, Anirudh Pai, Yuvan Sharma, Stefano Saravalle, Ruijie Zheng, Jing Wang, Ryan Punamiya, Mengda Xu, Yuqi Xie, Yunfan Jiang, Letian Fu, Konstantinos Kallidromitis, Matteo Gioia, Junyi Zhang, Jiaxin Ge, Haiwen Feng, Fabio Galasso, Wei Zhan, David M. Chan, Yutong Bai, Roei Herzig, Jiahui Lei, Fei-Fei Li, Ken Goldberg, Jitendra Malik, Pieter Abbeel, Yuke Zhu, Danfei Xu, Linxi Fan, Trevor Darrell Authors listed under Wednesday, Poster Session III Paper ↗ Do as I Do: Dexterous Manipulation Data from Everyday Human Videos Bhawna Paliwal, Haritheja Etukuru, William Liang, Pieter Abbeel, Nur Muhammad Mahi Shafiullah, Jitendra Malik Authors listed under Wednesday, Poster Session III Paper ↗ Workshop program ↗ 10:35–11:05 AMCST Workshop Everything Beneath the Policy · Posters Hilton Austin · Salon E, 4th floor TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning Dayou Li, Hao Wang, Qianqian Yang, Zihao Zhu, Haoquan Fang, Ziyao Zeng, Yan Han, Zihan Wang, Yan Wang, Baoru Huang, Dilin Wang, Kenji Shimada, Yiyue Luo, Manling Li, Teresa Lv, Mustafa Mukadam, Rakesh Ranjan, Ruohan Zhang, Qi He, Changliu Liu, Xu Chen, Marco Pavone, Bangya Liu, Jiachen Li, Masayoshi Tomizuka, Zhiwen Fan Workhorse: Learning Robust Whole-Body Humanoid Loco-Manipulation from Human Data Songbo Hu, Qiayuan Liao, Yufeng Chi, Kevin Zakka, Sophia Shao, Pieter Abbeel, Koushil Sreenath Paper ↗ Workshop program ↗ 10:40–11:10 AMCST Workshop Bimanual Dexterous Tool Use · Invited talk JW Marriott · Rooms 303–304 Professor Jitendra Malik Talk title has not been announced. Session details ↗ 11:50 AM–12:20 PMCST Workshop R2RL · Panel discussion JW Marriott · Room 213 Zhiyuan “Paul” Zhou Panel with workshop speakers and audience volunteers. Session details ↗ 2:45–3:05 PMCST Workshop Agentic Robotics · Invited talk Room to be announced Professor Ken Goldberg Talk title has not been announced. Session details ↗ Approx. 2:55–3:25 PMCST Workshop Sim-to-Real-to-Field · Invited talks Hilton Austin · Salon E, 4th floor Professor Claire Tomlin Shared invited-talk block with Michael Lutter (Boston Dynamics). Session details ↗ 4:15–5:55 PMCST Workshop Agentic Robotics · Live demos & posters Room to be announced ASPIRE: Agentic /Skills Discovery for Robotics Runyu Lu, Yubo Wu, Ethan Kou, Haotian Lin, Letian Fu, Wenli Xiao, Ajay Mandlekar, Yinzhen Xu, Guanya Shi, Ken Goldberg, Ang Chen, Mosharaf Chowdhury, Jan Kautz, Yuke Zhu, Linxi Fan, Guanzhi Wang Agentic World Action Models Zhuoyang Liu, Dantong Niu, Botong Zhang, Boning Shao, Jing Wang, Ruijie Zheng, Yuvan Sharma, Runyu Lu, Jiaming Liu, Renrui Zhang, Roei Herzig, Yutong Bai, Ken Goldberg, Danfei Xu, Yuke Zhu, Guanzhi Wang, Linxi Fan, Trevor Darrell Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents Yen-Jen Wang, Haozhe Jiang, Shuying Deng, Haoru Xue, Weirui Ye, Rocky Duan, Nika Haghtalab, S. Shankar Sastry, Pieter Abbeel, Haozhi Qi Authors listed under Monday, Late-breaking results, Agentic Robotics Workshop program ↗ Additional involvement Workshop organizers BAIR behind the program Workshop organizing roles. Do Robots Need World Models? Lasse Peters, Negar Mehr’s lab — organizer. Sim-to-Real-to-Field S. Shankar Sastry, Bike Zhang, Siming He, Jonas Frey — organizers. “Oops, I Erred” Sergey Levine — faculty advisor. Zhiyuan “Paul” Zhou and Renhao Wang — organizers. Dexterous Manipulation: Full-Stack Robotics on Learning and Design Jitendra Malik — organizer. Workshop: 2:00–6:00 PM. About this schedule Schedule compiled from the conference program, paper and project pages, and workshop programs. Workshop schedules may change; consult each linked program for updates. Coverage includes 33 main-conference papers, 4 late-breaking results posters and the workshop contributions listed above. The main conference demo track is not covered. Main-conference poster numbers follow an unofficial paper index; check the official program on the day. Hilton stage content will also be streamed to viewing rooms at the Marriott. Paper affiliations reviewed November 6, 2026; late-breaking results updated November 8, 2026. Official CoRL program ↗BAIR faculty ↗Unofficial paper & poster index ↗Back to top ↑ 0; status.textContent = (!search.value && !type.value) ? "Showing all sessions" : visible + " matching session" + (visible === 1 ? "" : "s"); } function reset() { search.value = ""; type.value = ""; filter(); } search.addEventListener("input", filter); type.addEventListener("change", filter); document.getElementById("corl-reset").addEventListener("click", function () { reset(); search.focus(); }); Array.prototype.forEach.call(root.querySelectorAll(".corl-nav a, .corl-see a"), function (a) { a.addEventListener("click", reset); }); window.addEventListener("beforeprint", reset); })();

2 days ago • 1 votes
AI and Irontown

For our last movie night we watched Princess Mononoke. I remembered loving that movie when I was younger, but watching it as an adult made me once again realize how Miyazaki has a way of expressing diverging sides without picking one. The film shows people, animals and spirits with legitimate and largely incompatible needs, and each faction is further engaged in infighting. A central arc of the story is Lady Eboshi’s Irontown, which gives people who were largely rejected by society work, protection and dignity. She gives shelter and work to those who have leprosy and to former sex workers. The mine and the weapon factory provide them with a livelihood, but they also destroy the forest around them. The forest’s inhabitants, in particular the boars, response choose war against the humans to defend themselves. Eboshi is not portrayed as a villain. She does real good for the people of Irontown, but also is an arms manufacturer who is destroying the forest around it. The film anchors around a character called Ashitaka, who has been condemned to death by an unfortunate encounter with a god that was turned into a demon after being fatally wounded by one of Eboshi’s guns. He doesn’t take sides, but he wants to understand and find the truth. Another character is San (whom they call Mononoke), a girl who was raised by wolves and, despite being human, considers herself part of the forest. This movie’s backdrop is human progress, and it reminded me so much of the challenges we are facing today with AI. AI is useful, empowering and consequential beyond the people who benefit from it. It’s similar to the iron in the story. Irontown gives the outcast purpose and power. There are people today who see themselves in an environment just like that: people who were never let in before and who can now build things that were out of reach. They are wielding AI to their benefit. But there are negative externalities. In Mononoke it’s not just the spirits and animals in the forest who bear the consequences, it’s also other people. Irontown produces guns, and its people are engaged in fights with other humans. There is conflict and power seeking everywhere. Another, unseen character is the power-hungry emperor, who wants the head of the forest spirit, a god, because he believes it will make him immortal. Eboshi is willing to support that cause, not because she thinks it’s right, but because she is at war with the forest’s gods who are preventing her expansion, so they have a common interest. But there really is no need for anyone to kill the forest spirit, in the same way there is no need for any one country to “win at AI”. What drives it is the fear of what happens if someone else wins first. And the emperor is willing to have a god killed for it. The people who help him with that desire, Eboshi included, don’t understand the chaos it will unleash. There it also shows the dangers of seeking overwhelming power without comprehension of how to control its consequences. The pursuit of power itself becomes destructive. In the end, the gods die, and what grows back is not the old forest anymore. Irontown is destroyed. Eboshi wants to rebuild it, and to build a better town this time. Ashitaka stays with the people of Irontown after his curse is lifted by the dying forest spirit. San cannot forgive the humans and stays with the wolves. In the end, nothing is truly resolved, but there is a truce that came at great cost. Ashitaka’s role in the movie is one of a neutral actor. He was wronged both by a spirit turned demon in the forest, who put a curse on him, and, indirectly, by Eboshi, who caused the spirit to turn evil. Yet he harbors no ill will towards either side. He sets out to “see with eyes unclouded by hate,” and in the course of the movie he is seen as an unaligned actor. He helps people on both sides, and stands in their way to prevent further conflict. He doesn’t feel like a pacifist, but he’s the closest to one there is. He’s a truth seeker in a changing world. Because he engages he’s seen as suspicious by both sides, and ends up shot by the people of Irontown when he tries to rescue San. I use AI every day and find it genuinely empowering. But I also recognize the pain it causes some, and the moral problems it poses. It’s genuine progress for humanity, but it comes at significant cost. And many of those costs will be carried by people who aren’t going to benefit from it. So it’s no surprise that AI feels like a threat to many today. Communities currently strongly align themselves with one side and fight against the other. The forst spirit did not have to die and the boars did not have to die in a senseless war. The question is not whether there will be iron, but what kind of Irontown we build with it. I want our conversation about AI to make room for nuance and genuine understanding, for us to look at it and each other with eyes unclouded by hate. We should take advantage of its real benefits, but also recognize the damage it can do, and the choices we still have about what we build and what we’re willing to sacrifice for it. I like to think there is a path for people with differing opinions can find a way to live alongside before losing something we can’t get back.

2 days ago • 1 votes
📚 BoredReading

You seem to be enjoying this.

Join free to unlock everything.

Create free account

Already have an account? Sign in