Salto has been one of our favorite robots since we were first introduced to it in 2016 as a project out of Ron Fearing’s lab at UC Berkeley. The palm-sized spring-loaded jumping robot has gone from barely being able to chain together a few open-loop jumps to mastering landings, bouncing around outside, powering through obstacle courses, and occasionally exploding. What’s quite unusual about Salto is that it’s still an active research project—nine years is an astonishingly long life time for any robot, especially one without any immediately obvious practical applications. But one of Salto’s original creators, Justin Yim (who is now a professor at the University of Illinois), has found a niche where Salto might be able to do what no other robot can: mid-air sampling of the water geysering out of the frigid surface of Enceladus, a moon of Saturn. What makes Enceladus so interesting is that it’s completely covered in a 40 kilometer thick sheet of ice, and underneath that ice is a 10 km-deep global ocean. And within that ocean can be found—we know not what. Diving in that buried ocean is a problem that robots may be able to solve at some point, but in the near(er) term, Enceladus’ south pole is home to over a hundred cryovolcanoes that spew plumes of water vapor and all kinds of other stuff right out into space, offering a sampling opportunity to any robot that can get close enough for a sip. “We can cover large distances, we can get over obstacles, we don’t require an atmosphere, and we don’t pollute anything.” —Justin Yim, University of Illinois Yim, along with another Salto veteran Ethan Schaler (now at JPL), have been awarded funding through NASA’s Innovative Advanced Concepts (NIAC) program to turn Salto into a robot that can perform “Legged Exploration Across the Plume,” or in an only moderately strained backronym, LEAP. LEAP would be a space-ified version of Salto with a couple of major modifications allowing it to operate in a freezing, airless, low-gravity environment. Exploring Enceladus’ Challenging Terrain As best as we can make out from images taken during Cassini flybys, the surface of Enceladus is unfriendly to traditional rovers, covered in ridges and fissures, although we don’t have very much information on the exact properties of the terrain. There’s also essentially no atmosphere, meaning that you can’t fly using aerodynamics, and if you use rockets to fly instead, you run the risk of your exhaust contaminating any samples that you take. “This doesn’t leave us with a whole lot of options for getting around, but one that seems like it might be particularly suitable is jumping,” Yim tells us. “We can cover large distances, we can get over obstacles, we don’t require an atmosphere, and we don’t pollute anything.” And with Enceladus’ gravity being just 1/80th that of Earth, Salto’s meter-high jump on Earth would enable it to travel a hundred meters or so on Enceladus, taking samples as it soars through cryovolcano plumes. The current version of Salto does require an atmosphere, because it uses a pair of propellers as tiny thrusters to control yaw and roll. On LEAP, those thrusters would be replaced with an angled pair of reaction wheels instead. To deal with the terrain, the robot will also likely need a foot that can handle jumping from (and landing on) surfaces composed of granular ice particles. LEAP is designed to jump through Enceladus’ many plumes to collect samples, and use the moon’s terrain to direct subsequent jumps.NASA/Justin Yim While the vision is for LEAP to jump continuously, bouncing over the surface and through plumes in a controlled series of hops, sooner or later it’s going to have a bad landing, and the robot has to be prepared for that. “I think one of the biggest new technological developments is going to be multimodal locomotion,” explains Yim. “Specifically, we’d like to have a robust ability to handle falls.” The reaction wheels can help with this in two ways: they offer some protection by acting like a shell around the robot, and they can also operate as a regular pair of wheels, allowing the robot to roll around on the ground a little bit. “With some maneuvers that we’re experimenting with now, the reaction wheels might also be able to help the robot to pop itself back upright so that it can start jumping again after it falls over,” Yim says. A NIAC project like this is about as early-stage as it gets for something like LEAP, and an Enceladus mission is very far away as measured by almost every metric—space, time, funding, policy, you name it. Long term, the idea with LEAP is that it could be an add-on to a mission concept called the Enceladus Orbilander. This US $2.5 billion spacecraft would launch sometime in the 2030s, and spend about a dozen years getting to Saturn and entering orbit around Enceladus. After 1.5 years in orbit, the spacecraft would land on the surface, and spend a further 2 years looking for biosignatures. The Orbilander itself would be stationary, Yim explains, “so having this robotic mobility solution would be a great way to do expanded exploration of Enceladus, getting really long distance coverage to collect water samples from plumes on different areas of the surface.” LEAP has been funded through a nine-month Phase 1 study that begins this April. While the JPL team investigates ice-foot interactions and tries to figure out how to keep the robot from freezing to death, at the University of Illinois Yim will be upgrading Salto with self-righting capability. Honestly, it’s exciting to think that after so many years, Salto may have finally found an application where it offers the actual best solution for solving this particular problem of low-gravity mobility for science.

Frustrated scientists turned to visual aids to help make their case for the lightning rod. The exploding thunder house is one example. When a small amount of gunpowder was deposited inside the dollhouse-size structure and a charge was applied, the house would either explode or not, depending on whether it was ungrounded or grounded. [For more on thunder houses, see “Tiny Exploding Houses Promoted 18th-Century Lightning Rods,.” IEEE Spectrum, 1 April 2023.] Three Experimental Illustrations of a General Law of Electrical Discharge made the case for Harris’s invention: a lightning rod for tall-masted wooden ships. The rod was attached to the mainmast, ran through the hull, and connected to copper sheeting on the underside of the ship, thus dissipating any electricity from a lightning strike into the sea. It was a great idea, and it seemed to work. So why did the British Navy refuse to adopt it? I’ll get to that in a bit. How to Illustrate the Principles of Lightning The “experimental illustrations” in Harris’s 16-page pamphlet were intended to be interactive, each one highlighting a specific principle of conductivity. The illustrations were plated with gold leaf to mimic the conducting path of lightning. When the reader applied a charge to one end, the current charred a black course along the page. In the illustration at top, someone has clearly done this on the right hand side. In the first experimental illustration in Harris’s book, the gold leaf is scattered haphazardly across the page. Linda Hall Library of Science, Engineering & Technology The second experiment addresses a problem that was common in the days of tall ships: the rise and fall of the lightning rod as the jibs and rigging were adjusted according to the weather. Whereas a church steeple and its lightning rod remain fixed, a movable mast and the constantly changing rigging altered the configuration of the lightning rod. The experiment demonstrates that Harris’s design wasn’t affected by such changes. A charge wouldn’t dead-end and detonate midship just because a jib had been lowered. It would still follow the conductor that leads to the best exit for dissipation—that is, the ship’s bottom. The second experiment was intended to show, in a stylized way, the effect of the lightning rod rising and falling as the jibs and rigging were adjusted.Linda Hall Library of Science, Engineering & Technology The experiment illustrates what would happen if the sailor were to accidentally come in contact with two points of a loose conductive cable during a lightning storm. Instead of following the cable, the discharge would course straight through him. As Harris wrote in the description, the poor seaman “would be probably destroyed.” Death was a clear risk for sailors on unprotected ships, just as it was for bell ringers in unprotected churches. Mr. Thunder-and-Lightning Harris William Snow Harris published Three Experimental Illustrations when he was about 70, and he died six years later. The booklet was his final salvo in a battle he had waged with the Royal Navy for decades. William Snow Harris (1791–1867) trained as a medical doctor but gave up his practice to focus on promoting his lightning rod for wooden ships. Plymouth Athenaeum An 1823 book on the effects of lightning on ships also featured his gold-leafed experimental illustrations, along with a vivid description of a lightning strike on an unprotected ship: “The main-top mast, from head to heel, was shivered into a thousand splinters….” Harris enlisted support for his system from leading scientists, such as Michael Faraday, Charles Wheatstone, and Humphry Davy. He eventually earned the nickname Mr. Thunder-and-Lightning Harris for his zealotry. Harris continued to press his case. A well-publicized lightning strike on the U.S. packet ship New York in 1827 helped. Three days into its transatlantic journey, lightning struck at dawn. The “electrical fluid,” as it was then called, ran down the mainmast, bursting three iron hoops and shattering the masthead and cap. It entered a storeroom and demolished the bulkheads and fittings before following a lead pipe into the ladies’ cabin and fragmenting a large mirror. Elsewhere, it overturned a piano, split the dining table into pieces, and magnetized the ship’s chronometer as well as most of the men’s watches. New York again. As the American Journal of Science and Arts reported, the chain was “literally torn to pieces and scattered to the winds,” but it did its job and saved the ship, and no passengers were killed. Beagle, which was about to set sail for a surveying trip of South America. After it returned five years later, one of its passengers, Charles Darwin, published an account that made the voyage famous. (His 1859 book, On the Origin of Species, was also based on his research aboard the Beagle.) The HMS Beagle, made famous by Charles Darwin, was one of 11 British navy ships to be outfitted with Harris’s fixed lightning rods. Bettmann/Getty Images described a strike that he witnessed while on deck: “The mainmast, for the instant, appeared to be a mass of fire, I felt certain that the lightning had passed down the conductor on that mast.” Thetis, whose foremast had been destroyed by lightning, so he was especially attuned to the destruction storms could cause. Yet on the Beagle, he wrote, “not the slightest ill consequence was experienced.” When Captain Robert FitzRoy made his report to the admiralty, he likewise endorsed Harris’s system: “Were I allowed to choose between masts so fitted and the contrary, I should decide in favor of those having Harris’s conductors.” Numbers Don’t Lie Not to be defeated, Harris turned to statistics, compiling a list of 235 British naval vessels damaged by lightning, from the Abercromby (26 October 1811, topmast shivered into splinters 14 feet down) to the Zebra (27 March 1838, main-topgallant and topmast shivered; fell on the deck; main-cap split; the jib and sails on mainmast scorched). Additionally, he cataloged the deaths of nearly 100 seamen and serious injury of about 250 others. During one particularly bad period of five or six years, Harris learned, lightning destroyed 40 ships of the line, 20 frigates, and 10 sloops, disabling about one-eighth of the British navy. Rodney. Sensing an opportunity to make a public case for his system, Harris bypassed the admiralty and petitioned the House of Commons to review his claims. A Naval Commission appointed to do that wound up firmly supporting Harris. if they petitioned the admiralty. Given how openly hostile the admiralty was toward Harris, I’m guessing many captains didn’t do that. A Lightning Rod for Every British Warship Finally, in June 1842, the admiralty ordered the use of Harris’s lightning rods on all Royal Navy vessels. According to Theodore Bernstein and Terry S. Reynolds, who chronicled Harris’s battle in their 1978 article “Protecting the Royal Navy from Lightning: William Snow Harris and His Struggle with the British Admiralty for Fixed Lightning Conductors” in IEEE Transactions on Education, the navy’s change of heart wasn’t due to better data or more appeals by Harris and his backers. It mostly came down to politics. A second argument was financial. Harris’s system was significantly more expensive than a simple cable or chain. In one 1831 estimate, the cost of Harris’s system ranged from £102 for a 10-gun brig to £365 for a 120-gun brig, compared to less than £5 for the simple cable. Sure, Harris’s system was effective, but was it more than 20 times as effective? Of course, the simple cable offered no protection at all if it was never deployed, as many captains admitted to. John Barrow (1764–1848), second secretary to the Royal Navy Admiralty, was singularly effective at blocking the adoption of Harris’s lightning rod. National Portrait Gallery But the ultimate reason for the navy’s resistance, argued Bernstein and Reynolds, was political. In 1830, when Harris seemed on the verge of success, the Whigs gained control of Parliament. In the course of a few months, many of Harris’s government supporters found themselves powerless or outright fired. It wasn’t until late 1841, when the Tories regained power, that Harris’s fortunes reversed. John Barrow, second secretary to the admiralty, as the key person standing in Harris’s way. Political appointees came and went, but Barrow held his office for over 40 years, from 1804 to 1845. Barrow managed the navy’s budget, and he apparently considered Harris a charlatan who was trying to sell the navy an expensive and useless technology. He used his position to continually block it. One navy supporter of Harris’s system called Barrow “the most obstinate man living.” Harris eventually proved victorious. By 1850, every vessel in the Royal Navy was equipped with his lightning rod. But the victory was fleeting. By the start of the next decade, the first British ironclad ship had appeared, and by the end of the century, all new naval ships were made of metal. Metal ships naturally conduct lightning to the surrounding water. There was no longer a need for a lightning rod. Part of a continuing series looking at historical artifacts that embrace the boundless potential of technology. An abridged version of this article appears in the March 2025 print issue as “The Path of Most Resistance.” References Finch Collins, assistant curator of rare books at the Linda Hall Library, in Kansas City, Mo., introduced me to the books of William Snow Harris. You should have seen his face when I asked if we could apply a battery to one of the lightning experiments in the book. You can see the books in person by visiting the library. Or you can enjoy fully scanned copies of Observations on the Effects of Lightning on Floating Bodies and Three Experimental Illustrations from your computer. Theodore Bernstein of the University of Wisconsin–Madison and Terry S. Reynolds of Michigan Technological University wrote “Protecting the Royal Navy from Lightning—William Snow Harris and His Struggle with the British Admiralty for Fixed Lightning Conductors” for the February 1978 issue of IEEE Transactions on Education. Many thanks to my colleague Cary Mock, a climatologist at the University of South Carolina who has an interest in extreme weather events throughout history. He has done amazing work re-creating paths of hurricanes based on navy logbooks. Cary patiently answered my questions about lightning and wooden ships and pointed me to additional resources, such as this fabulous “Index of 19th Century Naval Vessels.”

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion. RoboCup German Open: 12–16 March 2025, NUREMBERG, GERMANY German Robotics Conference: 13–15 March 2025, NUREMBERG, GERMANY European Robotics Forum: 25–27 March 2025, STUTTGART, GERMANY RoboSoft 2025: 23–26 April 2025, LAUSANNE, SWITZERLAND ICUAS 2025: 14–17 May 2025, CHARLOTTE, NC ICRA 2025: 19–23 May 2025, ATLANTA, GA London Humanoids Summit: 29–30 May 2025, LONDON IEEE RCAR 2025: 1–6 June 2025, TOYAMA, JAPAN 2025 Energy Drone & Robotics Summit: 16–18 June 2025, HOUSTON, TX RSS 2025: 21–25 June 2025, LOS ANGELES ETH Robotics Summer School: 21–27 June 2025, GENEVA IAS 2025: 30 June–4 July 2025, GENOA, ITALY ICRES 2025: 3–4 July 2025, PORTO, PORTUGAL IEEE World Haptics: 8–11 July 2025, SUWON, KOREA IFAC Symposium on Robotics: 15–18 July 2025, PARIS RoboCup 2025: 15–21 July 2025, BAHIA, BRAZIL Enjoy today’s videos! We’re introducing Helix, a generalist Vision-Language-Action (VLA) model that unifies perception, language understanding, and learned control to overcome multiple longstanding challenges in robotics. This is moderately impressive; my favorite part is probably the hand-offs and that extra little bit of HRI with what we’d call eye contact if these robots had faces. But keep in mind that you’re looking at close to best case for robotic manipulation, and that if the robots had been given the bag instead of well-spaced objects on a single color background, or if the fridge had a normal human amount of stuff in it, they might be having a much different time of it. Also, is it just me, or is the sound on this video very weird? Like, some things make noise, some things don’t, and the robots themselves occasionally sound more like someone just added in some ‘soft actuator sound’ or something. Also also, I’m of a suspicious nature, and when there is an abrupt cut between ‘robot grasps door’ and ‘robot opens door,’ I assume the worst. [ Figure ] Researchers at EPFL have developed a highly agile flat swimming robot. This robot is smaller than a credit card, and propels on the water surface using a pair of undulating soft fins. The fins are driven at resonance by artificial muscles, allowing the robot to perform complex maneuvers. In the future, this robot can be used for monitoring water quality or help with measuring fertilizer concentrations in rice fields [ Paper ] via [ Science Robotics ] I don’t know about you, but I always dance better when getting beaten with a stick. [ Unitree Robotics ] This is big news, people: Sweet Bite Ham Ham, one of the greatest and most useless robots of all time, has a new treat. All yours for about $100, overseas shipping included. [ Ham Ham ] via [ Robotstart ] MagicLab has announced the launch of its first generation self-developed dexterous hand product, the MagicHand S01. The MagicHand S01 has 11 degrees of freedom in a single hand. The MagicHand S01 has a hand load capacity of up to 5 kilograms, and in work environments, can carry loads of over 20 kilograms. [ MagicLab ] Thanks, Ni Tao! No, I’m not creeped out at all, why? [ Clone Robotics ] Happy 40th Birthday to the MIT Media Lab! Since 1985, the MIT Media Lab has provided a home for interdisciplinary research, transformative technologies, and innovative approaches to solving some of humanity’s greatest challenges. As we celebrate our 40th anniversary year, we’re looking ahead to decades more of imagining, designing, and inventing a future in which everyone has the opportunity to flourish. [ MIT Media Lab ] While most soft pneumatic grippers that operate with a single control parameter (such as pressure or airflow) are limited to a single grasping modality, this article introduces a new method for incorporating multiple grasping modalities into vacuum-driven soft grippers. This is achieved by combining stiffness manipulation with a bistable mechanism. Adjusting the airflow tunes the energy barrier of the bistable mechanism, enabling changes in triggering sensitivity and allowing swift transitions between grasping modes. This results in an exceptional versatile gripper, capable of handling a diverse range of objects with varying sizes, shapes, stiffness, and roughness, controlled by a single parameter, airflow, and its interaction with objects. [ Paper ] via [ BruBotics ] Thanks, Bram! In this article, we present a design concept, in which a monolithic soft body is incorporated with a vibration-driven mechanism, called Leafbot. This proposed investigation aims to build a foundation for further terradynamics study of vibration-driven soft robots in a more complicated and confined environment, with potential applications in inspection tasks. [ Paper ] via [ IEEE Transactions on Robots ] We present a hybrid aerial-ground robot that combines the versatility of a quadcopter with enhanced terrestrial mobility. The vehicle features a passive, reconfigurable single wheeled leg, enabling seamless transitions between flight and two ground modes: a stable stance and a dynamic cruising configuration. [ Robotics and Intelligent Systems Laboratory ] I’m not sure I’ve ever seen this trick performed by a robot with soft fingers before. [ Paper ] There are a lot of robots involved in car manufacturing. Like, a lot. [ Kawasaki Robotics ] Steve Willits shows us some recent autonomous drone work being done at the AirLab at CMU’s Robotics Institute. [ Carnegie Mellon University Robotics Institute ] Somebody’s got to test all those luxury handbags and purses. And by somebody, I mean somerobot. [ Qb Robotics ] Do not trust people named Evan. [ Tufts University Human-Robot Interaction Lab ] Meet the Mind: MIT Professor Andreea Bobu. [ MIT ]

About a year ago, Boston Dynamics released a research version of its Spot quadruped robot, which comes with a low-level application programming interface (API) that allows direct control of Spot’s joints. Even back then, the rumor was that this API unlocked some significant performance improvements on Spot, including a much faster running speed. That rumor came from the Robotics and AI (RAI) Institute, formerly The AI Institute, formerly the Boston Dynamics AI Institute, and if you were at Marc Raibert’s talk at the ICRA@40 conference in Rotterdam last fall, you already know that it turned out not to be a rumor at all. Today, we’re able to share some of the work that the RAI Institute has been doing to apply reality-grounded reinforcement learning techniques to enable much higher performance from Spot. The same techniques can also help highly dynamic robots operate robustly, and there’s a brand new hardware platform that shows this off: an autonomous bicycle that can jump. See Spot Run This video is showing Spot running at a sustained speed of 5.2 meters per second (11.6 miles per hour). Out of the box, Spot’s top speed is 1.6 meters per second, meaning that RAI’s spot has more than tripled (!) the quadruped’s factory speed. If Spot running this quickly looks a little strange, that’s probably because it is strange, in the sense that the way this robot dog’s legs and body move as it runs is not very much like a real dog at all. “The gait is not biological, but the robot isn’t biological,” explains Farbod Farshidian, roboticist at the RAI Institute. “Spot’s actuators are different from muscles, and its kinematics are different, so a gait that’s suitable for a dog to run fast isn’t necessarily best for this robot.” The best Farshidian can categorize how Spot is moving is that it’s somewhat similar to a trotting gait, except with an added flight phase (with all four feet off the ground at once) that technically turns it into a run. This flight phase is necessary, Farshidian says, because the robot needs that time to successively pull its feet forward fast enough to maintain its speed. This is a “discovered behavior,” in that the robot was not explicitly programmed to “run,” but rather was just required to find the best way of moving as fast as possible. Reinforcement Learning Versus Model Predictive Control The Spot controller that ships with the robot when you buy it from Boston Dynamics is based on model predictive control (MPC), which involves creating a software model that approximates the dynamics of the robot as best you can, and then solving an optimization problem for the tasks that you want the robot to do in real time. It’s a very predictable and reliable method for controlling a robot, but it’s also somewhat rigid, because that original software model won’t be close enough to reality to let you really push the limits of the robot. And if you try to say, “okay, I’m just going to make a super detailed software model of my robot and push the limits that way,” you get stuck because the optimization problem has to be solved for whatever you want the robot to do, in real time, and the more complex the model is, the harder it is to do that quickly enough to be useful. Reinforcement learning (RL), on the other hand, learns offline. You can use as complex of a model as you want, and then take all the time you need in simulation to train a control policy that can then be run very efficiently on the robot. Your browser does not support the video tag. In simulation, a couple of Spots (or hundreds of Spots) can be trained in parallel for robust real-world performance.Robotics and AI Institute In the example of Spot’s top speed, it’s simply not possible to model every last detail for all of the robot’s actuators within a model-based control system that would run in real time on the robot. So instead, simplified (and typically very conservative) assumptions are made about what the actuators are actually doing so that you can expect safe and reliable performance. Farshidian explains that these assumptions make it difficult to develop a useful understanding of what performance limitations actually are. “Many people in robotics know that one of the limitations of running fast is that you’re going to hit the torque and velocity maximum of your actuation system. So, people try to model that using the data sheets of the actuators. For us, the question that we wanted to answer was whether there might exist some other phenomena that was actually limiting performance.” Searching for these other phenomena involved bringing new data into the reinforcement learning pipeline, like detailed actuator models learned from the real world performance of the robot. In Spot’s case, that provided the answer to high-speed running. It turned out that what was limiting Spot’s speed was not the actuators themselves, nor any of the robot’s kinematics: It was simply the batteries not being able to supply enough power. “This was a surprise for me,” Farshidian says, “because I thought we were going to hit the actuator limits first.” Spot’s power system is complex enough that there’s likely some additional wiggle room, and Farshidian says the only thing that prevented them from pushing Spot’s top speed past 5.2 m/s is that they didn’t have access to the battery voltages so they weren’t able to incorporate that real world data into their RL model. “If we had beefier batteries on there, we could have run faster. And if you model that phenomena as well in our simulator, I’m sure that we can push this farther.” Farshidian emphasizes that RAI’s technique is about much more than just getting Spot to run fast—it could also be applied to making Spot move more efficiently to maximize battery life, or more quietly to work better in an office or home environment. Essentially, this is a generalizable tool that can find new ways of expanding the capabilities of any robotic system. And when real world data is used to make a simulated robot better, you can ask the simulation to do more, with confidence that those simulated skills will successfully transfer back onto the real robot. Ultra Mobility Vehicle: Teaching Robot Bikes to Jump Reinforcement learning isn’t just good for maximizing the performance of a robot—it can also make that performance more reliable. The RAI Institute has been experimenting with a completely new kind of robot that they invented in-house: a little jumping bicycle called the Ultra Mobility Vehicle, or UMV, which was trained to do parkour using essentially the same RL pipeline for balancing and driving as was used for Spot’s high speed running. There’s no independent physical stabilization system (like a gyroscope) keeping the UMV from falling over; it’s just a normal bike that can move forwards and backwards and turn its front wheel. As much mass as possible is then packed into the top bit, which actuators can rapidly accelerate up and down. “We’re demonstrating two things in this video,” says Marco Hutter, director of the RAI Institute’s Zurich office. “One is how reinforcement learning helps make the UMV very robust in its driving capabilities in diverse situations. And second, how understanding the robots’ dynamic capabilities allows us to do new things, like jumping on a table which is higher than the robot itself.” “The key of RL in all of this is to discover new behavior and make this robust and reliable under conditions that are very hard to model. That’s where RL really, really shines.” —Marco Hutter, The RAI Institute As impressive as the jumping is, for Hutter, it’s just as difficult (if not more difficult) to do maneuvers that may seem fairly simple, like riding backwards. “Going backwards is highly unstable,” Hutter explains. “At least for us, it was not really possible to do that with a classical [MPC] controller, particularly over rough terrain or with disturbances.” Getting this robot out of the lab and onto terrain to do proper bike parkour is a work in progress that the RAI Institute says they’ll be able to demonstrate in the near future, but it’s really not about what this particular hardware platform can do—it’s about what any robot can do through RL and other learning-based methods, says Hutter. “The bigger picture here is that the hardware of such robotic systems can in theory do a lot more than we were able to achieve with our classic control algorithms. Understanding these hidden limits in hardware systems lets us improve performance and keep pushing the boundaries on control.” Your browser does not support the video tag. Teaching the UMV to drive itself down stairs in sim results in a real robot that can handle stairs at any angle.Robotics and AI Institute Reinforcement Learning for Robots Everywhere Just a few weeks ago, The RAI Institute announced a new partnership with Boston Dynamics ‘to advance humanoid robots through reinforcement learning.’ Humanoids are just another kind of robotic platform, albeit a significantly more complicated one with many more degrees of freedom and things to model and simulate. But when considering the limitations of model predictive control for this level of complexity, a reinforcement learning approach seems almost inevitable, especially when such an approach is already streamlined due to its ability to generalize. “One of the ambitions that we have as an institute is to have solutions which span across all kinds of different platforms,” says Hutter. “It’s about building tools, about building infrastructure, building the basis for this to be done in a broader context. So not only humanoids, but driving vehicles, quadrupeds, you name it. But doing RL research and showcasing some nice first proof of concept is one thing—pushing it to work in the real world under all conditions, while pushing the boundaries in performance, is something else.” Transferring skills into the real world has always been a challenge for robots trained in simulation, precisely because simulation is so friendly to robots. “If you spend enough time,” Farshidian explains, “you can come up with a reward function where eventually the robot will do what you want. What often fails is when you want to transfer that sim behavior to the hardware, because reinforcement learning is very good at finding glitches in your simulator and leveraging them to do the task.” Simulation has been getting much, much better, with new tools, more accurate dynamics, and lots of computing power to throw at the problem. “It’s a hugely powerful ability that we can simulate so many things, and generate so much data almost for free,” Hutter says. But the usefulness of that data is in its connection to reality, making sure that what you’re simulating is accurate enough that a reinforcement learning approach will in fact solve for reality. Bringing physical data collected on real hardware back into the simulation, Hutter believes, is a very promising approach, whether it’s applied to running quadrupeds or jumping bicycles or humanoids. “The combination of the two—of simulation and reality—that’s what I would hypothesize is the right direction.”