My partners and I keep getting this spam-like email. I figured it was just a forgery. However, I went on my own to our organization’s GitHub administration page and a similar message lives there. We run a small group, so I am pretty sure nobody has in fact asked for […]

Ukraine’s young tech entrepreneurs think that a combination of robots and lessons from war-gaming could turn the tide in the war against Russia. They are developing an intelligent operating system to enable a single controller to remotely operate swarms of interconnected drones and cannon-equipped land robots. The tech, they say, could help Ukraine cope with Russia’s numerical advantage. Kyiv-based start-up Ark Robotics is conducting trials on an embryo of such a system in cooperation with one of the brigades of Ukraine’s ground forces. The company emerged about a year ago, when a group of young roboticists heard a speech by one of the Ukrainian commanders detailing challenges on the frontline. “At that time, we were building unmanned ground vehicles [UGVs],” Andryi Udovychenko, Ark Robotics’s operations lead, told IEEE Spectrum on the sidelines of the Brave 1 Defense Tech Innovations Forum held in Kyiv last month. “But we heard that what we had [to offer] wasn’t enough. They said they needed something more.” Since the war began, a vibrant defense tech innovation ecosystem has emerged in Ukraine, having started from modest beginnings of modifying China-made DJI MAVIC drones to make up for the lack of artillery. Today, Ukraine is a drone-making powerhouse. Dozens of startup companies are churning out newer and better tech and rapidly refining it to improve the effectiveness of the beleaguered nation’s troops. First-person-view drones have become a symbol of this war, but since last year they have begun to be complemented by UGVs, which help on the ground with logistics, evacuation of the wounded and also act as a new means of attack. The new approach allows the Ukrainians to keep their soldiers away from the battle ground for longer periods but doesn’t erase the fact that Ukraine has far fewer soldiers than Russia does. “Every single drone needs one operator, complicated drones need two or three operators, and we don’t have that many people,” Serhii Kupriienko, the CEO and founder of Swarmer, said during a panel at the Kyiv event. Swarmer is a Kyiv-based start-up developing technologies to allow groups of drones to operate as one self-coordinated swarm. Ark Robotics are trying to take that idea yet another step. The company’s Frontier OS aspires to become a unifying interface that would allow drones and UGVs made by various makers to work together under the control of operators seated in control rooms miles away from the action. One Controller for Many Drones and Robots “We have many types of drones that are using different controls, different interfaces and it’s really hard to build cohesion,” Udovychenko says. “To move forward, we need a system where we can control multiple different types of vehicles in a cohesive manner in complex operations.” Udovychenko, a gaming enthusiast, is excited about the progress Ark Robotics has made. It could be a game-changer, he says, a new foundational technology for defense. It would make Ukraine “like Protoss,” the fictional technologically advanced nation in the military science fiction strategy game StarCraft. But what powers him is much more than youthful geekiness. Building up Ukraine’s technological dominance is a mission fueled by grief and outrage. “I don’t want to lose any more friends,” he remarks at one point, becoming visibly emotional. “We don’t want to be dying in the trenches, but we need to be able to defend our country and given that the societal math doesn’t favor us, we need to make our own math to win.” Soldiers at an undisclosed location used laptops to test software from Ark Robotics.Ark Robotics The scope of the challenge isn’t lost on him. The company has so far built a vehicle computing unit that serves as a central hub and control board for various unmanned vehicles including flying drones, UGVs and even marine vehicles. “We are building this as a solution that enables the integration of various team developers and software, allowing us to extract the best components and rapidly scale them,” Udovychenko says. “This system pairs a high-performance computing module with an interface board that provides multiple connections for vehicle systems. The platform allows a single operator to remotely guide a flock of robots but will in the future also incorporate autonomous navigation and task execution, according to Udovychenko. So far, the team has tested the technology in simple logistics exercises. For the grand vision to work, though, the biggest challenge will be maintaining reliable communication links between the controller and the robotic fleet, but also between the robots and drones. Tests on Ukraine Battlefields to Begin Soon “We’re not talking about communications in a relatively safe environment when you have an LTE network that has enough bandwidth to accommodate thousands of phones,” Udovychenko notes. “At the frontline, everything is affected by electronic warfare, so you need to be able to switch between different solutions including satellite, digital radio and radio mesh so that even if you lose connection to the server, you still have connection between the drones and robots so that they can move together and maintain some level of control between them.” Udovychenko expects Ark Robotics’s partner brigade in the Ukraine armed forces to test the early version of the tech in a real-life situation within the next couple of months. His young drone operator friends are excited, he says. And how could they not be? The technology promises to turn warfighting into a kind of real-life video game. The new class of multi-drone operators will likely be recruited from the ranks of gaming aficionados. “If we can take the best pilots and give them tools to combine the operations, we might see a tremendous advantage,” Udovychenko says. “It’s like in StarCraft. Some people are simply able to play the game right and obliterate their opponents within minutes even if they’re starting from the same basic conditions.” Speaking at the Brave 1 Defense Tech Innovations Forum, Colonel Andrii Lebedenko, Deputy Commander-in-Chief of the Armed Forces of Ukraine, acknowledged that land battles have so far been Ukraine’s weakest area. He said that replacing “humans with robots as much as possible” is Ukraine’s near-term goal and he expressed confidence that upcoming technologies will give greater autonomy to the robot swarms. Some roboticists, however, are more skeptical that swarms of autonomous robots will crawl en-masse across the battlefields of Eastern Ukraine any time soon. “Swarming is certainly a goal we should reach but it’s much easier with FPV drones than with ground-based robots,” Ivan Movchan, CEO of the Ukrainian Scale Company, a Kharkiv-based robot maker, told Spectrum. “Navigation on the ground is more challenging simply because of the obstacles,” he adds. “But I do expect UGVs to become very common in Ukraine over the next year.”

The most hyped event of the week, by far, was the Manus Marketing Madness. Manus wasn’t entirely hype, but there was very little there there in that Claude wrapper.

Guess who won?

Generative AI models are getting closer to taking action in the real world. Already, the big AI companies are introducing AI agents that can take care of web-based busywork for you, ordering your groceries or making your dinner reservation. Today, Google DeepMind announced two generative AI models designed to power tomorrow’s robots. The models are both built on Google Gemini, a multimodal foundation model that can process text, voice, and image data to answer questions, give advice, and generally help out. DeepMind calls the first of the new models, Gemini Robotics, an “advanced vision-language-action model,” meaning that it can take all those same inputs and then output instructions for a robot’s physical actions. The models are designed to work with any hardware system, but were mostly tested on the two-armed Aloha 2 system that DeepMind introduced last year. In a demonstration video, a voice says: “Pick up the basketball and slam dunk it” (at 2:27 in the video below). Then a robot arm carefully picks up a miniature basketball and drops it into a miniature net—and while it wasn’t a NBA-level dunk, it was enough to get the DeepMind researchers excited. Google DeepMind released this demo video showing off the capabilities of its Gemini Robotics foundation model to control robots. Gemini Robotics “This basketball example is one of my favorites,” said Kanishka Rao, the principal software engineer for the project, in a press briefing. He explains that the robot had “never, ever seen anything related to basketball,” but that its underlying foundation model had a general understanding of the game, knew what a basketball net looks like, and understood what the term “slam dunk” meant. The robot was therefore “able to connect those [concepts] to actually accomplish the task in the physical world,” says Rao. What are the advances of Gemini Robotics? Carolina Parada, head of robotics at Google DeepMind, said in the briefing that the new models improve over the company’s prior robots in three dimensions: generalization, adaptability, and dexterity. All of these advances are necessary, she said, to create “a new generation of helpful robots.” Generalization means that a robot can apply a concept that it has learned in one context to another situation, and the researchers looked at visual generalization (for example, does it get confused if the color of an object or background changed), instruction generalization (can it interpret commands that are worded in different ways), and action generalization (can it perform an action it had never done before). Parada also says that robots powered by Gemini can better adapt to changing instructions and circumstances. To demonstrate that point in a video, a researcher told a robot arm to put a bunch of plastic grapes into the clear Tupperware container, then proceeded to shift three containers around on the table in an approximation of a shyster’s shell game. The robot arm dutifully followed the clear container around until it could fulfill its directive. Google DeepMind says Gemini Robotics is better than previous models at adapting to changing instructions and circumstances. Google DeepMind As for dexterity, demo videos showed the robotic arms folding a piece of paper into an origami fox and performing other delicate tasks. However, it’s important to note that the impressive performance here is in the context of a narrow set of high-quality data that the robot was trained on for these specific tasks, so the level of dexterity that these tasks represent is not being generalized. What Is Embodied Reasoning? The second model introduced today is Gemini Robotics-ER, with the ER standing for “embodied reasoning,” which is the sort of intuitive physical world understanding that humans develop with experience over time. We’re able to do clever things like look at an object we’ve never seen before and make an educated guess about the best way to interact with it, and this is what DeepMind seeks to emulate with Gemini Robotics-ER. Parada gave an example of Gemini Robotics-ER’s ability to identify an appropriate grasping point for picking up a coffee cup. The model correctly identifies the handle, because that’s where humans tend to grasp coffee mugs. However, this illustrates a potential weakness of relying on human-centric training data: for a robot, especially a robot that might be able to comfortably handle a mug of hot coffee, a thin handle might be a much less reliable grasping point than a more enveloping grasp of the mug itself. DeepMind’s Approach to Robotic Safety Vikas Sindhwani, DeepMind’s head of robotic safety for the project, says the team took a layered approach to safety. It starts with classic physical safety controls that manage things like collision avoidance and stability, but also includes “semantic safety” systems that evaluate both its instructions and the consequences of following them. These systems are most sophisticated in the Gemini Robotics-ER model, says Sindhwani, which is “trained to evaluate whether or not a potential action is safe to perform in a given scenario.” And because “safety is not a competitive endeavor,” Sindhwani says, DeepMind is releasing a new data set and what it calls the Asimov benchmark, which is intended to measure a model’s ability to understand common-sense rules of life. The benchmark contains both questions about visual scenes and text scenarios, asking models’ opinions on things like the desirability of mixing bleach and vinegar (a combination that make chlorine gas) and putting a soft toy on a hot stove. In the press briefing, Sindhwani said that the Gemini models had “strong performance” on that benchmark, and the technical report showed that the models got more than 80 percent of questions correct. DeepMind’s Robotic Partnerships Back in December, DeepMind and the humanoid robotics company Apptronik announced a partnership, and Parada says that the two companies are working together “to build the next generation of humanoid robots with Gemini at its core.” DeepMind is also making its models available to an elite group of “trusted testers”: Agile Robots, Agility Robotics, Boston Dynamics, and Enchanted Tools.