How Microsoft used ChatGPT to fly drones and robots


New member
Microsoft believes that ChatGPT can revolutionize the robotics industry. Indeed, thanks to the artificial intelligence of OpenAI, it is possible to control drones and robots without being a programming expert.

Eager to take the lead in the artificial intelligence race, Microsoft has pumped a fortune into OpenAI, the start-up behind ChatGPT. Buoyed by several huge investments, the Redmond giant quickly integrated the chatbot into Bing, its search engine, and Edge, its web browser. Microsoft account Keep it up by integrating ChatGPT with other leading products, such as Outlook, Word or Powerpoint.

The American company does not want to limit ChatGPT to the software sector. Microsoft also wants to use the advancements of OpenAI to revolutionize the world of robotics. In a report published a few days ago, the company reveals that it used the OpenAI chatbot to monitor “robotic arms, drones and home assistant robots”.

In detail, Microsoft relied on ChatGPT to control a flying drone. At the request of the AI, the craft performed a zigzag motion in the air before taking a selfie with the integrated camera. By following ChatGPT’s commands, the drone also managed to reach a specific location while avoiding obstacles in its path.

The engineers also used the chatbot to testing in Microsoft AirSim, the simulator for drones. In its labs, Microsoft has also used ChatGPT to order a robotic arm to stack blocks. The machine succeeded in stacking these blocks in a very specific pattern.

Facilitate communication between man and machine

Microsoft explains that it will facilitate human-robot interaction based on OpenAI language models. To achieve this, ChatGPT must be absolutely necessary “think beyond the text and reason about the physical world”. Clearly, the AI shouldn’t be content with generating responses based on its interlocutor’s requests. She must understand what the requests mean from an environmental and natural law point of view.

“It turns out that ChatGPT can do a lot by itself, but it still needs some help”says Microsoft, and assures that the initial results are encouraging.

Also Read: Microsoft Knew Bing’s ChatGPT Was Going to Panic… Months Before Launch
Use ChatGPT as a translator

As part of its research, the American giant has developed “design principles”. Thanks to these different principles, Microsoft has used ChatGPT generate code to control a robot. Based on these regulations, a complete methodology has been developed by the group’s engineers. This describes how it is possible to use a language model to facilitate the programming of a robot.

First of all, it is necessary to define a directory of functions of the robot. Each feature should have a descriptive title so that ChatGPT understands what it is. In a second step, the user writes a request in the chatbot’s interface describing the task that awaits the robot. As always, you need to give as much information as possible to the AI to get to the desired result.

When writing the request, the user must describe the purpose of the task, indicate which previously described functions should be used, and specify any restrictions. Based on the information provided, ChatGPT translate the query into programming language. The user can then evaluate the provided code and run a simulation as a precaution before deploying it to the PLC. At the end of this process, the robot will have followed a command generated by ChatGPT at the request of its interlocutor. The user does not necessarily need to know a complex programming language to communicate with a robot or a drone.

Note that Microsoft isn’t the only tech titan researching the usefulness of language models in robotics. Google is currently working on similar experiments based on PaLM (Pathways Language Model), a proprietary language model.

Source :