Hannover Messe 2024 was a resounding success! As a Google Partner, SOTEC was honored to showcase our cutting-edge technology at the Google Booth. We were excited to demonstrate how easy and intuitive it can be to control complex robotic systems using the power of natural language and Google’s Gemini language model.
Our demonstrator captivated audiences. Here’s how it works:
User Input:
In a web-based UI, visitors would type a command in plain English, describing the task they wanted the robot to perform.
Examples:
-
- “Stack all red objects.”
- “Arrange the squares in a line.”
- “Build me a snowman.”
- “Make me a cappuccino”
2. Object Detection:
A built-in camera captures an image of a workspace containing various shapes (squares, circles, triangles) in different colors (green, blue, red, etc.). Powerful object detection AI identified and located each item.
3. Gemini’s Magic:
This data – the object details and the user’s natural-language instruction – was sent to Google Gemini. Gemini’s astonishing capabilities allowed it to break down the task into logical steps and formulate a solution strategy for the robot, just using the available pieces. Even further, Gemini could easily be advised to provide the solution in a specific format (JSON).
4. Robot in Action:
Using that strategy provided by Gemini we could transfer it into precise commands the robot could execute, completing the visitor’s request. The results could either be visualized in 2D or 3D.
Why This Matters
This seemingly simple demonstration holds profound implications for the future of robotics:
- No Programming Expertise Needed: Previously, controlling a robot required specialized knowledge. Our demo proves that, with AI, anyone can interact with robots intuitively.
- Adaptability: The workspace changed constantly. Gemini dynamically adapted its strategies to handle new object combinations and tasks.
- The Power of Partnership: This showcase highlights the incredible potential of collaboration between SOTEC’s technology solutions and Google’s groundbreaking AI.
While this application is still static in the setup – the environment was especially developed for this kind of task – the way from a problem to the solution is highly flexible and surprisingly accurate. Still being at an early stage of development, AI can and will make the programming of robotic applications more efficient and faster. A good example of this is Intrinsic, an Alphabet company which is working on the control for the robot itself. See our blog post for more information about that.
Hannover Messe: A Global Tech Hub
Hannover Messe is the world’s leading industrial technology trade fair. It’s a place where innovators, industry leaders, and tech enthusiasts converge to share the transformative solutions shaping our future. 2024’s event was electrifying, showcasing advancements in AI, automation, and sustainability that will define tomorrow’s manufacturing landscape.
Learn More
Missed us at Hannover Messe? Want to explore how these technologies could revolutionize your operations? Contact SOTEC today! Let’s discuss how we can partner to leverage AI and natural language, bringing new flexibility and efficiency to your workplace.
You can find more news here.