The Nvidia Jetson platform continues to expand to support the development of autonomous robots
Nvidia announced what the company called its “largest-ever platform expansion for Edge AI and robotics,” and rightfully so. In typical Nvidia fashion, the company brings together years of work across several different software platforms to meet the needs of a very specific application: robotics. Nvidia has been developing solutions for robotics, or what might more appropriately be called autonomous machines, for over a decade. In conjunction with the company’s investment in artificial intelligence (AI) and autonomous vehicles, Nvidia was one of the first technology companies to see the potential of future robotic platforms leveraging AI and technology. developed for other segments, notably the automobile industry.
The announcement includes the adaptation of Generative AI (GenAI) models to the Jetson Orin platform, the availability of Metropolis APIs and microservices for vision applications, the release of the latest Isaac ROS (Robot Operating System) framework and the Isaac AMR (autonomous mobile robot) platform, and the release of the JetPack 6 software development kit (SDK). According to Nvidia, these releases are more valuable than all other releases in the last 10 years combined. All areas highlighted in green in the figure below are new or updated in this version of the platform.
The entire hardware and software stack for the Nvidia Jetson platform
One of the most significant improvements is support for GenAI. As we have stated at Tirias Research in previous articles, GenAI will be transformative not only for people, but also for machines. Nvidia is working to enable zero-shot learning, which will allow devices to learn and predict results based on classes of data rather than being trained on specific samples. This will enable shorter training cycles and more flexible interactions and usage with the resulting models. Nvidia also predicts a new cycle of innovation for autonomous machines, as much learning will shift from text-based solutions to video or multimodal (text, audio, and video) training. Nvidia also included its new PeopleNet transformer model to increase the accuracy of people identification. But most important is the Orin platform’s ability to run large language models (LLM). According to Nvidia, the platform can handle most LLMs available and has given examples of models as large as the 70 billion parameter version of Meta’s Llama 2 model. Additionally, Nvidia has created a Jetson Generative AI Lab so developers can start working with open source models at the edge.
To enable this rise in video learning and the further development of computer vision, Nvidia is also enhancing its Metropolis framework with APIs and microservices for rapid development and management of enterprise applications. This framework extends beyond simple vision, called “pixel perception” in the image below, to include other software solutions that allow a developer to create a complete computer vision platform for a variety of different applications in which equipment, connectivity, and visual requirements may vary.
Nvidia’s expanded Metropolis framework for robotics
The third part of the release is the enhancement of the Isaac software stack, including the general availability of the Isaac ROS 2.0 GPU-accelerated framework built on the open source Robot Operating System (ROS), a new version of Isaac Sim 2023.1 for synthetic data generation and robotic simulation, and Isaac AMR for the development of autonomous mobile robots.
And the last part of the release is JetPack 6, which offers new features including support for new commercial Linux distributions, new system services, easily scalable compute stack, and advanced security features.
While these announcements are not about a new hardware platform, they demonstrate how critical software is, which I believe has more impact than hardware when it comes to AI and autonomous machines. This also demonstrates Nvidia’s continued innovation at the platform/solution level. Tirias Research has had the opportunity to evaluate the various Nvidia platforms over the last decade and nothing compares to the ease of use, comprehensive tutorials, software resources and performance of the Nvidia platforms Jetson, in our opinion. Robots are essential to continually improving productivity across industries from agriculture to manufacturing, and like AI training, Nvidia appears to be in a strong leadership position with over 1 .2 million developers and 10,000 companies designing with the Jetson platform.