Meta CEO Mark Zuckerberg announces AGI project to build general, or artificial general, AI … [+]
Mark Zuckerberg, CEO of Meta announcement Thursday on Threads, he focuses Meta on creating complete general intelligence, or artificial general intelligence, and then releasing it as open source software for everyone.
“It has become clearer that the next generation of services requires building comprehensive general intelligence,” he said in a personal video. “Creating the best AI assistants, AIs for creators, AIs for businesses and more, requires advancements in all areas of AI, from reasoning and planning to coding, memory and more. other cognitive abilities.”
To support this effort, Zuckerberg said Meta will have a massive slate of computing power in its cloud facilities by the end of 2024: 350,000 Nvidia H100s, or about 600,000 H100 equivalents if you include other GPUs.
Only Microsoft is order enough H100 to build equivalent capacity, and with such huge orders, delivery times for the H100 extend up to a year.
H100 GPU Orders: Meta and Microsoft gobble up almost all available capacity.
Each Nvidia H100, announced in 2022, contains 80 billion transistors, is up to six times faster than previous models and has a memory bandwidth of up to three terabytes per second. A single implementation of just 4,600 H100 forms a supercomputer, Eoswhile the Meta will be the equivalent of 130 times larger.
Do the math, and Zuckerberg’s massive AI computing capacity will be 4.8e+16 transistors. That’s 48,000,000,000,000,000, or 48 quadrillion.
With this enormous computing power, Zuckerberg says Meta will continue to train Llama 3, and an “exciting roadmap of future models that we will also train responsibly and safely.”
Llama 3 is a generative AI text model that some believe could challenge or even surpass OpenAI’s GPT-4, currently the gold standard for generative AI models. Meta made Llama 2, its predecessor, freely available, and it looks like something similar will happen with Llama 3.
Zuckerberg believes AI and the Metaverse are intertwined, and he says smart glasses will be the way most people experience AI and the Metaverse together.
“A lot of us are going to be talking to AI all day long,” he said. “I think a lot of us are going to do this using glasses, because glasses are the ideal form factor to allow an AI to see what you see and hear what you hear.”
He also referenced Meta’s Ray-Ban Meta glasses, which he said are “off to a very good start.”