Welcome to our AI podcast, where we dive into the fascinating world of artificial intelligence! I'm your host, Sigurd, and I'm joined by my co-host, Carsten. Today, we're excited to discuss the seventh edition of the AI Index report from Stanford, which was released in 2024. Carsten, what can you tell us about this report?
####
Well, Sigurd, the 2024 AI Index report is the most comprehensive one to date, and it comes at a crucial time when AI's influence on society is more pronounced than ever. The report covers essential trends such as technical advancements in AI, public perceptions of the technology, and the geopolitical dynamics surrounding its development.
####
That's right, Carsten. The report also features more original data than ever before, including new estimates on AI training costs, detailed analyses of the responsible AI landscape, and an entirely new chapter dedicated to AI's impact on science and medicine. It's fascinating to see how AI is shaping our world in so many ways.
####
Absolutely! One of the key takeaways from the report is that AI has surpassed human performance on several benchmarks, such as image classification, visual reasoning, and English understanding. However, it still trails behind on more complex tasks like competition-level mathematics, visual commonsense reasoning, and planning.
####
That's a great point, Carsten. It's important to recognize that while AI has made significant strides, it's not a one-size-fits-all solution. Another interesting finding is that industry continues to dominate frontier AI research, with 51 notable machine learning models produced by industry in 2023, compared to only 15 from academia.
####
Yes, and it's worth noting that there were also 21 notable models resulting from industry-academia collaborations in 2023, which is a new high. This highlights the importance of collaboration between these two sectors in advancing AI research.
####
Speaking of advancements, the report also reveals that the training costs of state-of-the-art AI models have reached unprecedented levels. For example, OpenAI's GPT-4 used an estimated $78 million worth of compute to train, while Google's Gemini Ultra cost a staggering $191 million for compute.
####
It's incredible to see the resources being poured into AI development. The report also shows that the United States leads China, the EU, and the U.K. as the leading source of top AI models, with 61 notable AI models originating from U.S.-based institutions in 2023.
####
However, the report also highlights some areas where improvement is needed. For instance, robust and standardized evaluations for large language model responsibility are seriously lacking. Leading developers primarily test their models against different responsible AI benchmarks, which complicates efforts to systematically compare the risks and limitations of top AI models.
####
That's a crucial point, Sigurd. As AI becomes more prevalent in our lives, it's essential to have standardized evaluations to ensure its responsible development and deployment.
####
Definitely. On a more positive note, the report shows that generative AI investment has skyrocketed, nearly octupling from 2022 to reach $25.2 billion, despite a decline in overall AI private investment last year.
####
And the data also suggests that AI makes workers more productive and leads to higher quality work. Several studies in 2023 demonstrated AI's potential to bridge the skill gap between low- and high-skilled workers, although it's important to note that using AI without proper oversight can lead to diminished performance.
####
AI is also accelerating scientific progress, with the launch of even more significant science-related AI applications in 2023, such as AlphaDev, which makes algorithmic sorting more efficient, and GNoME, which facilitates the process of materials discovery.
####
It's exciting to see how AI is transforming various fields, but it's also important to consider the regulatory landscape. The report shows that the number of AI-related regulations in the United States has risen significantly over the past year and the last five years, with 25 AI-related regulations in 2023, up from just one in 2016.
####
And finally, the report reveals that people across the globe are more cognizant of AI's potential impact and more nervous about it. Surveys show that the proportion of those who think AI will dramatically affect their lives in the next three to five years has increased, and more than half of the respondents express nervousness toward AI products and services.
####
It's clear that as AI continues to advance, we need to have open and honest conversations about its impact on society and work towards responsible development and deployment of this powerful technology.
####
Absolutely, Carsten. The 2024 AI Index report provides us with valuable insights into the current state of AI and the challenges and opportunities that lie ahead. It's up to all of us to ensure that we harness the power of AI for the benefit of society as a whole.
####
Well said, Sigurd. That's all the time we have for today's episode. Thank you for joining us, and we look forward to bringing you more exciting discussions about AI in the future.
####
Until next time, stay curious and keep exploring the fascinating world of artificial intelligence!