Unlocking the Power of AI Foundation Models: Insights from the LEAM.AI Conference
Locations
I was blessed to be part of the team researching the LEAM-AI project. On January 24th, 2023, the official study was presented to the German Federal Ministry of Economics and Climate Protection. At the conference, experts outlined the study and its most substantial findings concerning AI technology. The event had a great turnout and was orchestrated nicely. But what were the most essential outcomes?
What does the LEAM.AI project aim at?
LEAM is a program sponsored by the German Artificial Intelligence Association (KI Bundesverband), made up of representatives from both industry and research, which seeks to further the development of large AI models.
In order to achieve this goal, six main steps have been proposed:
- the collection and creation of comprehensive training datasets;
- the support of high-quality research in the field of AI;
- providing access to powerful computing infrastructure;
- constructing organizational structures and processes which will orchestrate an ongoing cycle of model development and improvement;
- integrating the models into the European ecosystem of innovation;
- and developing methods, datasets and criteria which will ensure ethical standards and consider European values.
The initiators hope for the models to be open source and freely available to all market players, and for all European languages to be fully supported by them.
What makes this project so valuable to Germany?
ChatGPT is a leading example of how AI-based foundation models can launch groundbreaking applications. In the coming years, this advanced technology will be used to create new tools, platforms and economic models for virtually every aspect of life.
What are large AI Foundation Models?
AI-foundation-models are the latest way to build AI systems that can be adapted to several different tasks. Rather than requiring lengthy training on numerous labeled datasets, these models are trained on large amounts of unlabeled data and can be fine-tuned for various uses. One of the most widely used AI/ML models, Deep Neural Networks or DNN, provides an example of a foundation model. This model is based on artificial neurons, which are modeled after the neural networks existing in our brains. With this model, a relatively short prompt can produce a full essay or intricate image, even when it hasn’t been trained for that particular assignment or image creation.
In natural language processing, the introduction of foundation models has drastically reduced the quantity of labeled data needed to train an AI model for summarization. Certain models are so versatile that they can be used for many tasks, which means that only a thousand labeled examples are required to build an AI model for summarization. Furthermore, these models have demonstrated their ability to generate creative outputs, such as coherent arguments and original pieces of art. In other words, foundation models can be used in a wide array of situations, increasing the range of applications for AI in mission-critical circumstances with minimal effort and resources.
Overall, AI-foundation-models are revolutionizing the way we use AI, making it easier and more efficient to build and deploy AI models. They are already being used in many areas, such as natural language processing, and their potential is limitless.
ChatGPT is an excellent example for use cases of such a large AI Foundation model.
US dominated
The development of AI-Foundation models is mainly concentrated in the United States. American models have become a dominant force in the market. Since 2017, 73% of AI-Foundation models have been developed in the US and 15% in China. In the US, the development of such models has been heavily funded by major technology companies with investments in the billions of dollars.
Germany is facing considerable challenges
There is the potential danger that Germany could miss the paradigm shift of the KI foundation models and become increasingly dependent on American models. Digital sovereignty in the area of artificial intelligence and downstream applications is under threat, presenting a hazard to the competitiveness of the entire economy of Germany.
The demand for language models is particularly high. Of the models available from the KL Foundation, language models currently have the greatest relevance in terms of usage and development. 71% of KL companies are focusing on this area. Multimodal models (38%) and models based on business and production processes (34%) follow behind.
Industry and companies have a strong demand for services, and the decision makers were largely in agreement that Europe and Germany must build a response, to not be left behind by the US and China. A key factor in achieving this is providing computing power, data pools and support for training and consulting to customize applications to specific needs and use cases with open source models.
Germany needs a powerful AI computing infrastructure
In order to develop the necessary foundation models for AI, Germany must have a powerful supercomputing infrastructure, which is currently not in place. It is cost-efficient to build such a computer in Germany, with an estimated price tag of 350 to 400 million euros.
Promising discussions
At the meeting of the panelists, the proposal of constructing such an advanced AI supercomputing system in Germany was well-received by all present. Although there were questions still to be answered, enthusiasm for the mission was strong.
The discussion regarding the LEAM.AI project, which was completed within two years, was highly praised by members of the federal government. Conversations surrounding ChatGPT’s use in companies created a stir, however, it was noted that companies need to update their processes and procedures before use. Additionally, it was stressed that words can take on different meanings from company to company, meaning AI language models must be tailored for each one to prevent any inaccuracies from appearing.
The people in German boardrooms had not yet fully grasped what kind of effects the development of this AI could have on a data-driven economy in the future, and all the dependencies that could come out of it, particularly between the USA and Germany and the rest of Europe. It was our responsibility to make them aware of this in the near future, something which I agree with totally.
Nevertheless, everybody agreed that the progress was inevitable. Huge AI models would bring about a huge transformation in the economy, government and culture. It wouldn’t bode well for Germany if the German (or even European) economy were reliant on foreign providers (like those of the US or China). Hence, there was an urgent need to use the media coverage of ChatGPT as an impetus to propel the project forward.
At Fieldfisher Germany, we are wholeheartedly committed to the LEAM initiative and are excited to take part in supporting it. My colleagues and I believe strongly in its importance, so our enthusiasm for the cause is sincere.
Keep up to date with new developments in technology and law. Connect with me on Linkedin, listen to my podcast or follow me on Youtube.