What role does automated machine learning play in agentic AI systems ?

Marius Lindauer: Automated machine learning, or AutoML for short, plays a central role in agentic AI systems. AI agents are among the most complex technical systems we have developed to date. They consist of many components: Which tools should be used? Are sub-agents needed to handle specific subtasks? Are specialised agents required? Does a model need to be fine-tuned for specific tasks? How can prompts be optimised or data better represented? These are precisely the kinds of questions AutoML researchers are currently addressing. The aim is to make agentic AI more efficient and to develop its ability to find good solutions for specific tasks.

What are the opportunities and challenges?

Marius Lindauer: On the one hand, AutoML can be used to improve agentic AI systems. On the other hand, agentic AI systems itself can be used to further develop AI and machine learning.

The idea behind this is to use AI models and AI agents to enable new breakthroughs in AI and ML research: for example, finding better architectures, developing the next architecture after the Transformer, improving feature pre-processing, automatically generating features, identifying fundamentally more powerful learning methods, and so on.

At the same time, it is becoming clear that such systems cannot tackle these tasks on their own. What is needed is a combination of specialised methods and powerful agent systems. These include, for example, optimisation methods such as hyperparameter optimisation, which can be used in conjunction with AI agents to develop more powerful systems.

What is the situation regarding the energy efficiency of agentic AI systems?

Marius Lindauer: Agentic AI systems are typically not particularly energy-efficient. Particularly in large contexts, processing large numbers of tokens requires computing power and thus electricity – regardless of whether information is retrieved automatically or added to the context.

AutoML offers starting points here for making such systems significantly more efficient. This includes the question of how models can be compressed and made smaller. One approach is pruning, i.e. the targeted removal of less relevant parameters. Quantisation reduces the numerical precision of weights. Linear attention or similar approaches reduce the computational complexity of sequence processing. It is also a matter of combining various efficiency measures in such a way that the best agentic AI systems can be identified under specific conditions.

What role is left for humans?

Marius Lindauer: Humans remain central to the design of AI systems, even though their development can increasingly be automated. Questions such as: What should be developed? To what end? – And how can it specifically help people? – remain crucial. These questions must be answered by human developers and subject matter experts.

One example is fairness: when an AI system is deployed in a large-scale context with many users, the question arises as to how they can be treated as fairly as possible. However, there are many different definitions of fairness. It can be described and quantified mathematically, but ultimately someone must decide which fairness metric should be applied in a specific case. Should the focus be on group fairness or fairness at the individual level? This is precisely where human expertise is needed to decide what AI should achieve, how people interact with AI, and how AI can be used for the benefit of people.