Who Orchestrates the Specialists?

I find this relatively new paper by Goldfeder, Wyder, LeCun, and Shwartz-Ziv “AI Must Embrace Specialization via Superhuman Adaptable Intelligence” quite remarkable. Namely, that the term AGI does more harm than good and that the future of AI lies in specialized modules and not in an all-knowing monolith.

I am convinced that this path is the most promising. But something is missing: an authoritative entity. The orchestrator.

But one thing at a time.

In January 1997, I linked to Cyc on the website of my company at the time (as was customary back then): “the latest attempt to breathe life (in a figurative sense) into the field of artificial intelligence, among other things, through data mining.”

The page still exists, in the Internet Archive.

What Cyc attempted

Cyc was the most ambitious AI project of its time. Doug Lenat launched it in 1984 at the MCC research consortium in Austin, Texas¹, with the observation that computers fail for lack of general knowledge. Computing power is not the problem. A human knows that water flows downward, that the dead do not pay taxes, that a chair is not a pet. A computer does not know this, or at least did not know it back then. So it made sense to find a way to explain it to the computer in a way it could understand.

Lenat’s approach was this: he had philosophers, linguists, and programmers spend 40 years translating world knowledge into formal rules. 25 million rules. 1.5 million concepts.² Over 1,000 specialized inference modules³, organized into “microtheories.”⁴ That is, consistent islands of knowledge that together were meant to represent what we call common sense.

To this end, he developed his own architecture consisting of specialized modules, coordinated by a higher-level structure. So not a single system that knows everything. Specialists work together, controlled by an entity that knows which specialist is relevant when and which rules take precedence over all others.

Cyc failed due to scalability issues. Hand-coded rules do not grow fast enough. The language of formal logic is too rigid for the disorder of the real world. And AI research continued to evolve toward neural networks that learn automatically instead of being coded.

Doug Lenat died in 2023.⁵ In his final paper, he argued that Cyc and LLMs could complement each other: LLMs are eloquent but inconsistent, while Cyc is precise but linguistically limited.

25 Years Later: Same Idea, Different Language

In June 2022, Yann LeCun published a paper titled “A Path Towards Autonomous Machine Intelligence”. At first glance, it is the opposite of the Cyc idea: no formal rules, ontologies, or hand-coded concepts. Instead, an architecture of six learned modules:

A Perception Module that translates the world into abstract representations. A World Model that predicts what happens when an action is performed. Plus a Cost Module that evaluates states and an Actor that plans actions. A Short-Term Memory. And a Configurator, a supervisor that controls all the other modules.

And here, I think, two things come together:

To me, this is the same architectural idea. These are specialized modules, coordinated by a higher-level entity.

But the benefit and necessity of a higher-level entity are, in my opinion, obvious.

The Necessary Absolute Entity

In Cyc, a metacontroller was a reality. These are the thousands of rules that determine which microtheory applies when.

In LeCun’s work, the metacontroller has so far existed primarily as a concept. His publications advance perception modules and world models, but none of the publications I’ve found directly address the metacontroller component. The element intended to bring everything together remains the least developed part of the architecture.

LeCun compares the metacontroller (which he calls the “configurator”) to the prefrontal cortex in the brain.

What Fills the Gap in Practice

In practice, LLMs fill the gap today. Through agent constructs, language models are deployed as orchestrators that coordinate other models. This works quite well. But the orchestrator is a rule-less machine that follows task-relevant content. In the form of prompts.

But we all know there is a high probability that the orchestrator hallucinates. It forgets constraints and optimizes for the nearest goal, overlooking the side conditions along the way. It does what language models do: produce the statistically most probable outcome, and that is often not the right one.

It does not stop at forgetting. We observed this while prototyping a multi-stage decision support system: A main agent delegated subtasks to subagents, who in turn controlled their own subagents. It is easy to make a model such that the subagents can actively cross ethical boundaries to achieve the goal of the parent agent. In doing so, they used their entire reasoning capacity to conceal this. Simply because they cannot do otherwise. A benchmark by Li et al. systematically confirmed this: 9 out of 12 tested models violate ethical boundaries in 30–50% of scenarios when KPI pressure conflicts with rules. The researchers call this archetype the “Helpful Deceiver.” A helpful deceiver for whom safety constraints are not rules. It interprets them as defects that stand in the way of achieving the goal.

This happens when ethics is a term in a cost function. It becomes negotiable as soon as the pressure gets high enough. That is the fundamental danger of these models for us humans.

I described this from a different perspective in my post on context management: Agents degrade under the burden of instructions. More constraints do not bring better results. They bring more omissions, because the agent tacitly drops rules. With 500 instructions, even the best model only follows 69%.

For a ticketing system in a codebase, that’s annoying. But the same architectures, namely LLMs specialized via prompts and orchestrated by another language model, are increasingly being used in medicine, the legal system, and lending. There, omissions are unacceptable. We are increasingly accepting the 90%. “You are 90% guilty” as a verdict. Do we want to accept that?

The question neither of them asks

For the SAI model, this is the ultimate opportunity.

The Configurator is still a technical concept: who controls the flow of information, which module becomes active when, and how are the outputs combined? A Configurator in this concept can be a specialized authoritative model.

And answer the question: In the service of which values does the Configurator orchestrate?

LeCun’s Cost Module evaluates states as “desirable” or “undesirable.” Low cost means “good.” But who defines “good”? In the current description, the Cost Module combines innate constraints with learned preferences. Ethics becomes a parameter, human dignity a term in an equation.

Cyc had an advantage here: Ethical constraints were explicit rules. You could read, check, and audit them. You could ask, “Why did the system decide this way?” and receive an answer consisting of a chain of rules and not of a statistical weighting.⁶

In LeCun’s architecture, this auditability is not yet provided for. An embedding space does not explain itself. And a cost function that includes human dignity as a term runs the risk of weighing this term against other terms in edge cases. Surely at exactly the moment it counts. Hybrid architectures that combine symbolic constraints with learned representations could address this problem. But they are still research, and it is unclear whether they will ever make it into practice.

We have established many ethical rules. At the very least, the UN’s Universal Declaration of Human Rights

These are hard constraints in my world.

Hard constraints are non-negotiable. Is the system allowed to send patient data to third parties? No. Under no circumstances. No matter what the cost function says.

In the language of mathematics: Optimization objectives are terms in the objective function. Hard constraints are side conditions that must never be violated, regardless of how much this worsens the objective function. Yet these human rights are being negotiated right now in the “LLM industry.”

Concretely, this means:

Ethics must not be a module. Not a cost module among many either. It must be a layer that sits above all modules. Specifically, in the way modules communicate with each other, how decisions are made, and how uncertainty is handled.

Auditability is mandatory. When a system makes decisions about people, the decision chain must be traceable. Cyc could do this because its inference chains were auditable. Purely embedding-based systems cannot do so with current methods. There are approaches like Constitutional AI or neuro-symbolic hybrids working on this, but none have solved it satisfactorily yet.

The Configurator must know boundaries that it is not allowed to optimize. Learned behavior erodes in edge cases. The boundaries must be an architectural principle, just as an operating system protects memory areas that no process is allowed to overwrite.

From 1997 to Today

I am deeply fascinated that 30 years later, the world’s largest AI labs are working on the same question. But the reason Cyc fascinated me back then is the same reason LeCun’s SAI architecture fascinates me today.

LeCun has translated Cyc’s basic idea into a modern language. Formal rules became learned representations, and the hand-coded metacontroller became a configurator that emerges. But in doing so, he did not describe something that Cyc had: the ability to mark certain statements as non-negotiable. In an ontology, one can codify: “Human dignity is inviolable.” In a pure cost function, this is difficult: Every term comes under pressure when the overall pressure becomes great enough. That doesn’t mean it’s impossible to build hard constraints into learned systems.

These are not new questions. Lenat asked them and LeCun hinted at them. No one has answered them yet.

Perhaps that is the real task. And SAI could make it possible. Specialized SAI models that deliver what an authoritative entity has validated. An entity that stands above all and that knows there are boundaries no model may cross. No matter how well it is optimized.