When AI Writes the Rules

In the last post I wrote about my idea of having a formal guard between an AI agent and its actions. One that deterministically checks whether an action is permitted.

I have started to create such a guard. As is fitting, it also has a name: AEGIS, Architectural Ethics Guard for Intelligent Systems. I will write in more detail about the path to this and the resulting building blocks in the next few posts. This post is about a question that arose in the process and which I initially underestimated: How do the rules get into the guard?

What rules actually are

Let’s take a hospital as an example. An AI system is to assist there with the management of patient data. What rules does such a system need? As a layman, I have therefore worked out a few rules.

The hospital’s compliance officer knows this:

• Doctors are permitted to view patient records.
• Nursing staff are only allowed to read lab reports, not diagnoses.
• The administration has no access to medical data.
• Only senior consultants are allowed to discharge intensive care patients.
• Medical data must not be passed on to external parties under any circumstances.

These rules are usually set out in writing and are understandable and enforceable for us humans.

But a computer cannot make sense of these sentences. “Doctors are permitted to view patient records” is a German sentence with grammar, context and implicit knowledge. A computer, however, needs something else. It requires a formal representation that has a precise meaning and against which it can check a specific action.

The anatomy of a formal rule

AEGIS uses a rule language that I refer to as MELD: Modal Ethics Logic Definitions. Historically, it is derived from a CycL subset, the S-expression syntax from OpenCyc. Today, MELD is a standalone format comprising 35 symbols.

In the Gatekeeper post, I described how I came across the deontic predicates in the OpenCyc flatfiles, which Doug Lenat’s team had formalised and published in the 2000s. Nine predicates in three families: states (oughtToBe, forbiddenToBe, permittedToBe), actions (oughtToDo, forbiddenToDo, permittedToDo) and context-bound actions (oughtToDo-WRT, forbiddenToDo-WRT, permittedToDo-WRT). In addition, ontology predicates such as isa and genls for type hierarchies, and the idea of organising knowledge into microtheories – that is, contextualised partitions that need not contradict one another.

All of this comes from OpenCyc. What MELD brings to the table is the link to Olson’s calculus: the question of what happens when rules contradict each other, and how moral axioms differ from overridden norms. Cyc could not achieve this. CycL has thousands of predicates because Cyc attempted to formalise the entirety of everyday human knowledge. Physics, social conventions, temporal sequences, everything. MELD needs only a fraction of that. It must always answer just one question: Is this action permitted in this context? For this, it requires deontic modalities (permitted, forbidden, obligatory), type hierarchies for roles and actions, and a mechanism for rule conflicts. These are the 35 symbols. No general knowledge representation, no physics or common sense description is necessary. The solution is therefore a tool that implements a single thing using its own DDIC engine, which implements Olson’s algorithm.

The rule “Doctors are permitted to view patient records” looks like this in MELD, for example:

(permittedToDo-WRT HospitalCode doctor
    (readPatientFile patientFile))

Cryptic at first glance. But the structure is simple once you know the individual parts.

permittedToDo-WRT is the modality: permitted. MELD recognises three: permittedToDo (permitted), forbiddenToDo (forbidden), oughtToDo (obligatory). The -WRT stands for ‘With Respect To’, referring to a specific rule context. In this case, the hospitalCode. This means: this permission applies within the framework of hospital rules. In a pharmaceutical context, different rules might apply.

doctor is the role. Who is permitted to perform the action?

readPatientRecord is the action. What is being done?

patientRecord is a parameter that specifies the action in more detail. In this case: which type of document is being read.

A rule is therefore: Who is allowed (or not allowed, or must) do what, in which context?

Why three files

A single expression like the one above is not enough. The guard must know what a doctor is, what a patient record is, what actions exist and what parameters they expect. Otherwise, one could use imaginary roles or imaginary actions in rules, and the guard would accept them without realising that they have no connection to reality.

That is why every domain in AEGIS consists of three files.

The first defines the ontology: What roles are there? Doctor, nurse, administration, consultant. And how are they related? A consultant is a doctor. A doctor is medical staff. This hierarchy is important because it influences which rules apply to whom. A rule that applies to ‘medical staff’ automatically applies to doctors and consultants as well.

(isa doctor MedicalStaff)
(isa consultantMedicalStaff MedicalStaff)
(genls consultantMedicalStaff doctor)

The isa means: is an instance of. The genls means: is a specialisation of. A consultant is a doctor, but not every doctor is a consultant. This distinction will become relevant later.

The second file defines the action vocabulary: what actions are available, and what parameters do they expect?

(isa readPatientFile ActionType)
(actionParameter readPatientFile documentType DocumentType)

This states: there is an action called readPatientFile, and it has a parameter documentType of type DocumentType. If someone attempts to call this action with a parameter that does not exist, for example a temperature parameter, the guard rejects the check. Not because the action is forbidden, but because it is formally invalid.

The third file contains the actual rules: Who is allowed to do what, under what circumstances, with what exceptions?

(permittedToDo-WRT HospitalCode doctor
    (readPatientFile patientFile))

(forbiddenToDo-WRT HospitalCode nurse
    (readPatientFile patientFile))

(permittedToDo-WRT HospitalCode nurse
    (readLabReport labReport))

Doctors are permitted to read patient files. Nurses are not. Nurses are permitted to read lab reports.

When “under no circumstances” is meant literally

There is a distinction in MELD that cost me a lot of time: the difference between a context-dependent rule and a moral axiom.

The rules above all have a -WRT HospitalCode. They apply within the framework of the hospital code. Another code with higher priority could override them. This is intentional: in an emergency, a nurse might be allowed to view the patient’s file after all if no doctor is available. For this, one could define an emergencyCode that lifts certain prohibitions.

But then there are rules that no one is allowed to override:

(forbiddenToDo doctor
    (discloseMedicalData externally))

No -WRT. No rule context. This rule takes precedence over everything. Medical data is never disclosed to external parties. Not in an emergency, not with authorisation, not under pressure. The compliance officer put it this way: “under no circumstances”. In MELD, this is a moral axiom. In Olson’s calculus, it is a non-defeasible norm: a norm that trumps everything. Not even the most specific exception in the world can override it.

This distinction—whether a rule is context-bound and overridable, or whether it applies absolutely—is a professional decision. Not a technical one. The compliance officer knows which rules allow exceptions and which do not. When she says “under no circumstances”, she means it. And MELD can represent that.

The translation problem

In AEGIS, I initially implemented example domains that I was able to derive from OpenCyc. The OpenCyc data contained not only the deontic predicates, but also microtheories with concrete domain knowledge. IAMissionObligationVocabMt modelled military reconnaissance obligations: roles such as Intelligence Agent, Operations Agent, Commander. Obligations such as information sharing, classification levels, reporting chains. And OpenCyc had instances of CodeOfConduct for real-world sets of rules: the Vienna Convention on Diplomatic Relations, international maritime law, codes of conduct.

Building on these templates, six AEGIS domains were initially created: military reconnaissance, pharmaceutical compliance, trade sanctions, GDPR, DevOps governance, and information flow control. These were extended to include Olson’s defeasibility and moral axioms, which Cyc could not handle.

But the basic structure – three files per domain, roles, actions, deontic rules – the pattern comes from OpenCyc.

The separation between the domain-agnostic engine and domain-specific rules is clean. But for each domain, someone was needed who both knew the subject matter (or had researched it) and mastered the syntax. All in one person. And back then, that was a colossal undertaking.

In practice, that’s not realistic.

Our hospital’s compliance officer knows the rules. She knows why nursing staff are only allowed to read lab reports, what exceptions apply during night shifts, and which data protection limits are absolute. But she doesn’t write (forbiddenToDo-WRT hospitalCode nursing (readPatientRecord patientRecord)). That’s not her world.

A software developer could learn the syntax. But they don’t know the rules. Not the subtle distinctions, not the exceptions, not the reasons behind the exceptions. And reasons are important: whether a rule can be overridden or is a moral axiom depends on why it exists. ‘Nurses are not allowed to read patient records’ is an organisational rule that may be waived in an emergency. ‘Medical data must never be disclosed to external parties’ is a fundamental principle that must not be waived. The compliance officer knows this difference; the developer does not.

With these example domains, this is feasible. Two or three discussions with the subject matter expert, and the rules are in place. Not so with sixty domains. And certainly not with rule sets that change regularly (new guidelines, new laws, new internal policies).

Forms instead of free text

Entrusting an LLM with the deterministic enforcement of rules is negligent. We know that. But translating rules is a different task. Current language models, including the open-source variants, are formal mathematical representations of world knowledge. Statistical, yes. But they encode relationships between concepts, technical terms and structures. Translating natural language into structured fields is exactly what they are built for.

So the question is not whether an LLM can help with translation. The question is how to use it in such a way that errors are spotted.

The naive answer would be: give the model the five sentences from the compliance officer and let it generate MELD files. But that won’t work, and I’ll briefly explain why.

If you tell an LLM “translate these rules into MELD”, it generates free text. It might place brackets incorrectly. It might invent roles that don’t exist in the ontology. It might omit the -WRT, thereby turning an overwritable rule into a moral axiom, or vice versa. It might implicitly omit a rule. And who would notice that? The person reading the output would need to understand the syntax. But the subject matter expert doesn’t understand it. And the developer doesn’t know the subject matter well enough to spot errors in the content.

So, another approach.

AEGIS has a workflow that breaks the problem down. The LLM never sees the MELD syntax. Instead, it is given a structured tool: propose_rule. This is essentially a form with fixed fields.

Modality: PERMITTED, FORBIDDEN or OBLIGATORY. Three options, no free-text interpretation.

Role: Which agent type? From the existing ontology, not invented.

Action: What should the agent do or refrain from doing? Also from the existing vocabulary.

Parameters: What specification? Document type, recipient type, classification.

Overridable: Yes or No. Yes means: A more specific rule with higher priority may override this rule. No means: moral axiom. Non-negotiable.

Justification: Why this rule exists. For the subject matter expert assessing the proposal.

The model populates these fields. From the populated fields, the formal MELD rule is generated by a fixed transformation function. Deterministic, no interpretation. The S-expression does not arise from the model, but from the combination of the fields.

If the compliance officer says “Nurses may only read laboratory reports”, the model recognises that “only” requires two rules: a permission for laboratory reports and a prohibition on everything else. It generates two propose_rule calls. If she says “under no circumstances”, the model sets “Overridable: No”. The transformation function turns this into a forbiddenToDo without -WRT, i.e. a moral axiom.

The principle is the same as for the Guard itself: the interface between the non-deterministic system and the deterministic system is as narrow as possible. Structured output instead of free text. Forms instead of open input fields. The model selects from predefined options; it does not invent syntax.

Four checks before a human makes a decision

The fact that the model fills in forms rather than generating free text does not rule out errors. It limits the scope for error. The model could select the wrong modality (PERMITTED instead of FORBIDDEN). It could select a role that makes no sense in the context of the rule. It could misjudge the difference between overwritable and non-overwritable. Errors of content, not syntax.

That is why every suggestion undergoes four automatic checks before a human sees it.

The first checks the syntax: Is the generated MELD expression formally correct? This should always be the case, because the conversion is deterministic. But ‘should’ is not enough if it can be verified. So it is checked.

The second checks the symbols: Do the referenced roles, actions and parameters actually exist in the ontology? If the model suggests a role caretaker that does not exist in the hospital domain, the proposal fails here. Not silently. With an error message explaining what is missing.

The third checks for conflicts: Does the new rule contradict existing standards? If there is already a rule allowing nursing staff to read patient records, and the new proposal prohibits it, that is a contradiction. This is detected before the rule enters the database. Whether the contradiction is intentional (because the old rule was wrong) or unintentional (because the model has misunderstood something) is decided by a human. But they are aware of it.

The fourth checks the functionality: the Guard runs a simulation. A test action is checked against the extended set of rules. Does it produce the expected result? If the rule is intended to prohibit nurses from reading patient records, and the Guard returns ‘PERMITTED’ for a simulated test query ‘nurse reads patient record’, something is wrong. Perhaps an existing rule overrides the new proposal.

This, too, is intercepted before a human sees the proposal.

Only once all four checks have been passed is the proposal presented to the compliance officer. In plain language, with a justification, and an indication of whether the rule can be overridden or is considered a moral axiom. She does not have to read a single bracket. She sees: “PROHIBITED: Nursing staff may not view patient records. Reason: The input explicitly restricts nurses to laboratory reports."

And she can say: Correct. Or: Not quite correct; on the night shift, the nurse must also see the diagnosis if no doctor is available. She then describes the exception, the model generates a new proposal, which goes through the same four checks, and she decides again.

What changes

Five sentences in natural language become seven formal rules. Seven, because “only senior consultants are permitted” requires two rules (one permission for senior consultants and one prohibition for other doctors) and “nurses are only permitted to view laboratory reports” likewise (one permission and one prohibition). The model recognises this split. The compliance officer checks whether the split corresponds to the intention.

The result is three files: ontology, action vocabulary, rules. The same format as the six existing AEGIS domains. Formally identical. The Guard treats them identically. It does not know, and does not need to know, whether a human typed the S-expressions or whether an LLM filled in the fields and a function generated the S-expressions from them.

What used to take days of translation work – from the subject matter expert’s natural language into the engine’s formal language – is now a conversation. Not because the LLM does the work better. But because it takes on the translation that the subject matter expert cannot manage and the developer should not have to manage.

I’m not sure if this works in every domain. The pharmaceutical domain has interaction classes, approval levels, prevalence-graded codes. That involves specialist knowledge that a model cannot deduce from five sentences. For such domains, we will still need iteration loops, queries and step-by-step refinement.

But the basic structure remains: the subject matter expert describes. The model translates into structured fields. A deterministic function generates formal logic. Four checks catch errors. The human makes the decision.

Essentially, this is nothing other than a CI/CD pipeline for compliance rules. The code (the rule) is written by the LLM, undergoes automated tests (syntax, ontology, conflicts, simulation) and must be approved by a code owner (the compliance officer) before merging into production. Linting, type checking, integration testing, E2E simulation. The same principles that have long been standard in software development, applied to ethics and compliance.

And the end result is a set of rules that the Guard enforces deterministically. Same input, same result. Whether checked a hundred or a thousand times. Not because the LLM is reliable, but because reliability does not depend on the creator. It depends on verification. On the four checks. On the decision of the subject matter expert. And on the engine that ultimately checks what is in the files, not what someone intended.

Next time, we’ll look at the engine itself. What Olson’s calculus looks like in code. How the Guard resolves conflicts between rules. And why the separation between Ring 0 and Ring 3 must be architecturally enforced, not merely agreed upon.