Deontological Logic and AI

The Problem Is Already Here

A self-driving car travelling at highway speed detects an unavoidable collision scenario: continue forward and strike an infant on the road, or swerve and hit an elderly pedestrian on the sidewalk. No human hand is on the wheel. No gut instinct fires. Only an algorithm decides. This is not a thought experiment. It is a live engineering problem that autonomous vehicle manufacturers are solving right now, through the moral values they embed in their training pipelines. The question is not whether AI will make moral decisions. It already does. The question is which ethical framework should govern those decisions, expressed with enough formal rigour that an AI system can actually be trained and verified on it.

This post argues for deontology, and specifically for encoding a hierarchy of duties in which the prohibition against killing is a stricter and more inviolable rule than the obligation to save lives. It also introduces the branch of symbolic logic that makes this idea precise enough to use in practice: deontic logic.

What Is Deontology?

Deontology, from the Greek deon meaning duty, holds that certain actions are intrinsically right or wrong regardless of their consequences. This stands in direct opposition to consequentialism, which evaluates an action solely by its outcomes. Immanuel Kant, the framework’s most influential architect, grounded morality in the idea that rational beings must be treated as ends in themselves, never merely as instruments toward someone else’s benefit. Killing one person to produce a better aggregate outcome violates that person’s dignity as an autonomous end, regardless of the arithmetic.

The philosopher W.D. Ross refined deontology into a theory of prima facie duties: moral obligations that bind by default and can be overridden only by a genuinely stronger competing obligation. Ross identified non-maleficence (the duty not to harm others) and beneficence (the duty to promote the good of others) as distinct duties, and he was explicit that non-maleficence is the weightier of the two. As he wrote in The Right and the Good: “the primary duty here is the duty not to harm others” (RG 22), and “it is not permissible to harm one person to prevent two other people from being harmed” (RG 22; FE 75). This gives us the asymmetry the entire argument rests on: the duty to refrain from harming someone is stricter than the duty to go out of your way to save them.

Deontic Logic: The Formal Language of Duty

Deontology is a philosophical framework. Deontic logic is its mathematical implementation. It is a branch of modal logic that was formally established as an academic discipline by Georg Henrik von Wright in 1951 and provides a symbolic language for expressing obligations, permissions, and prohibitions in a way that can be reasoned about, checked for consistency, and, crucially, implemented in software.

The three core operators of Standard Deontic Logic (SDL), the most widely cited formalization, are the following. $\mathsf{O}(p)$ O(p) reads as “it is obligatory that $p$ p.” $\mathsf{P}(p)$ P(p) reads as “it is permitted that $p$ p.” $\mathsf{F}(p)$ F(p) reads as “it is forbidden that $p$ p.”

These are not independent: permission and prohibition are both definable from obligation. Specifically:

$\mathsf{P}(p) \leftrightarrow \neg\mathsf{O}(\neg p)$ P(p)↔¬O(¬p)

In plain English: something is permitted if and only if it is not obligatory that it not happen. And:

$\mathsf{F}(p) \leftrightarrow \mathsf{O}(\neg p)$ F(p)↔O(¬p)

Something is forbidden if and only if it is obligatory that it not happen. These two definitions mean the entire normative universe can be built from a single primitive operator, $\mathsf{O}$ O.

The axioms of SDL are formally stated as follows (von Wright, 1951; Stanford Encyclopedia of Philosophy). The first is the Distribution Axiom, known as Axiom K:

$\mathsf{O}(p \rightarrow q) \rightarrow (\mathsf{O}p \rightarrow \mathsf{O}q)$ O(p→q)→(Op→Oq)

This says: if it is obligatory that $p$ p implies $q$ q, and it is obligatory that $p$ p, then it is obligatory that $q$ q. In other words, all logical consequences of what is obligatory are themselves obligatory.

The second is the Deontic Consistency Axiom, known as Axiom D:

$\mathsf{O}p \rightarrow \mathsf{P}p$ Op→Pp

This says: if something is obligatory, it must also be permitted. You cannot be required to do something that is simultaneously forbidden. This axiom rules out logically self-contradictory normative systems, which is a crucial property when verifying AI behaviour.

The third is the Rule of Necessitation:

$\text{If } \vdash p, \text{ then } \vdash \mathsf{O}p$ If ⊢p, then ⊢Op

This says: if $p$ p is a logical tautology, then it is obligatory. Any action that is logically necessary is also normatively required.

A further derived rule, known as Rule RM, states:

$\text{If } \vdash p \rightarrow q, \text{ then } \vdash \mathsf{O}p \rightarrow \mathsf{O}q$ If ⊢p→q, then ⊢Op→Oq

If $p$ p logically entails $q$ q, then the obligation to do $p$ p entails the obligation to do $q$ q. This is essential for systems that need to derive downstream duties from a core set of stated rules.

Encoding the Car’s Moral Rules in Deontic Logic

Now we can start translating the autonomous vehicle problem into formal notation. Let us define our propositions. Let $K$ K stand for “the vehicle kills a person,” let $S$ S stand for “the vehicle saves a person,” and let $A$ A stand for a specific available action the vehicle can take.

The first-tier rule, the absolute prohibition, is stated as:

$\mathsf{F}(K) \quad \equiv \quad \mathsf{O}(\neg K)$ F(K)≡O(¬K)

It is forbidden for the vehicle to kill a person. Equivalently, it is obligatory that the vehicle not kill a person. This is a hard constraint encoded at the highest level of the decision architecture.

The second-tier rule, the positive duty, is stated as:

$\mathsf{O}(S)$ O(S)

It is obligatory that the vehicle save persons when it can. This positive duty operates within the space of actions not ruled out by the first-tier prohibition.

Now consider the conflict scenario. The vehicle faces a situation where every available action $A_i$ Ai either results in $K$ K or fails to bring about $S$ S. In SDL, the first thing to notice is that the axiom $\mathsf{O}p \rightarrow \mathsf{P}p$ Op→Pp (Axiom D) protects us from an impossible obligation. If saving a life in a given scenario logically requires killing someone else, we cannot derive a coherent obligation to do so, because that would simultaneously generate $\mathsf{O}(K)$ O(K) and $\mathsf{F}(K)$ F(K), a direct contradiction. The system’s formal consistency requirement prevents the lower-priority duty from overriding the higher-priority prohibition.

This is expressed by a priority ordering over conflicting norms. Using the notation $\succ$ ≻ to represent “is stricter than,” we state:

$\mathsf{F}(K) \succ \mathsf{O}(S)$ F(K)≻O(S)

The prohibition against killing is strictly stronger than the obligation to save. When both cannot be jointly satisfied, the obligation to save is the one that is overridden, not the prohibition against killing. This mirrors Ross’s explicit claim that non-maleficence outweighs beneficence, now stated in machine-readable symbolic form.

Why SDL Needs an Extension: Conditional Obligations

Standard SDL has a well-known limitation for real-world scenarios: it struggles with conditional obligations, cases where what you ought to do depends on the specific situation you are in. The autonomous vehicle dilemma is precisely such a case: obligations change as the physical environment evolves in real time.

SDL can be extended with a dyadic (two-argument) operator $\mathsf{O}(p \mid q)$ O(p∣q), which reads as “ppp must give that $q$ q is the case.” This allows us to write, for instance:

$\mathsf{O}(\text{brake} \mid \text{obstacle detected})$ O(brake∣obstacle detected)

It is obligatory to apply braking given that an obstacle has been detected. And more specifically for the dilemma:

$\mathsf{O}(\neg K \mid \text{collision imminent})$ O(¬K∣collision imminent)

Even when a collision is imminent, the prohibition against killing remains obligatory. The conditional form is essential because it allows obligations to be activated and updated as the vehicle’s sensor state changes over time, which is exactly the temporal obligation structure that Shea-Blymyer and Abbas (2021) demonstrated can be model-checked on weighted transition systems for self-driving car controllers (DOI: 10.1145/3460975).

Why the Black Box Makes Consequentialism Dangerous

With the formal tools in place, we can now state precisely why a purely consequentialist approach to the same problem is dangerous. A consequentialist system does not encode explicit obligations. It encodes a utility function $U$ U and instructs the agent to choose the action $A^*$ A∗ that maximises expected utility:

$A^* = \underset{A}{\arg\max} \; \mathbb{E}[U(A)]$ A∗=AargmaxE[U(A)]

The problem is that $U$ U is learned from training data inside a deep neural network whose internal parameters are opaque. If $U$ U was learned on subtly biased data, or generalises incorrectly to novel scenarios, the system can arrive at conclusions that are internally consistent to the optimization process and morally indefensible to any human observer, with no mechanism for external audit. There is no explicit $\mathsf{F}(K)$ F(K) in the model. There is only a weight landscape that may or may not have learned that killing is undesirable, in approximately the right range of circumstances, from the data it was given.

Deontic logic rules, by contrast, are explicit propositions. The constraint $\mathsf{F}(K)$ F(K) is a statement that can be checked, challenged, updated, and formally verified. Shea-Blymyer and Abbas (2021) demonstrated this explicitly: they encoded a subset of Intel’s Responsibility-Sensitive Safety (RSS) proposal for autonomous vehicles into deontic logic (specifically the Dominance Act Utilitarianism logic developed by Horty, 2001), ran a model-checking algorithm over it, and formally derived that certain logical consequences of the RSS rules were undesirable, something that would be completely invisible inside a black-box utility function (DOI: 10.1145/3460975).

What Empirical Research Actually Tells Us About Public Preferences

The MIT Moral Machine experiment gathered 40 million decisions from people in 233 countries on trolley-style autonomous vehicle dilemmas and found that respondents expressed preferences for saving younger lives over older ones, more lives over fewer, and people crossing legally over those crossing illegally (Awad et al., 2018, DOI: 10.1038/s41586-018-0637-6). This result has frequently been cited to justify building consequentialist trade-offs into AI moral decision-making.

However, a direct follow-up study published in the same journal found that these preferences were an artefact of the forced-choice trolley paradigm used in the experiment. When the paradigm was changed, people overwhelmingly expressed a preference for autonomous vehicles that treat all human lives equally, without discrimination by age, gender, or social status (Bigman & Gray, 2020, DOI: 10.1038/s41586-020-1987-4). The framing of a dilemma as a forced numerical trade-off systematically elicits consequentialist responses that do not reflect people’s actual normative preferences when asked more directly. This is strong empirical support for the deontological egalitarian position encoded in our formal rule $\mathsf{F}(K)$ F(K), which makes no distinction between the value of any person’s life.

The Two-Tier System in Practice

Putting it all together, the proposed decision architecture for an autonomous vehicle operates on two formal tiers.

Tier one is the absolute prohibition encoded as $\mathsf{O}(\neg K)$ O(¬K), which holds unconditionally, is never overridden by any other obligation, and is not subject to expected-utility trade-offs of any kind. It applies to every person in the vehicle’s environment equally.

Tier two is the positive duty encoded as $\mathsf{O}(S)$ O(S) subject to the constraint $\mathsf{F}(K) \succ \mathsf{O}(S)$ F(K)≻O(S), meaning the vehicle must act to minimise harm and save lives within the full space of actions that do not violate tier one. Path optimisation, braking trajectories, collision avoidance algorithms, and harm-minimisation strategies all belong here. The vehicle has considerable engineering latitude at this tier. It simply cannot cross the threshold set by tier one to exercise it.

In the infant-versus-elderly-pedestrian scenario, this means the vehicle does not compute a utility trade-off between two human lives. It applies tier-one constraints to both, exhausts every available action within those constraints, and if a collision remains unavoidable despite all of this, the moral responsibility for the outcome shifts from the algorithm to the structural conditions that created an unavoidable collision: road design, speed limits, sensor limitations, and system-level safety margins. This is not a loophole. It is the correct attribution of moral causal responsibility.

Conclusion: Rules That Machines Can Keep

Teaching an autonomous vehicle that it must never kill is not naive idealism. It is the only approach that produces a system whose moral reasoning can be formally stated, model-checked for consistency, and publicly audited. The deontic logic framework gives us the precise symbolic tools to express this: a hard prohibition $\mathsf{F}(K)$ F(K), a weaker positive duty $\mathsf{O}(S)$ O(S), a strict priority ordering $\mathsf{F}(K) \succ \mathsf{O}(S)$ F(K)≻O(S), and a conditional obligation structure that allows these rules to evolve in real time as the vehicle’s sensor state changes.

A utility-maximising system operating inside a black box has none of these properties. Its moral commitments are implicit, unverifiable, and vulnerable to failure modes its designers cannot anticipate or inspect. Failures of commission, where a machine can be attributed direct causal responsibility for a death it deliberately chose to cause, are not just tragic but legally, socially, and politically destabilising.

The infant and the elderly pedestrian both deserve a rule that says: this machine will not decide between you. That rule is not a failure of the technology. It is the technology working exactly as it should.

References

Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J-F., & Rahwan, I. (2018). The Moral Machine experiment. Nature, 563, 59–64. DOI: 10.1038/s41586-018-0637-6

Bigman, Y. E., & Gray, K. (2020). Life and death decisions of autonomous vehicles. Nature, 579, E1–E2. DOI: 10.1038/s41586-020-1987-4

Shea-Blymyer, C., & Abbas, H. (2021). Algorithmic Ethics: Formalization and Verification of Autonomous Vehicle Obligations. ACM Transactions on Cyber-Physical Systems, 5(4), Article 38. DOI: 10.1145/3460975

Ross, W. D. (1930). The Right and the Good. Oxford University Press. [Cited as RG throughout; page references are to the original edition.]

von Wright, G. H. (1951). Deontic Logic. Mind, 60(237), 1–15. [Founding paper of Standard Deontic Logic; SDL notation as presented in the Stanford Encyclopedia of Philosophy entry on Deontic Logic.]