The Cunning Edge

From Architecture to Implementation

In my previous article, we looked at the worst-case scenario facing shipping agents on the Mississippi River: a hurricane making landfall, a flooded corporate inbox, and thousands of dollars in hourly demurrage fees mounting because an update for the Obscure Steamboat was buried in a shared inbox. My solution to this scenario is the Rundown Router. In this article, I will walk you through the technical implementation decisions behind the Rundown Router including a local AI infrastructure, a machine-readable (but human-written) state, and a deterministic Python orchestration layer designed to constrain, validate, and audit every probabilistic decision before it touches the production mail server.

At a high level, the technical implementation looks like this:

The goal here is to build an AI system that can operate within existing infrastructure while keeping organizational policy human-defined, execution deterministic, and AI-driven decisions transparent.

Integrating With Existing Email Infrastructure

Instead of rebuilding the organization’s email system around an AI application, I designed the Rundown Router as an additional processing layer that works with the existing mail infrastructure. This keeps the underlying communication system familiar to users while allowing the AI to observe and act on incoming messages independently. It also has the added benefit of being mail client agnostic — it can route to and receive emails from any mail client.

I used Postfix as the Mail Transfer Agent (MTA) responsible for receiving and delivering messages, with Dovecot providing the underlying mailbox infrastructure where appropriate. The entire email infrastructure is locally hosted, ensuring full control over routing and data flow. The Rundown Router itself is implemented in Python, which handles the orchestration logic between the mail server, local storage, document processing, AI inference, and downstream delivery.

The key integration point is an email alias dedicated to the Rundown Router. Rather than modifying the normal delivery path, Postfix pipes a copy of messages addressed to this alias directly into the Router application. This gives the Router access to the raw email payload while leaving the organization’s existing mailboxes and workflows intact.

This separation is a key part of the system. The Rundown Router does not need to become the organization’s mail server (though that is an option), and the AI does not need to sit in the path of normal email delivery. If the AI model is unavailable, the document-processing pipeline fails, or the Router itself encounters an error, the underlying email infrastructure can continue operating independently.

That creates an important layer of redundancy: the existing inbox remains the system of record for communication, while the Rundown Router operates as an intelligent triage layer alongside it. The AI can improve how information reaches the people responsible for acting on it without becoming a single point of failure for the whole organization’s communication. This also makes the AI layer easily replaceable. The organization can change the local model, modify the processing pipeline, or temporarily disable the Rundown Router without having to redesign its underlying email infrastructure.

Preserve the Original Message First

Before the AI processes an incoming message, the orchestrator writes the raw email to the Local Store to ensure the original message is preserved. This is always the first step because an AI processing failure has potential to become a data-loss event. If the local model is unavailable, an attachment cannot be processed, or a downstream component encounters an error, the original message remains available for investigation and reprocessing.

Preserving the raw message also gives the system a stable source of truth. Rather than relying on whatever information was extracted or transformed during processing, I can always return to the original email and run it through a modified pipeline. This is particularly important when working with unstructured attachments, where improvements to OCR or document extraction may make previously processed messages worth revisiting.

It also keeps the AI pipeline replaceable. The archived message does not depend on a particular model, parser, or extraction method. I can change the processing layer without changing the underlying record of what was received.

This creates a clear boundary between preserving information and interpreting information. The system first guarantees that the original data exists; only then does it allow the AI pipeline to make decisions about what that data means.

The Local AI Processing Layer

For the AI layer, I chose local inference over sending potentially sensitive emails and attachments to a third-party API. This keeps sensitive information within the organization’s private servers while giving me control over the model, inference environment, and system availability.

I evaluated several locally hosted instruction-tuned models and found Qwen particularly effective for the Router. Its reasoning worked well for structured routing tasks, while its vision capabilities allowed the same model to interpret PDFs, scanned documents, and other ambiguous attachments. Using one model for both tasks also simplifies the processing pipeline, eliminating an additional model handoff between document extraction and routing and allowing the system to interpret and reason over the same input within a single AI layer.

I served the model through Ollama, exposing inference through a local API rather than embedding model-specific logic throughout the Python application to keep the architecture modular. The orchestrator controls the workflow and decides when AI processing is needed, while the model handles interpretation of unstructured content. If I replace the model or inference infrastructure, the rest of the application does not need to change.

Represent Organizational Context Separately Using The Rundown

The Rundown is the defining piece of the system because it separates organizational policy from AI interpretation. I initially represented the Rundown as JSON: a structured, machine-readable representation of who is responsible for what. As the system scales into larger organizations, this state can move into a database while preserving the same conceptual workflow.

The important characteristic is that the Rundown remains human-defined and mutable. It can represent standing responsibilities, temporary assignments, exceptions, and time-dependent states without requiring changes to the model or application code.

For example, an assignment might change for a single vessel during a particular period without changing the underlying routing logic. The organizational state changes; the AI system does not.

I deliberately kept these organizational rules out of the model prompt, model weights, and application logic. Instead, the architecture maintains three distinct responsibilities:

• Rundown: Mutable organizational state.
• Model: The interpreter.
• Orchestrator: The executor.

This separation means the organization can change who is responsible for what as often as it wants without retraining the model or rewriting the application. The model interprets incoming information against the current state, while the orchestrator validates and executes the resulting decision.

This makes the system easier to maintain, adapt, and audit — and demonstrates an important principle: the model should interpret organizational context, not own it.

Keep Execution Deterministic

The most important boundary in the Rundown Router is between probabilistic interpretation and deterministic execution. The LLM is responsible for understanding an incoming message and returning a structured routing decision. It does not directly send emails, modify records, or change organizational state.

Once the model returns its decision, the Python orchestrator takes over. It validates that the proposed recipient exists, checks the model’s reported confidence against a defined threshold, and escalates uncertain decisions rather than acting on them automatically.

Only after these checks does the system perform consequential operations such as email delivery and database writes. Errors at any stage are handled by the application rather than delegated back to the model.

This creates a clear trust boundary: the LLM can propose an action, but the application decides whether that action is valid and executes it deterministically.

That distinction is what allows an LLM to participate in a reliable production system without making the LLM itself the system.

Make Every Decision Auditable And Infrastructure Secure

Because the Router makes decisions that affect where operational information goes, I wanted every decision to be traceable after the fact. I implemented a Decision Log that records the context, AI recommendation, and final outcome for each routed message.

For each decision, the system records:

• Message ID — identifies the original message.
• Rundown state/version — captures the organizational context used at the time.
• AI decision — records the proposed routing action.
• Confidence — records how certain the model was.
• Rationale — captures why the model made the recommendation.
• Final action — records what the system actually executed.
• Human override — records when a person changed the AI’s recommendation and why.

This creates a historical record of not just what the system did, but why it did it and what information it had available at the time.

That makes the Decision Log useful beyond troubleshooting. It provides a way to debug failures, evaluate routing performance, identify patterns in model errors, and maintain human oversight as the system operates. Human overrides also create a feedback loop: accumulated decisions and corrections can be analyzed to improve prompts, routing logic, confidence thresholds, and model selection over time.

The result is an AI system where decisions are not only executed, but observable, reviewable, and capable of improving over time.

Hardening the Administrative Stack And Protecting Proprietary Data

An audit log is only useful if it is secure and accessible to the team. Because this system processes high-stakes corporate communications, sending logs to a third-party cloud-managed dashboard or unencrypted external API was not recommended. To maintain strict data sovereignty, I built a dedicated, local administrative frontend.

The dashboard runs on WordPress and is served through NGINX as the public-facing gateway to the PHP-FPM application layer. Because exposing a local administration layer to distributed operations teams introduces an immediate attack surface, I implemented strict defense-in-depth security:

• Custom DNS Architecture: Custom DNS records were configured for the domain to securely manage network traffic, mapping A records to an NGINX proxy and defining an MX record for inbound email, alongside implementing strict SPF, DKIM, and DMARC records to prevent spoofing and verify cryptographic identity. This setup provides essential routing and security, connecting external traffic to the self-hosted infrastructure. 
• Network & Web Application Firewalls: The NGINX layer is fortified with a ModSecurity v3 Web Application Firewall (WAF) to intercept and drop malicious web traffic before it hits the application server. Two layers of firewalls were also implemented — one for the LXC/LXD container running the application (including the MTA discussed above), and one for the computer running the container.
• Application Security: The WordPress layer is wrapped in Wordfence plugins to enforce aggressive rate-limiting, brute-force protection, and strict IP whitelisting.

By wrapping our administrative interface and system backend in a hardened infrastructure layer, operators can safely view decisions, track errors, and intervene when necessary without exposing internal system controls to the open web or risking an open relay in the Postfix mailing administration.

The resulting system is not an LLM connected to an inbox. It is the practical implementation of the architecture introduced in the previous article, with organizational state, AI interpretation, deterministic execution, and infrastructure defense deliberately separated. Keeping these responsibilities distinct allows each layer to evolve independently while maintaining long-term organizational control over the system.