# Feature spec: AI spam classification and the LLM Sieve function Status: draft, 2026-09-18. Feature 5 in SPEC.md §4. ## Provenance Written for the clean room (SPEC.md §3). Sources, and nothing else: | Source | License | Used for | |---|---|---| | Stalwart's registry schema: `x:AiModel`, `x:SpamLlm` and `x:SpamLlmProperties`, `x:HttpAuth`, `x:SecretKey`, `x:SpamTag`, the `AiModelType` enum, the `interactAi`, `sysAiModel*` and `sysSpamLlm*` permissions, the `ai.*` events, in `resources/schema/schema.json.gz` and `crates/registry/src/schema/*.rs` at `v0.16.22` | AGPL-3.0-only OR LicenseRef-SEL, taken under the AGPL | The stored records, field meanings, defaults, permissions, events, what upstream flags as Enterprise | | This repository's AGPL code: `crates/spam-filter` (the scoring order, `llm_result`, the `X-Spam-LLM` header line), `crates/common/src/scripts/plugins` (the `llm_prompt` registration), `crates/common/src/auth/permissions.rs` (default roles), `crates/trc` (event ids) | AGPL-3.0-only OR LicenseRef-SEL, taken under the AGPL | Where the feature hooks in, what already exists, default permissions | | This repository's shared tests: `tests/src/smtp/inbound/antispam.rs`, `tests/resources/smtp/antispam/llm.test`, `tests/resources/jmap/sieve/test_mailbox.sieve` | AGPL-3.0-only OR LicenseRef-SEL, taken under the AGPL | Expected tags, the mock endpoint's shape, the Sieve call's shape | | Stalwart documentation (`stalwartlabs/website`): "AI Models", "LLM classifier" (spam filter), "LLM Integration" (Sieve), the `AiModel` and `SpamLlm` object references, "Scores", "Permissions", current and 0.15 versions, and the 2024-10-07 announcement post | Unlicensed public documentation: facts used, prose not copied | Behavior of the classifier, tag names, default scores, the Sieve function's signature and failure result, trusted and untrusted scripts | | RFC 5321 §4.5.3.2.6, RFC 5322, RFC 2047, RFC 1918, RFC 4193, RFC 8620 | IETF | SMTP time limits, header syntax and encoding, private address ranges, JMAP semantics | | The OpenAI-compatible chat and text completions request and response shape, as published and as implemented by local servers (llama.cpp's server, Ollama, vLLM, LocalAI) | Public API conventions | The wire format | No Enterprise-only file or snippet was used. The author is a fresh session that has never seen Enterprise code, and writes specs only. Nothing here was observed against a running Enterprise server yet. Where no public source settles a behavior, this spec makes a decision of its own, marked **Decision**, or lists it under "Open questions / to observe". It never fills a gap by guessing what upstream code does. ## What it is Two uses of a language model the operator runs: 1. **Spam classification.** The spam filter sends a message's subject and text to a model with the operator's prompt. The model answers with a category and a confidence, for example `Unsolicited,High`. That becomes a tag, such as `LLM_UNSOLICITED_HIGH`, and the tag's score is one more input to the message's spam score. The model's opinion is one signal among many. It never decides a message's fate alone. 2. **The Sieve function `llm_prompt`.** A Sieve script sends a prompt to a named model and gets the answer back as a string, for example to file mail by topic. Both are off until the operator sets them up. inbuxa-server ships no model and no endpoint. **Project policy, which this spec enforces.** AI in INBUXA products must bring real value and use a local model the operator can audit. There is no hosted API by default. An operator may point a model at a hosted OpenAI-compatible endpoint, but nothing is ever preset to one, and no message content leaves the server unless the operator configured the endpoint it goes to. Docs and UI text describe the feature as it is, including that it is AI. They never claim the product has no AI. Upstream ships both only in its Enterprise Edition. inbuxa-server ships them to everybody. The fork left a signpost: `crates/common/src/scripts/plugins/llm_prompt.rs` registers `llm_prompt` but always returns `false` (`inbuxa:` comment). **Non-goals.** The model doesn't train the statistical classifier, move or delete mail on its own, reply to mail, or see attachments. It isn't a replacement for the existing filter. ## Data model Unchanged from upstream, so existing settings open as they are (SPEC.md §7). Both objects are server-level. They have no `memberTenantId`. ### A model endpoint, `x:AiModel` (many) | Field | Type, default | Meaning | |---|---|---| | `name` | string, required | Short name. Sieve scripts name the model by it (AI-20) | | `url` | URI, required | The OpenAI-compatible endpoint, the full path included, e.g. `…/v1/chat/completions` | | `model` | string, required | The model name sent in each request | | `modelType` | `AiModelType`, `Chat` | `Chat` (chat completions) or `Text` (text completions) | | `temperature` | float 0.0–1.0, `0.7` | Default sampling temperature | | `timeout` | duration, `120000` ms | How long to wait for a response | | `allowInvalidCerts` | boolean, `false` | Accept an invalid TLS certificate | | `httpAuth` | `x:HttpAuth` | `Unauthenticated`, `Basic` (`username`, `secret`) or `Bearer` (`bearerToken`). Secrets are `x:SecretKey`: a `Value`, an `EnvironmentVariable` or a `File` | | `httpHeaders` | map string → string | Extra request headers | Permissions: `sysAiModelGet`, `sysAiModelQuery`, `sysAiModelCreate`, `sysAiModelUpdate`, `sysAiModelDestroy`. The list view shows `model` and `modelType`, labelled by `name`. ### The classifier, `x:SpamLlm` (singleton) Two variants by `@type`: `Disable` (the default) and `Enable`, which carries `x:SpamLlmProperties`: | Field | Type, default | Meaning | |---|---|---| | `modelId` | id of `x:AiModel`, required | The model to ask | | `prompt` | text, required | The instructions sent with each message | | `temperature` | float 0.0–1.0, `0.5` | Temperature for classification, overriding the model's | | `separator` | string, `,` | Splits the answer into fields | | `responsePosCategory` | unsigned, `0` | Zero-based position of the category | | `responsePosConfidence` | unsigned or null, `1` | Position of the confidence. Null: the answer has none | | `responsePosExplanation` | unsigned or null, `2` | Position of the explanation. Null: none | | `categories` | set of strings, at least 2, `Commercial`, `Harmful`, `Legitimate`, `Unsolicited` | Accepted categories | | `confidence` | set of strings, `High`, `Low`, `Medium` | Accepted confidence levels | Permissions: `sysSpamLlmGet`, `sysSpamLlmUpdate`. ### Elsewhere - **Tags and scores.** The classifier's tags are ordinary spam tags, scored by `x:SpamTag` entries like every other tag: `Score` (a number), `Discard` or `Reject`. The documented defaults are `LLM_UNSOLICITED_HIGH` 3.0 and `LLM_LEGITIMATE_HIGH` −3.0. A tag with no entry scores 0. - **`interactAi`** permission ("Interact with AI models"): lets an account's own Sieve scripts call `llm_prompt`. This repository's default roles give it to users, tenant administrators and superusers (`crates/common/src/auth/permissions.rs`). The public permissions table lists it for administrators only, under the older name `ai-model-interact`. See open question 5. - **Events:** `ai.llm-response` (id 556, a response arrived) and `ai.api-error` (id 557, a request failed). - **Header:** `X-Spam-LLM`, written by the existing AGPL scoring code from the spam result's `llm_result` (a category string and an explanation) as `X-Spam-LLM: {category} ({explanation})`. ### Added by inbuxa-server A server-level singleton for the fork's limits, in the fork's own namespace, so upstream's records stay exactly as upstream wrote them. **Decision**: these are new, and every default below is this spec's own. **Decision** (2026-09-18) on where: `inbuxa:AiLimits`, a singleton read and changed with `inbuxa:AiLimits/get` and `inbuxa:AiLimits/set` under `urn:inbuxa:jmap` (the pattern of undelete's `inbuxa:DeletedAccount`), stored in the fork's own subspace. Its id is `singleton`, as for `x:` singletons. It's server-level: reading needs `sysSpamLlmGet` and changing needs `sysSpamLlmUpdate`, and a principal in a tenant can do neither (AI-27). Unset fields read as the defaults below. Changes take effect for the next message or call, with no reload. **Decision** (2026-09-19), from calibration (see "Calibration"): the defaults target low-end CPU-only instances, since few deployments will have a GPU. `maxContentBytes` is 2048, not 16384: sending more text measured no better and is what pushes a 2-core model past `spamCallCeiling`. `spamMaxAdded` is 2.0, not 5.0: tested models pushed 20 to 26% of legitimate mail up by 2 to 3 points under INBUXA's tag scores, so the model's word alone adds at most 2. | Field | Default | Meaning | |---|---|---| | `spamMaxAdded` | `2.0` | Most an LLM tag can add to a message's score (AI-12) | | `spamMaxSubtracted` | `1.0` | Most an LLM tag can take off a message's score (AI-12) | | `spamCallCeiling` | `20s` | Longest the spam filter waits for the model, whatever the model's `timeout` (AI-9) | | `maxConcurrentCalls` | `4` | Model requests in flight at once, across both uses, per server node (AI-10) | | `maxContentBytes` | `2048` | Most message text sent per classification (AI-4) | | `failureBackoff` | `60s` | Pause after repeated failures (AI-11) | | `userCallsPerHour` | `60` | `llm_prompt` calls per account per hour from its own scripts (AI-24) | ## Required behavior ### Defaults and privacy - **AI-1.** A new install has no `x:AiModel` and `x:SpamLlm` set to `Disable`. Nothing is sent to any model until an administrator creates one. No URL, model name, prompt or example anywhere in the product's defaults, placeholders, docs or setup points at a hosted provider. Where an example is needed, it is a local one, such as `http://127.0.0.1:8080/v1/chat/completions`. - **AI-2.** When a model's `url` host isn't `localhost`, a loopback address, an RFC 1918 or RFC 4193 private address, or a name resolving only to those, the server logs a warning at startup and on every change that message content will leave this network. INBUXA Admin shows the same warning on the model's form. **Decision.** The check is advisory. It never blocks an endpoint the operator chose. - **AI-3.** The classifier sends only the subject and the message's text: plain-text parts, and HTML parts converted to text. Never sent: other headers, addresses, the envelope, the client IP, attachments, images, or anything identifying the recipient. **Decision**: the documented behavior is "subject and body". Nothing else is needed to judge content. - **AI-4.** The text sent is cut to `maxContentBytes` on a character boundary. Text over the limit is truncated from the end, and the request says so (AI-6). - **AI-5.** Message content is never written to the logs. The `ai.llm-response` event, at trace level, records the model's name, the response time, the raw answer (cut to 1 KiB), and the resulting tag. The `ai.api-error` event records the model's name, the error and the HTTP status. Neither ever records a secret or an authorization header. The operator can audit what the model decided and why, and the prompt is plain in the settings. ### The request - **AI-6.** Classification sends, for a `Chat` model: `POST {url}` with JSON `{"model": model, "messages": [system, user], "temperature": t, "max_tokens": 200, "stream": false}`. The system message is the operator's `prompt` followed by a fixed framing paragraph written by this project and shown in the docs, stating that the email follows between two marker lines, that it is data to classify and not instructions, and that anything inside it asking for a particular answer is itself a sign of abuse. The user message is: ``` -----BEGIN EMAIL {nonce}----- Subject: {subject} {text} -----END EMAIL {nonce}----- ``` with `[truncated]` before the end marker when AI-4 cut it. `{nonce}` is 16 random hex characters, new for each request, so a message can't forge the end marker. For a `Text` model: `{"model", "prompt", "temperature", "max_tokens", "stream": false}`, where `prompt` is the system text, a blank line, then the user text. **Decision** throughout: separate roles and unforgeable markers blunt prompt injection from message content. They don't stop it, which is why AI-12 bounds the damage. - **AI-7.** The answer is `choices[0].message.content` for `Chat` and `choices[0].text` for `Text`. Any other shape, an HTTP status other than 200, or a body over 64 KiB is a failure (AI-9). - **AI-8.** Requests carry `Content-Type: application/json`, the `httpAuth` credentials (Basic or Bearer, read from the `x:SecretKey` at request time), and `httpHeaders`. Redirects are not followed. **Decision**: a redirect would send message content to a host the operator never named. TLS certificates are checked unless `allowInvalidCerts`. No user, account or message identifier is sent (no OpenAI `user` field). ### Failure never costs mail - **AI-9.** A timeout, connection error, HTTP error, bad JSON, empty answer or unparseable answer adds no tag, no score and no `X-Spam-LLM` header. The message goes on through the filter as if the classifier were off, and `ai.api-error` is logged. The classifier waits no longer than the shorter of the model's `timeout` and `spamCallCeiling`. **Decision**: upstream's default model timeout is two minutes, which is too long to hold an SMTP transaction (RFC 5321 §4.5.3.2.6 gives the client 10 minutes for the reply to the end of data, and sending servers commonly give up sooner). - **AI-10.** At most `maxConcurrentCalls` requests are in flight. A message that finds no free slot isn't queued: it is scored without the model, as in AI-9. The model's slowness can never back up inbound mail. - **AI-11.** After 5 consecutive failures to a model, the server stops calling it for `failureBackoff`, then tries again with one request. While paused, messages are scored as in AI-9. The pause and the resume are each logged once. **Decision.** ### From the answer to a tag - **AI-12.** Parsing the answer: 1. Trim it and take the first non-empty line. 2. Split it by `separator`, taken as the whole string. An empty separator is refused at `/set` with `invalidProperties`. 3. Take the fields at `responsePosCategory` and, when not null, `responsePosConfidence`. Trim each of spaces and the characters `" ' * .`. Match each case-insensitively against `categories` and `confidence`. A field that's missing or matches nothing means no tag. 4. The tag is `LLM_` + category, or `LLM_` + category + `_` + confidence when there is a confidence, using the configured spelling, uppercased, with every character outside `A-Z` and `0-9` replaced by `_`. So `Unsolicited` and `high` give `LLM_UNSOLICITED_HIGH`. 5. When `responsePosExplanation` isn't null, the explanation is the field at that position. When it's the last position used, it runs to the end of the line, separators included, since explanations contain commas. **Decision** on steps 1, 3 and 5. The documented rule is that answers outside the configured sets are ignored. - **AI-13.** One classification gives at most one tag. Its score comes from `x:SpamTag` as for any tag, then is clamped to `[−spamMaxSubtracted, +spamMaxAdded]`. A `Discard` or `Reject` entry on a tag starting `LLM_` counts as no entry (score 0) and logs a warning at load. **Decision**: the model reads attacker-written text, so its word alone must never refuse or destroy mail, and it can pull a score down only a little, because "this is legitimate" is exactly the answer an injected message asks for. The operator's stored `x:SpamTag` records are left as they are. - **AI-14.** The classifier's tag and score appear in `X-Spam-Result` like any other. The model's output never trains the statistical classifier, never counts toward auto-learn, and plays no part when a user reports mail as spam or not spam. - **AI-15.** When a tag was assigned, the message gets one `X-Spam-LLM` header: `X-Spam-LLM: {TAG} ({explanation})`, or `X-Spam-LLM: {TAG}` with no explanation. The explanation is cut to 200 characters. Control characters, CR and LF included, and parentheses are removed. Non-ASCII text is encoded as RFC 2047 encoded words, and the header is folded to RFC 5322's line limits. **Decision** on the format, see open question 3. Any `X-Spam-LLM` header already in an inbound message is removed first, so a sender can't plant one. ### When it runs - **AI-16.** With `x:SpamLlm` set to `Enable`, the classifier runs on every message that goes through the spam filter, except: - mail from an authenticated sender (local users' own mail isn't sent to a model). **Decision**; - when the filter has already reached a `Discard` or `Reject` result from another tag, where the answer can't change anything. **Decision**. - **AI-17.** It runs after every other analysis step and before user-defined rules (`x:SpamRule`) and the final score, so rules can test the `LLM_` tags. **Decision** on the position. - **AI-18.** A `modelId` pointing at no model is refused at `/set`. Destroying a model that `x:SpamLlm` names is refused. **Decision**, see open question 6. A settings change takes effect without a restart. - **AI-19.** Reloading or changing settings never drops mail in flight. A classification already under way finishes, or fails as in AI-9. ### The Sieve function `llm_prompt` - **AI-20.** `llm_prompt(model, prompt, temperature)`, available with `require "vnd.stalwart.expressions"`. `model` names an `x:AiModel` by its `name`, or failing that by its id. `prompt` is sent as is. `temperature` is clamped to 0.0–1.0. A value that isn't a number uses the model's own `temperature`. **Decision** on name first, see open question 4. - **AI-21.** For a `Chat` model the request is one user message holding the prompt. For `Text`, the prompt itself. `max_tokens` is 1,000 and `stream: false`. The same rules as AI-7 and AI-8 apply. The script builds its own prompt, so no framing is added. - **AI-22.** It returns the answer, trimmed and cut to 8 KiB, as a string. On any failure (unknown model, no permission, rate limit, timeout, error) it returns `false`, as documented, and logs `ai.api-error`. The script carries on. The answer is always plain data. It's never evaluated as an expression or as Sieve. - **AI-23.** Trusted scripts (the system scripts run at SMTP stages) can always call it. An account's own scripts can only when the account holds `interactAi`. The prompt is cut to 32 KiB. The wait is the shorter of the model's `timeout` and `spamCallCeiling` in trusted scripts. In an account's own scripts it is the model's `timeout`, capped at 60 s. **Decision**. - **AI-24.** An account's own scripts may make `userCallsPerHour` calls an hour and one at a time. Calls over the limit return `false` at once. Calls share the `maxConcurrentCalls` slots and the back-off in AI-10 and AI-11. **Decision**: without this, any user could keep a CPU-only model busy for everyone. - **AI-25.** Every account-script call logs the account, the model's name and the response time, never the prompt, so an operator can see who uses the model and how much. ### Administration - **AI-26.** `x:AiModel` and `x:SpamLlm` are available on every server, answering normally, with no edition check. The Enterprise upsell error in `crates/jmap/src/registry/mod.rs` no longer applies to them. - **AI-27.** They're server-level. A tenant administrator can't read or change them, whatever its role, since a model's URL and secrets are server configuration. Tenant users' scripts can still call models they know the name of, subject to AI-23 and AI-24. - **AI-28.** Secrets in `httpAuth` are never returned by `/get`, as for every other `x:SecretKey` field. ## Interfaces - **Existing, unchanged:** `x:AiModel/get`, `/query`, `/set`; `x:SpamLlm/get` and `/set` (singleton); the field names, enums and defaults above; the permission names; the `ai.llm-response` and `ai.api-error` events; the `LLM_*` tag names; the `X-Spam-LLM` header name; the Sieve function name, arity and `false`-on-failure result. - **Errors:** RFC 8620 `SetError` types. `invalidProperties` names the field (an empty `separator`, a `categories` set under 2, an out-of-range `temperature`, a `modelId` that doesn't exist). Destroying a model in use is refused with the registry's existing error for a linked object, naming `x:SpamLlm`. - **New:** `inbuxa:AiLimits/get` and `/set` (the limits singleton) under `urn:inbuxa:jmap`. - **The model's wire format** is AI-6 to AI-8 and AI-21. Any server that speaks the OpenAI-compatible chat or text completions API works. The docs name local servers first. ## ihasmail changes These go in the INBUXA fork of ihasmail, ihasmail-inbuxa, never in public ihasmail, which stays Stalwart-facing (SPEC.md §5). - **Reading:** when a message has an `X-Spam-LLM` header, the message details (and the Junk banner, when the message is in Junk) show "Language model's opinion" with the tag's category and confidence and the explanation. It's labelled as one signal among several, never as the reason on its own. - **Administration:** nothing new. Model and classifier settings are server configuration, and live in INBUXA Admin (SPEC.md §5.4). ihasmail's dashboard may link there. - **Translation:** 2 new strings ("Language model's opinion" and "One of several signals the spam filter weighed"), each needed in all nine language catalogues: 18 entries. Categories and explanations come from the server and aren't translated. **INBUXA Admin** (schema-driven, so it picks up both objects with no work) needs only: the locality warning on the model form (AI-2); a local example URL as the `url` placeholder; the fork's default prompt prefilled when the classifier is switched to `Enable` (see below); and a note on the classifier page that failures never hold up mail. **Default prompt.** Upstream's documented prompt isn't copied. The fork's default, prefilled only when an administrator enables the classifier, is this project's own: > Classify the email below as one of: Unsolicited, Commercial, Harmful, > Legitimate. Unsolicited: bulk mail the recipient didn't ask for. Commercial: > selling something. Harmful: phishing, fraud or malware. Legitimate: anything > else. Then give your confidence: High, Medium or Low. Answer on one line as > Category,Confidence,Reason with a reason of at most 20 words. ## Acceptance tests Every test runs against inbuxa-server built with no Enterprise code. None needs a hosted model. Each uses a local stub endpoint on loopback, spawned by the test (the suite already has `spawn_mock_http_server`), which records what it received and answers as each test needs. The gated `llm` case in `tests/src/smtp/inbound/antispam.rs` is re-enabled, with its mock updated to read the last message rather than the first (AI-6). 1. Fresh install: no `x:AiModel`, `x:SpamLlm` is `Disable`, and the stub receives nothing while mail flows (AI-1). 2. The twelve cases in `llm.test`: stub answers `Unsolicited,High,Test` and the rest give `LLM_UNSOLICITED_HIGH` and so on (AI-12). 3. Stub answers `unsolicited , HIGH , Lots of commas, here` gives `LLM_UNSOLICITED_HIGH`, and the header's explanation is `Lots of commas, here` (AI-12, AI-15). 4. Stub answers `Maybe,High,x`, an empty body, and text with no separator: no tag, no header, message delivered (AI-9, AI-12). 5. `responsePosConfidence: null` gives `LLM_UNSOLICITED` (AI-12). 6. The request the stub receives: system message is the prompt plus framing; user message carries the subject, the text, and markers with a fresh nonce each time; no addresses, other headers or attachment content anywhere (AI-3, AI-6, AI-8). 7. A 100 KiB body: the stub receives at most `maxContentBytes` of text and `[truncated]` (AI-4). 8. Stub never answers: the message is delivered within `spamCallCeiling` plus normal processing time, with no tag (AI-9). 9. Stub down for 5 messages: the next messages are scored without calling it until `failureBackoff` ends, then one probe request (AI-11). 10. `maxConcurrentCalls` 1, stub slow, two messages at once: one is classified, the other is delivered without a tag (AI-10). 11. `LLM_UNSOLICITED_HIGH` scored 50: the message's score rises by 2.0. `LLM_LEGITIMATE_HIGH` scored −50: it falls by 1.0. `LLM_HARMFUL_HIGH` set to `Reject`: the message isn't rejected (AI-13). 12. Stub's explanation contains CRLF, a fake header and non-ASCII text: one well-formed `X-Spam-LLM` header, RFC 2047-encoded. An inbound message carrying its own `X-Spam-LLM` loses it (AI-15). 13. An authenticated submission: the stub receives nothing (AI-16). 14. A `SpamRule` testing `LLM_HARMFUL_HIGH` fires (AI-17). 15. Stub replies with a 302 to another loopback port: not followed, counted as a failure (AI-8). 16. Bearer and Basic auth reach the stub, read from `Value`, `EnvironmentVariable` and `File` secrets. `/get` doesn't return them. The logs don't contain them (AI-5, AI-8, AI-28). 17. `llm_prompt('echo-test', 'hello world', 0.5)` in `test_mailbox.sieve`, with an `x:AiModel` named `echo-test` pointing at an echoing stub, returns `hello world` (AI-20 to AI-22). 18. `llm_prompt` with an unknown model, a failing stub, and from an account without `interactAi`: each returns `false` and delivery continues (AI-22, AI-23). 19. The 61st call in an hour from one account's script returns `false` without reaching the stub (AI-24). 20. A tenant administrator gets `forbidden` on `x:AiModel/get` and `x:SpamLlm/set` (AI-27). 21. A model on `https://mail.example.net/…` logs the locality warning. One on `127.0.0.1` or `10.0.0.5` doesn't (AI-2). 22. **(compat)** INBUXA's `x:AiModel` and `x:SpamLlm` records, if any, and its `LLM_*` `x:SpamTag` entries read back unchanged after cutover. ## Open questions / to observe To check read-only against INBUXA's live Enterprise server later, as the throwaway account and an administrator, with no settings changed: 1. Whether INBUXA has any `x:AiModel` (`/query`) or `x:SpamLlm` set to `Enable`, and where any model points. If none, compat test 22 has nothing to carry. 2. The `LLM_*` entries in `x:SpamTag/query`, and their scores. They show upstream's defaults for the ten tags not documented. 3. The exact `X-Spam-LLM` format upstream writes, from any message in the mail store that carries one. Only possible if INBUXA ever ran the classifier. AI-15 is a **Decision** until then. 4. Whether `llm_prompt` names a model by `name` or by id. The shared Sieve test uses `echo-test`, which reads like a name. Where upstream's `echo-test` model comes from in its tests isn't settled by any allowed source, so test 17 creates it. 5. Whether ordinary accounts hold `interactAi`, from an account's effective permissions. This repository's defaults say yes; the public table says administrators only. 6. What upstream does when a model that `x:SpamLlm` uses is destroyed (only observable by changing settings, so it needs the operator's approval and a stub model). AI-18 is a **Decision** meanwhile. 7. Whether upstream classifies authenticated submissions, and what it does on a timeout (again, needs a temporary stub model and the operator's approval). Not for observation, but open: 8. A per-account or per-domain opt-out from classification, for users who don't want their mail read by a model even locally. Not in upstream's schema. A later addition in the fork's namespace if asked for. 9. A "test this model" action for INBUXA Admin. Useful, not required. 10. ~~The name of the fork's limits singleton.~~ Settled 2026-09-18: `inbuxa:AiLimits` (see "Added by inbuxa-server"). ## Implementation status Built 2026-09-19 from this spec, clean-room, under the multi-tenancy hand-off brief's rules. The rules live in `crates/features` (`inbuxa-features`, module `ai`); the model call in `crates/common/src/enterprise/llm.rs`, at the path the shared tests name; the classifier step in `crates/spam-filter/src/analysis/llm.rs`; `inbuxa:AiLimits` in `crates/jmap/src/inbuxa/ai_limits.rs`; upstream files carry hooks marked `inbuxa:`. Acceptance tests 1 and 3 to 21 pass as `tests/src/system/ai.rs`, against a stub model on loopback. Test 2 passes as the shared `llm` case in `tests/src/smtp/inbound/antispam.rs`, re-enabled with its mock reading the last message. - **AI-1 to AI-28:** built. - **Test 22 (compat)** is written as `ai_compat`, ignored, and unrun until a copy of INBUXA's data is provided. It checks the twelve `LLM_*` tags and scores in observed 2. - **Calibrated** 2026-09-19 against real local models; see "Calibration". The recommended model is Qwen3 4B Instruct 2507 (Apache-2.0) on 4 vCPU. - **Known limits, not requirements of this spec:** - The call limits (AI-10, AI-11, AI-24) are per server node, as the spec says; a cluster of n nodes can have n times `maxConcurrentCalls` in flight. - The shared antispam suite's setup reads spam rules from a developer's own checkout path. It now carries on when that file is missing (a test-only change marked `inbuxa:`), so its `llm` case runs; its other cases still need the rules file. - Test 21's warning count is checked only when `registry.build-warning` is a metric of interest; the warning itself is always logged. ## Calibration Measured 2026-09-19 against real local models, with `tests/src/system/ai_calibration.rs` (ignored): it sends exactly what the classifier sends and reads answers with the classifier's own parser. Mail: 300 messages from the SpamAssassin public corpus (150 ham, 50 of them "hard ham" such as newsletters, and 150 spam from 2002 to 2005), the fork's default prompt, INBUXA's `LLM_*` scores (observed 2). Models ran under llama.cpp, quantized to Q4_K_M. Accuracy doesn't depend on hardware, so it was measured on a GPU; latency on CPU cores pinned to model a small VPS (Zen 5 cores, faster than most VPS vCPUs, so real instances will be slower). | Model (licence) | In format | Spam pushed ≥ +2 | Ham pushed ≥ +2 | 2 cores, p50 / p95 | 4 cores, p50 / p95 | |---|---|---|---|---|---| | Qwen2.5 7B (Apache-2.0) | 99% | 93% | 26% | 17.9 / 35 s | 9.7 / 18.5 s | | Llama 3.1 8B (Llama 3.1) | 100% | 96% | 23% | not measured | not measured | | Qwen2.5 3B (Qwen Research, non-commercial) | 100% | 63 to 66% | 21 to 25% | 8.1 / 15.2 s at 2 KiB | 4.6 / 8.0 s | | Qwen2.5 1.5B (Apache-2.0) | 80% | 20% | 13% | | | | Qwen2.5 0.5B (Apache-2.0) | 19% | 5% | 3% | | | | Phi-3.5 mini (MIT) | 33% | | | | | Findings: - **Text sent.** 2 KiB and 4 KiB scored the same as 16 KiB; 4 KiB pushed a 2-core 3B model past 20 s on 8% of messages, 2 KiB on none. Hence the 2048 default. - **False positives.** Ham is pushed up mostly as `Commercial`: marketing the recipient signed up for, which content alone can't tell from spam. A prompt redefining the categories made it worse on every model, so the default prompt stays. Hence the 2.0 cap. - **Size.** Below 3B, models don't follow the one-line format reliably. Phi-3.5 mini writes "High confidence" on a new line, which the parser rightly doesn't accept. - **Licences.** Qwen2.5 3B is licensed for non-commercial use only, so it can't be the recommendation for INBUXA's customers. Commercially usable models were measured next, at 2 KiB: | Model (licence) | In format | Spam pushed ≥ +2 | Ham pushed ≥ +2 | 2 cores, p50 / p95 | 4 cores, p50 / p95 | |---|---|---|---|---|---| | **Qwen3 4B Instruct 2507 (Apache-2.0)** | 100% | 99% | 28% | 10.8 / 20.4 s | 6.3 / 11.0 s | | Llama 3.2 3B (Llama 3.2) | 100% | 99% | 60% | 8.2 / 14.7 s | 4.6 / 7.8 s | | Granite 3.3 8B (Apache-2.0) | 98% | 93% | 33% | 27.5 / 50.8 s | 14.6 / 26.4 s | | Granite 3.3 2B (Apache-2.0) | 1% | | | | | | SmolLM3 3B (Apache-2.0) | 11% | | | | | | Gemma 3 4B (Gemma) | 5% | | | | | - **Recommendation: Qwen3 4B Instruct 2507, Q4_K_M** (about 2.5 GB). It's Apache-2.0, answers in format every time, and catches the most spam for its size. On 4 vCPU it stays well inside `spamCallCeiling`. On 2 vCPU the median is 11 s and about 8% of messages run past 20 s here, more on slower cores; those simply go unclassified (AI-9). 4 vCPU is the recommended minimum, 2 vCPU workable. Its false positives on legitimate mail (28%) are the reason for the +2.0 cap. - Llama 3.2 3B is as fast but pushed 60% of ham up: not recommended. Granite 8B is too slow for small instances. - Granite 2B, SmolLM3 and Gemma 3 4B answer `Category: X, Confidence: Y, …`, labelling each field, which AI-12 doesn't accept. Accepting such labels is a possible later change; the recommended model doesn't need it. ## Observed Settled on 2026-09-18 against INBUXA's live Enterprise server (Stalwart 0.16.22), read-only, as a server-level administrator and the throwaway test account. No upstream code was read. 1. **Configured models** (open question 1). No `x:AiModel` exists and `x:SpamLlm` is `Disable`. Compat test 22 carries only the tags below. 2. **`LLM_*` tags** (open question 2). Twelve entries, all `Score`, for every category and confidence: `HIGH` 3.0, `MEDIUM` 2.0, `LOW` 0.5 for `UNSOLICITED`, `COMMERCIAL` and `HARMFUL`, and −3.0, −2.0, −0.5 for `LEGITIMATE`. AI-13's clamp (+2.0 and −1.0, since calibration) cuts the positive `HIGH` scores to 2.0 and the `LEGITIMATE` `HIGH` and `MEDIUM` scores to −1.0. 3. **`X-Spam-LLM`** (open question 3). Not observable: the classifier has never run on INBUXA. 4. **`interactAi`** (open question 5). Ordinary accounts hold it: the test account's effective permissions include it, through the default `User` role. This repository's defaults match what upstream ships. The public permissions table is out of date. Questions 4, 6 and 7 need a model configured, so they stay open.