inbuxa/inbuxa-server
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Import upstream v0.16.22, stripped

Browse Source 
Upstream commit: 474dd0229cb20cf513036619781ed97bd8073c3f
Enterprise-only files removed or emptied: 63
Enterprise-only snippets removed: 117 in 50 files
Dangling module declarations removed: 5
Cargo edits turning enterprise off: 14
Verification: clean
Enterprise feature gates left for rebuilt features: 19 in 18 files

Produced by tools/fork/strip.py. The full report is in docs/fork/strip-reports/ on main.
This commit is contained in:
jcoffey-dev
2026-09-18 10:21:56 -07:00
commit 7dae9b29fd
1650 changed files with 485521 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.dockerignore
+10
View File
@@ -0,0 +1,10 @@
// Ignore everything
*
// Allow what is needed
!crates
!tests
!resources
!Cargo.lock
!Cargo.toml
.editorconfig
+11
View File
@@ -0,0 +1,11 @@
# https://EditorConfig.org
root = true
[*]
charset = utf-8
indent_size = 4
indent_style = space
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
max_line_length = 100
.github/DISCUSSION_TEMPLATE/community-discussions.yml
+30
View File
@@ -0,0 +1,30 @@
body:
- type: markdown
attributes:
value: |
> [!IMPORTANT]
> **Stalwart support has moved to [support.stalw.art](https://support.stalw.art).**
>
> For an official response from the Stalwart maintainers, please post your question on the support portal. You can sign in there with your existing GitHub account; no separate registration is required.
>
> You are welcome to start a discussion here and other community members may still reply, but the maintainers no longer answer support questions through GitHub Discussions, so your question may go unanswered unless you also post it at [support.stalw.art](https://support.stalw.art).
>
> Before opening a new report, please review the [documentation](https://stalw.art/docs/) and the [FAQ](https://stalw.art/docs/faq). Most reported issues turn out to be configuration problems rather than actual bugs.
- type: textarea
attributes:
label: Topic
description: Describe the question, problem, or topic you want to discuss with the community.
placeholder: |
I am trying to configure SMTP relay with Stalwart and I am running into ...
validations:
required: true
- type: checkboxes
attributes:
label: Acknowledgement
options:
- label: I have reviewed the [documentation](https://stalw.art/docs/) and the [FAQ](https://stalw.art/docs/faq) and confirm that my question is not addressed there.
required: true
- label: I understand that maintainers no longer answer support questions through GitHub Discussions, and that for an official response from the Stalwart team I need to post my question at [support.stalw.art](https://support.stalw.art).
required: true
- label: I agree to follow the project's [Code of Conduct](https://github.com/stalwartlabs/.github/blob/main/CODE_OF_CONDUCT.md).
required: true
.github/ISSUE_TEMPLATE/config.yml
+17
View File
@@ -0,0 +1,17 @@
blank_issues_enabled: false
contact_links:
- name: Report an Issue
url: https://support.stalw.art
about: Report a potential bug at support.stalw.art. Confirmed bugs will be converted to Issues. Sign in with your GitHub account.
- name: Questions & Support
url: https://support.stalw.art
about: Get help with configuration, troubleshooting, or general questions at support.stalw.art. Sign in with your GitHub account.
- name: Feature Requests
url: https://support.stalw.art
about: Suggest new features or improvements at support.stalw.art. Sign in with your GitHub account.
- name: Join Stalwart's Reddit
url: https://www.reddit.com/r/stalwartlabs
about: Join our subreddit for community discussions and release announcements.
- name: Join Stalwart's Discord
url: https://discord.com/servers/stalwart-923615863037390889
about: Join our Discord server for community chat and release announcements.
.github/ISSUE_TEMPLATE/confirmed_issue.yml
+25
View File
@@ -0,0 +1,25 @@
name: Bug Report (auto-closed)
description: Issues opened here are automatically closed. Please report bugs at support.stalw.art instead.
labels: ["bug"]
title: "🪲: "
body:
- type: markdown
attributes:
value: |
> [!CAUTION]
> **Issues opened directly in this repository are automatically closed and locked.**
>
> All bug reports must first be triaged at our support portal: **[support.stalw.art](https://support.stalw.art)**. If a maintainer confirms that your report is a genuine bug, they will create an Issue on your behalf; you do not need to (and should not) open one yourself.
>
> You can sign in to support.stalw.art with your existing GitHub account, so no separate registration is required.
>
> **What to do instead:**
> - Suspected bug, question, or feature request? Post it at [support.stalw.art](https://support.stalw.art).
>
> If you proceed and submit this form anyway, your issue will be closed automatically and a comment will be posted explaining this policy.
- type: checkboxes
attributes:
label: Acknowledgement
options:
- label: I understand that this issue will be automatically closed and that I should post my report at [support.stalw.art](https://support.stalw.art) instead.
required: true
.github/allowed-pr-authors.txt
+2
View File
@@ -0,0 +1,2 @@
# GitHub usernames allowed to open pull requests directly.
mdecimus
.github/dependabot.yml
+19
View File
@@ -0,0 +1,19 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
version: 2
updates:
- package-ecosystem: "cargo" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "weekly"
# Enable version updates for GitHub Actions
- package-ecosystem: "github-actions"
# Workflow files stored in the default location of `.github/workflows`
# You don't need to specify `/.github/workflows` for `directory`. You can use `directory: "/"`.
directory: "/"
schedule:
interval: "weekly"
.github/workflows/auto-close-issues.yml
+67
View File
@@ -0,0 +1,67 @@
name: Auto-close untriaged issues
on:
issues:
types: [opened, reopened]
permissions:
issues: write
jobs:
auto-close:
runs-on: ubuntu-latest
steps:
- name: Close issues from non-allowed authors
uses: actions/github-script@v7
with:
script: |
// Users allowed to open issues directly. All other authors will have
// their issues auto-closed. Add GitHub usernames (lowercase) here to
// grant additional contributors permission to open issues.
const allowedAuthors = [
'mdecimus',
];
const issue = context.payload.issue;
const author = (issue.user && issue.user.login) || '';
if (allowedAuthors.includes(author.toLowerCase())) {
core.info(`Issue #${issue.number} opened by allowed author '${author}'. Skipping.`);
return;
}
const comment = [
`Hi @${author}, thanks for taking the time to file this report.`,
``,
`This issue is being **automatically closed** because all bug reports must first be triaged at our support portal: **[support.stalw.art](https://support.stalw.art)**. Please re-post this report there so that a maintainer can review it; once confirmed as a bug, an Issue will be created on your behalf.`,
``,
`You can sign in to support.stalw.art with your existing GitHub account, so no separate registration is required.`,
``,
`Thank you for understanding.`,
].join('\n');
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue.number,
body: comment,
});
await github.rest.issues.update({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue.number,
state: 'closed',
state_reason: 'not_planned',
});
try {
await github.rest.issues.lock({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue.number,
lock_reason: 'off-topic',
});
} catch (err) {
core.warning(`Could not lock issue #${issue.number}: ${err.message}`);
}
.github/workflows/auto-close-prs.yml
+131
View File
@@ -0,0 +1,131 @@
name: Auto-close PRs from non-allowed authors
on:
pull_request_target:
types: [opened, reopened]
permissions:
pull-requests: write
issues: write
jobs:
auto-close:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
sparse-checkout: .github/allowed-pr-authors.txt
sparse-checkout-cone-mode: false
- name: Close PRs from non-allowed authors
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
let allowedAuthors = [];
try {
allowedAuthors = fs.readFileSync('.github/allowed-pr-authors.txt', 'utf8')
.split('\n')
.map(line => line.trim())
.filter(line => line && !line.startsWith('#'))
.map(line => line.toLowerCase());
} catch (err) {
core.warning(`Could not read allowed-pr-authors.txt: ${err.message}`);
}
const pr = context.payload.pull_request;
const author = (pr.user && pr.user.login) || '';
const login = author.toLowerCase();
if (author.endsWith('[bot]')) {
core.info(`PR #${pr.number} opened by bot '${author}'. Skipping.`);
return;
}
if (allowedAuthors.includes(login)) {
core.info(`PR #${pr.number} opened by allowed author '${author}'. Skipping.`);
return;
}
const actor = (context.payload.sender && context.payload.sender.login) || '';
const isCollaborator = async (username) => {
if (!username) {
return false;
}
try {
const { data: perm } = await github.rest.repos.getCollaboratorPermissionLevel({
owner: context.repo.owner,
repo: context.repo.repo,
username,
});
return perm.permission === 'admin' || perm.permission === 'write';
} catch (err) {
core.info(`Could not resolve collaborator permission for '${username}': ${err.message}`);
return false;
}
};
if (await isCollaborator(author)) {
core.info(`PR #${pr.number} author '${author}' is a collaborator. Skipping.`);
return;
}
if (actor.toLowerCase() !== login && await isCollaborator(actor)) {
core.info(`PR #${pr.number} action triggered by collaborator '${actor}'. Skipping.`);
return;
}
const contributingUrl = `https://github.com/${context.repo.owner}/${context.repo.repo}/blob/HEAD/CONTRIBUTING.md`;
const haystack = `${pr.title || ''}\n${pr.body || ''}`;
const aiPatterns = [
/[—―]/,
];
const looksAiGenerated = aiPatterns.some(re => re.test(haystack));
const aiMessage = [
`Hi @${author}, thanks for your interest in contributing.`,
``,
`This pull request is being **automatically closed and locked**. The description contains strong indicators of AI-generated content, and this project does not accept AI-generated code or unsolicited machine-authored contributions.`,
``,
`Please read [CONTRIBUTING.md](${contributingUrl}) to learn what kinds of contributions are currently accepted. If this is a genuine hand-written change that fits those guidelines, please open a discussion at **[support.stalw.art](https://support.stalw.art)** before submitting.`,
].join('\n');
const standardMessage = [
`Hi @${author}, thanks for taking the time to open this pull request.`,
``,
`This PR is being **automatically closed** because it was submitted by an author who is not on the list of approved contributors. This policy helps us keep review capacity focused and filter out unsolicited or low-quality contributions.`,
``,
`Please read [CONTRIBUTING.md](${contributingUrl}) to learn what kinds of contributions are currently accepted. If your change fits those guidelines, please first discuss it at our support portal: **[support.stalw.art](https://support.stalw.art)**. You can sign in with your existing GitHub account.`,
``,
`Thank you for understanding.`,
].join('\n');
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pr.number,
body: looksAiGenerated ? aiMessage : standardMessage,
});
await github.rest.pulls.update({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: pr.number,
state: 'closed',
});
if (looksAiGenerated) {
try {
await github.rest.issues.lock({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pr.number,
lock_reason: 'spam',
});
} catch (err) {
core.warning(`Could not lock PR #${pr.number}: ${err.message}`);
}
}
.github/workflows/auto-redirect-discussions.yml
+47
View File
@@ -0,0 +1,47 @@
name: Redirect new discussions to the support portal
on:
discussion:
types: [created]
permissions:
discussions: write
jobs:
redirect:
runs-on: ubuntu-latest
steps:
- name: Post support portal redirect
uses: actions/github-script@v7
with:
script: |
const discussion = context.payload.discussion;
const author = (discussion.user && discussion.user.login) || '';
const body = [
`Hi @${author}, thanks for posting!`,
``,
`Stalwart support has moved to **[support.stalw.art](https://support.stalw.art)**. The support portal is now the canonical place to ask questions, request help, and report issues for triage. Other community members may still reply here, but the maintainers no longer answer support questions through GitHub Discussions, so your question may go unanswered unless you also post it on the portal.`,
``,
`You can sign in to support.stalw.art with your existing GitHub account, so no separate registration is required. Google, Discord, LinkedIn, and email/password sign-in are also available.`,
``,
`**Why we are unifying our support channels**`,
``,
`Until now, Stalwart support has been spread across GitHub Discussions, Discord, Matrix, and Reddit. As the project has grown, tracking parallel inboxes and deduplicating threads has become unsustainable; the result has been slower answers, repeated work for the people helping out, and good information buried in chat scrollback where the next person with the same question would never find it.`,
``,
`[support.stalw.art](https://support.stalw.art) is a Discourse instance that we operate ourselves, hosted at Hetzner in Germany and GDPR-compliant.`,
``,
`Thank you for helping us keep the conversation in one place.`,
].join('\n');
await github.graphql(
`mutation($discussionId: ID!, $body: String!) {
addDiscussionComment(input: { discussionId: $discussionId, body: $body }) {
comment { id }
}
}`,
{
discussionId: discussion.node_id,
body,
}
);
.github/workflows/ci-retry.yml
+29
View File
@@ -0,0 +1,29 @@
name: "CI retry"
on:
workflow_run:
workflows: ["CI"]
types: [completed]
permissions:
actions: write
jobs:
rerun:
name: Re-run failed jobs
if: >
github.event.workflow_run.conclusion == 'failure' &&
github.event.workflow_run.run_attempt < 3
runs-on: ubuntu-latest
steps:
- name: Re-run failed jobs
env:
GH_TOKEN: ${{ secrets.CI_RETRY_TOKEN || github.token }}
GH_REPO: ${{ github.repository }}
RUN_ID: ${{ github.event.workflow_run.id }}
RUN_ATTEMPT: ${{ github.event.workflow_run.run_attempt }}
run: |
set -eu
echo "Run $RUN_ID failed on attempt $RUN_ATTEMPT, re-running failed jobs"
sleep 60
gh run rerun "$RUN_ID" --failed
.github/workflows/ci.yml
+567
View File
@@ -0,0 +1,567 @@
name: "CI"
on:
workflow_dispatch:
inputs:
Docker:
required: false
default: false
type: boolean
Release:
required: false
default: false
type: boolean
push:
tags: ["v*.*.*"]
env:
SCCACHE_GHA_ENABLED: true
RUSTC_WRAPPER: sccache
CARGO_TERM_COLOR: always
CARGO_NET_RETRY: 10
CARGO_NET_GIT_FETCH_WITH_CLI: true
AWS_LC_SYS_PREBUILT_NASM: 1
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
multiarch:
strategy:
fail-fast: false
matrix:
include:
- variant: gnu
- variant: musl
name: Merge image / ${{matrix.variant}}
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
attestations: write
packages: write
needs: [linux]
if: github.event_name == 'push' || inputs.Docker
steps:
- name: Install Cosign
uses: sigstore/[email protected]
- name: Log In to GitHub Container Registry
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{github.repository_owner}}
password: ${{github.token}}
- name: Log In to DockerHub
uses: docker/login-action@v4
with:
username: ${{secrets.DOCKERHUB_USERNAME}}
password: ${{secrets.DOCKERHUB_TOKEN}}
- name: Download ${{matrix.variant}} meta bake definition
uses: actions/download-artifact@v8
with:
name: bake-meta-${{matrix.variant}}
path: ${{ runner.temp }}/${{matrix.variant}}
- name: Download ${{matrix.variant}} digests
uses: actions/download-artifact@v8
with:
path: ${{ runner.temp }}/${{matrix.variant}}/digests
pattern: digests-${{matrix.variant}}-*
merge-multiple: true
- name: Create ${{matrix.variant}} manifest list and push
working-directory: ${{ runner.temp }}/${{matrix.variant}}/digests
run: |
docker buildx imagetools create $(jq -cr '.target."docker-metadata-action".tags | map(select(startswith("ghcr.io/${{github.repository}}")) | "-t " + .) | join(" ")' ${{ runner.temp }}/${{matrix.variant}}/bake-meta.json) \
$(printf 'ghcr.io/${{github.repository}}@sha256:%s ' *)
docker buildx imagetools create $(jq -cr '.target."docker-metadata-action".tags | map(select(startswith("index.docker.io/${{github.repository}}")) | "-t " + .) | join(" ")' ${{ runner.temp }}/${{matrix.variant}}/bake-meta.json) \
$(printf 'index.docker.io/${{github.repository}}@sha256:%s ' *)
- name: Inspect ${{matrix.variant}} image
id: manifest-digest
run: |
docker buildx imagetools inspect --format '{{json .Manifest}}' ghcr.io/${{github.repository}}:$(jq -r '.target."docker-metadata-action".args.DOCKER_META_VERSION' ${{ runner.temp }}/${{matrix.variant}}/bake-meta.json) | jq -r '.digest' > GHCR_DIGEST_SHA
echo "GHCR_DIGEST_SHA=$(cat GHCR_DIGEST_SHA)" | tee -a "${GITHUB_ENV}"
docker buildx imagetools inspect --format '{{json .Manifest}}' index.docker.io/${{github.repository}}:$(jq -r '.target."docker-metadata-action".args.DOCKER_META_VERSION' ${{ runner.temp }}/${{matrix.variant}}/bake-meta.json) | jq -r '.digest' > DOCKERHUB_DIGEST_SHA
echo "DOCKERHUB_DIGEST_SHA=$(cat DOCKERHUB_DIGEST_SHA)" | tee -a "${GITHUB_ENV}"
cosign sign --yes $(jq --arg GHCR_DIGEST_SHA "$(cat GHCR_DIGEST_SHA)" -cr '.target."docker-metadata-action".tags | map(select(startswith("ghcr.io/${{github.repository}}")) | . + "@" + $GHCR_DIGEST_SHA) | join(" ")' ${{ runner.temp }}/${{matrix.variant}}/bake-meta.json)
cosign sign --yes $(jq --arg DOCKERHUB_DIGEST_SHA "$(cat DOCKERHUB_DIGEST_SHA)" -cr '.target."docker-metadata-action".tags | map(select(startswith("index.docker.io/${{github.repository}}")) | . + "@" + $DOCKERHUB_DIGEST_SHA) | join(" ")' ${{ runner.temp }}/${{matrix.variant}}/bake-meta.json)
- name: Attest GHCR
uses: actions/attest-build-provenance@v4
with:
subject-name: ghcr.io/${{github.repository}}
subject-digest: ${{ env.GHCR_DIGEST_SHA }}
push-to-registry: true
- name: Attest Dockerhub
uses: actions/attest-build-provenance@v4
with:
subject-name: index.docker.io/${{github.repository}}
subject-digest: ${{ env.DOCKERHUB_DIGEST_SHA }}
push-to-registry: true
linux:
permissions:
id-token: write
contents: write
attestations: write
packages: write
strategy:
fail-fast: false
matrix:
include:
- target: x86_64-unknown-linux-gnu
platform: linux/amd64
suffix: ""
build_env: ""
- target: x86_64-unknown-linux-musl
platform: linux/amd64
suffix: "-alpine"
build_env: ""
- target: aarch64-unknown-linux-gnu
platform: linux/arm64
suffix: ""
build_env: "JEMALLOC_SYS_WITH_LG_PAGE=16 "
- target: aarch64-unknown-linux-musl
platform: linux/arm64
suffix: "-alpine"
build_env: "JEMALLOC_SYS_WITH_LG_PAGE=16 "
- target: armv7-unknown-linux-gnueabihf
platform: linux/arm/v7
suffix: ""
build_env: "JEMALLOC_SYS_WITH_LG_PAGE=16 "
- target: armv7-unknown-linux-musleabihf
platform: linux/arm/v7
suffix: "-alpine"
build_env: "JEMALLOC_SYS_WITH_LG_PAGE=16 "
- target: arm-unknown-linux-gnueabihf
platform: linux/arm/v6
suffix: ""
build_env: ""
- target: arm-unknown-linux-musleabihf
platform: linux/arm/v6
suffix: "-alpine"
build_env: ""
name: Build / ${{matrix.target}}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v7
- name: Free disk space (heavy ARM targets)
if: contains(matrix.target, 'arm') || contains(matrix.target, 'aarch64')
run: |
df -h /mnt /
sudo rm -rf /usr/share/dotnet /opt/ghc /usr/local/lib/android /usr/local/.ghcup /usr/local/share/powershell /usr/share/swift /opt/hostedtoolcache/CodeQL
sudo docker image prune --all --force || true
df -h /mnt /
- name: Add swap (heavy ARM targets)
if: contains(matrix.target, 'arm') || contains(matrix.target, 'aarch64')
run: |
mnt_avail=$(df --output=avail -k /mnt | tail -1)
if [ "$mnt_avail" -lt 18874368 ]; then
echo "Insufficient space on /mnt (${mnt_avail}K available), aborting swap setup"
exit 1
fi
sudo fallocate -l 16G /mnt/swapfile
sudo chmod 600 /mnt/swapfile
sudo mkswap /mnt/swapfile
sudo swapon /mnt/swapfile
sudo sysctl vm.swappiness=80
free -h
swapon --show
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
with:
platforms: "arm64,arm"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
with:
buildkitd-config-inline: |
[registry."docker.io"]
mirrors = ["https://mirror.gcr.io"]
driver-opts: |
network=host
- name: Log In to GitHub Container Registry
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{github.repository_owner}}
password: ${{github.token}}
- name: Log In to DockerHub
uses: docker/login-action@v4
with:
username: ${{secrets.DOCKERHUB_USERNAME}}
password: ${{secrets.DOCKERHUB_TOKEN}}
- name: Calculate shasum of external deps
id: cal-dep-shasum
run: |
echo "checksum=$(yq -p toml -oy '.package[] | select((.source | contains("")) or (.checksum | contains("")))' Cargo.lock | sha256sum | awk '{print $1}')" >> "$GITHUB_OUTPUT"
- name: Cache apt
uses: actions/[email protected]
id: apt-cache
with:
path: |
var-cache-apt
var-lib-apt
key: apt-cache-${{ hashFiles('Dockerfile.build') }}
- name: Cache Cargo
uses: actions/[email protected]
id: cargo-cache
with:
path: |
usr-local-cargo-registry
usr-local-cargo-git
key: cargo-cache-${{ steps.cal-dep-shasum.outputs.checksum }}
- name: Inject cache into docker
uses: reproducible-containers/[email protected]
with:
cache-map: |
{
"var-cache-apt": "/var/cache/apt",
"var-lib-apt": "/var/lib/apt",
"usr-local-cargo-registry": "/usr/local/cargo/registry",
"usr-local-cargo-git": "/usr/local/cargo/git"
}
skip-extraction: ${{ steps.cargo-cache.outputs.cache-hit }} && ${{ steps.apt-cache.outputs.cache-hit }}
- name: Extract Metadata for Docker
uses: docker/metadata-action@v6
id: meta
with:
images: |
index.docker.io/${{github.repository}}
ghcr.io/${{github.repository}}
flavor: |
suffix=${{matrix.suffix}},onlatest=true
tags: |
type=ref,event=tag
type=ref,event=branch,prefix=branch-
type=edge,branch=main
type=semver,pattern=v{{major}}.{{minor}}
- name: Build Artifact
id: bake
uses: docker/bake-action@v7
env:
DOCKER_BUILD_RECORD_UPLOAD: false
TARGET: ${{matrix.target}}
GHCR_REPO: ghcr.io/${{github.repository}}
BUILD_ENV: ${{matrix.build_env}}
DOCKER_PLATFORM: ${{matrix.platform}}
SUFFIX: ${{matrix.suffix}}
with:
source: .
set: |
*.tags=
image.output=type=image,"name=ghcr.io/${{github.repository}},index.docker.io/${{github.repository}}",push-by-digest=true,name-canonical=true,push=true,compression=zstd,compression-level=9,force-compression=true,oci-mediatypes=true
files: |
docker-bake.hcl
${{ steps.meta.outputs.bake-file }}
targets: ${{(github.event_name == 'push' || inputs.Docker) && 'build,image' || 'build'}}
- name: Upload Artifacts
uses: actions/[email protected]
with:
name: artifact-${{matrix.target}}
path: |
artifact
!artifact/*.json
- name: Export digest & Rename meta bake definition file
if: github.event_name == 'push' || inputs.Docker
run: |
mv "${{ steps.meta.outputs.bake-file }}" "${{ runner.temp }}/bake-meta.json"
mkdir -p ${{ runner.temp }}/digests
digest="${{ fromJSON(steps.bake.outputs.metadata).image['containerimage.digest'] }}"
touch "${{ runner.temp }}/digests/${digest#sha256:}"
- name: Upload digest
if: github.event_name == 'push' || inputs.Docker
uses: actions/[email protected]
with:
name: digests-${{matrix.suffix == '' && 'gnu' || 'musl'}}-${{ matrix.target }}
path: ${{ runner.temp }}/digests/*
if-no-files-found: error
retention-days: 1
- name: Upload GNU meta bake definition
uses: actions/[email protected]
if: (github.event_name == 'push' || inputs.Docker) && endsWith(matrix.target,'gnu') && startsWith(matrix.target,'x86')
with:
name: bake-meta-gnu
path: ${{ runner.temp }}/bake-meta.json
if-no-files-found: error
retention-days: 1
- name: Upload musl meta bake definition
uses: actions/[email protected]
if: (github.event_name == 'push' || inputs.Docker) && endsWith(matrix.target,'musl') && startsWith(matrix.target,'x86')
with:
name: bake-meta-musl
path: ${{ runner.temp }}/bake-meta.json
if-no-files-found: error
retention-days: 1
windows:
name: Build / ${{matrix.target}}
runs-on: windows-latest
strategy:
fail-fast: false
matrix:
include:
# - target: aarch64-pc-windows-msvc
- target: x86_64-pc-windows-msvc
steps:
- name: Checkout
uses: actions/checkout@v7
- name: Run sccache-cache
uses: mozilla-actions/[email protected]
with:
disable_annotations: true
- name: Build
run: |
rustup target add ${{matrix.target}}
cargo build --release --target ${{matrix.target}} -p stalwart --no-default-features --features "sqlite postgres mysql rocks s3 redis azure nats enterprise"
mkdir -p artifacts
mv ./target/${{matrix.target}}/release/stalwart.exe ./artifacts/stalwart.exe
- name: Upload Artifacts
uses: actions/[email protected]
with:
name: artifact-${{matrix.target}}
path: artifacts
macos:
name: Build / ${{matrix.target}}
runs-on: macos-latest
strategy:
fail-fast: false
matrix:
include:
- target: aarch64-apple-darwin
- target: x86_64-apple-darwin
steps:
- name: Checkout
uses: actions/checkout@v7
- name: Run sccache-cache
uses: mozilla-actions/[email protected]
with:
disable_annotations: true
#- name: Build FoundationDB Edition
# env:
# GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# run: |
# rustup target add ${{matrix.target}}
# # Pin FoundationDB 7.4.x (Apple publishes these as prereleases)
# curl --retry 5 -Lso foundationdb.pkg "$(gh api -X GET /repos/apple/foundationdb/releases --jq '[.[] | select(.tag_name | startswith("7.4."))] | sort_by(.tag_name | split(".") | map(tonumber)) | reverse | .[0].assets[] | select(.name | test("${{startsWith(matrix.target, 'x86') && 'x86_64' || 'arm64'}}" + ".pkg$")) | .browser_download_url')"
# echo "=== Package contents ==="
# pkgutil --payload-files foundationdb.pkg || true
# sudo installer -allowUntrusted -verbose -dumplog -pkg foundationdb.pkg -target /
# cargo build --release --target ${{matrix.target}} -p stalwart --no-default-features --features "foundationdb s3 redis nats enterprise"
# mkdir -p artifacts
# mv ./target/${{matrix.target}}/release/stalwart ./artifacts/stalwart-foundationdb
- name: Build
run: |
rustup target add ${{matrix.target}}
cargo build --release --target ${{matrix.target}} -p stalwart --no-default-features --features "sqlite postgres mysql rocks s3 redis azure nats enterprise"
mkdir -p artifacts
mv ./target/${{matrix.target}}/release/stalwart ./artifacts/stalwart
- name: Upload Artifacts
uses: actions/[email protected]
with:
name: artifact-${{matrix.target}}
path: artifacts
freebsd:
name: Build / ${{matrix.target}}
runs-on: ubuntu-latest
timeout-minutes: 360
strategy:
fail-fast: false
matrix:
include:
- target: x86_64-unknown-freebsd
arch: x86_64
# - target: aarch64-unknown-freebsd
# arch: aarch64
steps:
- name: Checkout
uses: actions/checkout@v7
- name: Build in FreeBSD VM
uses: vmactions/freebsd-vm@v1
with:
release: "15.1"
arch: ${{matrix.arch}}
usesh: true
mem: 14336
cpu: 4
sync: rsync
copyback: true
# gmake: required by jemalloc-sys on BSD hosts
# llvm: provides libclang for bindgen (librocksdb-sys)
# rust: libsqlite3-sys 0.38 uses cfg_select!, stabilized in Rust
# 1.95. The default 'quarterly' pkg repo still ships rust 1.94, so
# switch to the 'latest' repo (currently 1.96.1). rustup is not an
# option here: aarch64-unknown-freebsd has no rustup toolchains yet.
prepare: |
set -e
mkdir -p /usr/local/etc/pkg/repos
echo 'FreeBSD: { url: "pkg+https://pkg.freebsd.org/${ABI}/latest", mirror_type: "srv" }' > /usr/local/etc/pkg/repos/FreeBSD.conf
pkg update -f
env ASSUME_ALWAYS_YES=yes pkg bootstrap -f
pkg update -f
pkg install -y rust gmake llvm rocksdb
rustc --version
run: |
set -e
export CARGO_TARGET_DIR=/tmp/target
export CARGO_TERM_COLOR=always
export CARGO_NET_RETRY=10
cargo build --release -p stalwart --no-default-features --features "sqlite postgres mysql rocks s3 redis azure nats enterprise"
mkdir -p artifacts
cp /tmp/target/release/stalwart artifacts/stalwart
- name: Upload Artifacts
uses: actions/[email protected]
with:
name: artifact-${{matrix.target}}
path: artifacts
release:
name: Release
permissions:
id-token: write
contents: write
attestations: write
if: github.event_name == 'push' || inputs.Release
needs: [linux, windows, macos, freebsd]
runs-on: ubuntu-latest
steps:
# Must run before artifacts are downloaded — checkout cleans the workspace.
- name: Checkout (for CHANGELOG)
if: startsWith(github.ref, 'refs/tags/')
uses: actions/checkout@v7
- name: Download Artifacts
uses: actions/download-artifact@v8
with:
path: archive
pattern: artifact-*
- name: Compress
run: |
set -eux
BASE_DIR="$(pwd)/archive"
compress_files() {
local dir="$1"
local archive_dir_name="${dir#artifact-}"
cd "$dir"
# Process each file in the directory
for file in `ls`; do
filename="${file%.*}"
extension="${file##*.}"
if [ "$extension" = "exe" ]; then
7z a -tzip "${filename}-${archive_dir_name}.zip" "$file" > /dev/null
else
tar -czf "${filename}-${archive_dir_name}.tar.gz" "$file"
fi
done
cd $BASE_DIR
}
cd $BASE_DIR
for arch_dir in `ls`; do
dir_name=$(basename "$arch_dir")
compress_files "$dir_name"
done
- name: Attest binary
id: attest
uses: actions/attest-build-provenance@v4
with:
subject-path: |
archive/**/*.tar.gz
archive/**/*.zip
- name: Use cosign to sign existing artifacts
uses: sigstore/[email protected]
with:
inputs: |
archive/**/*.tar.gz
archive/**/*.zip
- name: Build release body
run: |
if [ "${{ startsWith(github.ref, 'refs/tags/') }}" = "true" ]; then
awk '/^## \[/{c++} c==1' CHANGELOG.md > release_body.md
echo "" >> release_body.md
else
: > release_body.md
fi
cat >> release_body.md <<EOF
<hr />
### Check binary attestation [here](${{ steps.attest.outputs.attestation-url }})
EOF
- name: Release
uses: softprops/action-gh-release@v3
with:
files: |
archive/**/*.tar.gz
archive/**/*.zip
archive/**/*.sigstore.json
prerelease: ${{!startsWith(github.ref, 'refs/tags/') || null}}
tag_name: ${{!startsWith(github.ref, 'refs/tags/') && 'nightly' || null}}
# Tag-push releases are created as drafts; the `publish` job un-drafts
# them only after all build jobs succeed, so watcher notifications
# don't fire on broken builds.
draft: ${{ startsWith(github.ref, 'refs/tags/') || null }}
body_path: release_body.md
publish:
name: Publish release
needs: [linux, windows, macos, freebsd, multiarch, release]
if: startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Un-draft release
env:
GH_TOKEN: ${{ github.token }}
run: gh release edit "${{ github.ref_name }}" --draft=false --latest --repo "${{ github.repository }}"
cleanup:
name: Cleanup failed release
needs: [linux, windows, macos, freebsd, multiarch, release]
if: failure() && startsWith(github.ref, 'refs/tags/') && github.run_attempt >= 3
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Delete draft release and tag
env:
GH_TOKEN: ${{ github.token }}
run: gh release delete "${{ github.ref_name }}" --yes --cleanup-tag --repo "${{ github.repository }}" || true
.github/workflows/scorecard.yml
+78
View File
@@ -0,0 +1,78 @@
# This workflow uses actions that are not certified by GitHub. They are provided
# by a third-party and are governed by separate terms of service, privacy
# policy, and support documentation.
name: Scorecard supply-chain security
on:
# For Branch-Protection check. Only the default branch is supported. See
# https://github.com/ossf/scorecard/blob/main/docs/checks.md#branch-protection
branch_protection_rule:
# To guarantee Maintained check is occasionally updated. See
# https://github.com/ossf/scorecard/blob/main/docs/checks.md#maintained
schedule:
- cron: '31 6 * * 0'
push:
branches: [ "main" ]
# Declare default permissions as read only.
permissions: read-all
jobs:
analysis:
name: Scorecard analysis
runs-on: ubuntu-latest
# `publish_results: true` only works when run from the default branch. conditional can be removed if disabled.
if: github.event.repository.default_branch == github.ref_name || github.event_name == 'pull_request'
permissions:
# Needed to upload the results to code-scanning dashboard.
security-events: write
# Needed to publish results and get a badge (see publish_results below).
id-token: write
# Uncomment the permissions below if installing in a private repository.
# contents: read
# actions: read
steps:
- name: "Checkout code"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4.2.2
with:
persist-credentials: false
- name: "Run analysis"
uses: ossf/scorecard-action@2d1146689b8cda280b9bc96326124645441f03bc # v2.4.4
with:
results_file: results.sarif
results_format: sarif
# (Optional) "write" PAT token. Uncomment the `repo_token` line below if:
# - you want to enable the Branch-Protection check on a *public* repository, or
# - you are installing Scorecard on a *private* repository
# To create the PAT, follow the steps in https://github.com/ossf/scorecard-action?tab=readme-ov-file#authentication-with-fine-grained-pat-optional.
# repo_token: ${{ secrets.SCORECARD_TOKEN }}
# Public repositories:
# - Publish results to OpenSSF REST API for easy access by consumers
# - Allows the repository to include the Scorecard badge.
# - See https://github.com/ossf/scorecard-action#publishing-results.
# For private repositories:
# - `publish_results` will always be set to `false`, regardless
# of the value entered here.
publish_results: true
# (Optional) Uncomment file_mode if you have a .gitattributes with files marked export-ignore
# file_mode: git
# Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
# format to the repository Actions tab.
- name: "Upload artifact"
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: SARIF file
path: results.sarif
retention-days: 5
# Upload the results to GitHub's code scanning dashboard (optional).
# Commenting out will disable upload of results to your repo's Code Scanning dashboard
- name: "Upload to code-scanning"
uses: github/codeql-action/[email protected]
with:
sarif_file: results.sarif
.github/workflows/test.yml
+57
View File
@@ -0,0 +1,57 @@
name: Test
on:
workflow_dispatch:
jobs:
style:
name: Check Style
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v7
- name: Check Style
run: cargo fmt --all --check
test:
name: Test
needs: style
runs-on: ubuntu-latest
env:
STORE: RocksDb
RUST_MIN_STACK: "16777216"
steps:
- name: Checkout
uses: actions/checkout@v7
# External services (OpenLDAP, Keycloak, PostgreSQL, MySQL, Redis, NATS,
# MinIO, OpenSearch, Meilisearch) are provisioned on demand by the test
# suite via testcontainers using the Docker daemon available on the
# runner; see tests/src/utils/containers.rs.
- name: Rust Cache
uses: Swatinem/rust-cache@v2
- name: JMAP Protocol Tests
run: cargo test -p jmap_proto -- --nocapture
- name: IMAP Protocol Tests
run: cargo test -p imap_proto -- --nocapture
- name: Full-text search Tests
run: cargo test -p store -- --nocapture
- name: Directory Tests
run: cargo test -p tests directory -- --nocapture
- name: SMTP Tests
run: cargo test -p tests smtp -- --nocapture
- name: IMAP Tests
run: cargo test -p tests imap -- --nocapture
- name: JMAP Tests
run: cargo test -p tests jmap -- --nocapture
.github/workflows/trivy.yml
+41
View File
@@ -0,0 +1,41 @@
# trivy ci workflow
name: trivy
on:
workflow_dispatch:
push:
branches: [ "main" ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ "main" ]
schedule:
- cron: '00 12 * * *'
permissions:
contents: read
jobs:
build:
permissions:
contents: read # for actions/checkout to fetch code
security-events: write # for github/codeql-action/upload-sarif to upload SARIF results
actions: read # only required for a private repository by github/codeql-action/upload-sarif to get the Action run status
name: Check
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v7
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
scan-type: 'fs'
ignore-unfixed: true
format: 'sarif'
output: 'trivy-results.sarif'
severity: 'CRITICAL,HIGH'
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/[email protected]
with:
sarif_file: 'trivy-results.sarif'
.gitignore
+9
View File
@@ -0,0 +1,9 @@
/target
*.failed
*_failed
run.sh
.*
!.gitignore
!.gitattributes
!.github
CLAUDE.md
CHANGELOG.md
+1968
View File
File diff suppressed because it is too large Load Diff
CNAME
+1
View File
@@ -0,0 +1 @@
get.stalw.art
CONTRIBUTING.md
+61
View File
@@ -0,0 +1,61 @@
# Contributing
Thank you for your interest in contributing to Stalwart. We appreciate the support and enthusiasm of the open-source community. To keep the project maintainable and the review process sustainable, contributions are subject to the policies described below. Please read them in full before opening a pull request.
## Vouched Contributors Only
Due to the high volume of low-quality, AI-generated submissions, pull requests are limited to a list of vouched contributors. Pull requests opened by anyone who is not on this list are closed automatically.
To be added as a vouched contributor, post a message at [support.stalw.art](https://support.stalw.art) explaining the code changes you would like to submit, and include a link to the proposed change (a branch, diff, or draft). Once a maintainer has reviewed your request and vouched for you, you will be able to open pull requests directly.
This policy lets us focus limited review capacity on contributions from people who have taken the time to understand the codebase and discuss their changes first.
## What Contributions Are Accepted
At this stage of the project we accept a narrow set of contributions:
- **Bug fixes.** Corrections to existing, incorrect behavior are welcome. Please include steps to reproduce the bug and describe the fix.
- **Translations.** Additions and corrections to existing translations are welcome.
New features are generally **not** accepted, unless they involve only a few lines of code. Larger features fall outside the scope of what we can review and integrate while the architecture is still evolving.
If you would like to see a new feature, please request it at [support.stalw.art](https://support.stalw.art) under the **Feature Ideas** category rather than opening a pull request. This lets the community discuss and prioritize ideas before any code is written.
## No AI-Generated Code
AI-generated code is not accepted in this project.
Even the most advanced models write inefficient Rust code. Beyond raw performance, AI creates technical debt by generating large amounts of code that not even the authors who submitted it can fully understand or maintain. Reviewing and untangling such contributions costs the maintainers far more time than it saves.
Using AI as a fancy autocomplete is perfectly fine. What matters is that every line generated by a model is read, understood, and reviewed by a human before it is submitted. You are responsible for every line in your pull request, regardless of how it was produced. If you cannot explain why a change is written the way it is, it is not ready to be submitted.
## Pull Request Process
Once you are a vouched contributor:
1. Keep each pull request small and focused on a single logical change.
2. Match the style and conventions of the surrounding code.
3. Make sure the project builds and the test suite passes before opening the pull request.
4. In the pull request description, explain what the change does and why, and link to the [support.stalw.art](https://support.stalw.art) discussion where the change was vouched.
## Code of Conduct
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation. We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
You can read the full Code of Conduct [here](https://github.com/stalwartlabs/.github/blob/main/CODE_OF_CONDUCT.md).
## Licensing
This project is licensed under the Affero General Public License (AGPL) version 3.0. By contributing to this project, you agree that your contributions will be licensed under the AGPL-3.0 license.
## Fiduciary Contributor License Agreement
Before making any contributions, all contributors are required to sign the Fiduciary Contributor License Agreement (FLA). The FLA is a legal agreement that assigns the copyright of contributions to a designated fiduciary, who manages these rights on behalf of the project. This arrangement ensures that the software remains free and open, even as contributors come and go.
Key points of the FLA:
- Ensures the software remains free and open source
- Protects the project from potential copyright issues
- Includes a reversion clause: if the fiduciary violates Free Software principles, rights revert to the original contributors
For more details about the FLA, please refer to the [FLA FAQ](https://fsfe.org/activities/fla/fla.en.html).
Cargo.lock
Generated
+10792
View File
File diff suppressed because it is too large Load Diff
Cargo.toml
+79
View File
@@ -0,0 +1,79 @@
[workspace]
resolver = "2"
members = [
"crates/main",
"crates/types",
"crates/http",
"crates/http-proto",
"crates/jmap",
"crates/jmap-proto",
"crates/email",
"crates/imap",
"crates/imap-proto",
"crates/smtp",
"crates/managesieve",
"crates/pop3",
"crates/dav-proto",
"crates/scim-proto",
"crates/scim",
"crates/dav",
"crates/groupware",
"crates/spam-filter",
"crates/nlp",
"crates/store",
"crates/coordinator",
"crates/directory",
"crates/registry",
"crates/services",
"crates/utils",
"crates/common",
"crates/trc",
"crates/migration",
"tests",
]
[workspace.lints.clippy]
result_unit_err = "allow"
[profile.dev]
opt-level = 0
debug = 1
#codegen-units = 4
lto = false
incremental = true
panic = 'unwind'
debug-assertions = true
overflow-checks = false
rpath = false
[profile.release]
opt-level = 3
debug = false
codegen-units = 1
lto = true
incremental = false
panic = 'unwind'
debug-assertions = false
overflow-checks = false
rpath = false
strip = true
[profile.test]
opt-level = 0
debug = 1
#codegen-units = 16
lto = false
incremental = true
debug-assertions = true
overflow-checks = true
rpath = false
[profile.bench]
opt-level = 3
debug = false
codegen-units = 1
lto = true
incremental = false
debug-assertions = false
overflow-checks = false
rpath = false
Dockerfile
+46
View File
@@ -0,0 +1,46 @@
FROM --platform=$BUILDPLATFORM docker.io/lukemathwalker/cargo-chef:latest-rust-slim-trixie AS chef
WORKDIR /build
FROM --platform=$BUILDPLATFORM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-path /recipe.json
FROM --platform=$BUILDPLATFORM chef AS builder
ARG TARGETPLATFORM
RUN case "${TARGETPLATFORM}" in \
"linux/arm64") echo "aarch64-unknown-linux-gnu" > /target.txt && echo "-C linker=aarch64-linux-gnu-gcc" > /flags.txt ;; \
"linux/amd64") echo "x86_64-unknown-linux-gnu" > /target.txt && echo "-C linker=x86_64-linux-gnu-gcc" > /flags.txt ;; \
*) exit 1 ;; \
esac
RUN export DEBIAN_FRONTEND=noninteractive && \
apt-get update && \
apt-get install -yq --no-install-recommends build-essential libclang-19-dev \
g++-aarch64-linux-gnu binutils-aarch64-linux-gnu \
g++-x86-64-linux-gnu binutils-x86-64-linux-gnu
RUN rustup target add "$(cat /target.txt)"
COPY --from=planner /recipe.json /recipe.json
RUN RUSTFLAGS="$(cat /flags.txt)" cargo chef cook --target "$(cat /target.txt)" --release --no-default-features --features "sqlite postgres mysql rocks s3 redis azure nats enterprise" --recipe-path /recipe.json
COPY . .
RUN RUSTFLAGS="$(cat /flags.txt)" cargo build --target "$(cat /target.txt)" --release -p stalwart --no-default-features --features "sqlite postgres mysql rocks s3 redis azure nats enterprise"
RUN mv "/build/target/$(cat /target.txt)/release" "/output"
FROM docker.io/debian:trixie-slim
RUN export DEBIAN_FRONTEND=noninteractive && \
apt-get update && \
apt-get install -yq --no-install-recommends ca-certificates curl libcap2-bin && \
rm -rf /var/lib/apt/lists/* && \
groupadd -r -g 2000 stalwart && \
useradd -r -u 2000 -g 2000 -s /usr/sbin/nologin -M stalwart && \
mkdir -p /etc/stalwart /var/lib/stalwart && \
chown stalwart:stalwart /etc/stalwart /var/lib/stalwart
COPY --from=builder --chmod=0755 /output/stalwart /usr/local/bin/stalwart
RUN setcap 'cap_net_bind_service=+ep' /usr/local/bin/stalwart
USER stalwart
WORKDIR /var/lib/stalwart
VOLUME ["/etc/stalwart", "/var/lib/stalwart"]
EXPOSE 443 25 110 587 465 143 993 995 4190 8080
ENV STALWART_HEALTHCHECK_URL=https://127.0.0.1:443/healthz/live
HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
CMD curl -fsSk -H "X-Forwarded-For: 127.0.0.1" "$STALWART_HEALTHCHECK_URL" || curl -fsS -H "X-Forwarded-For: 127.0.0.1" http://127.0.0.1:8080/healthz/live || exit 1
ENTRYPOINT ["/usr/local/bin/stalwart"]
CMD ["--config", "/etc/stalwart/config.json"]
Dockerfile.build
+195
View File
@@ -0,0 +1,195 @@
# syntax=docker/dockerfile:1
# check=skip=FromPlatformFlagConstDisallowed,RedundantTargetPlatform
# *****************
# Base image for planner & builder
# *****************
FROM --platform=$BUILDPLATFORM rust:slim-trixie AS base
ENV DEBIAN_FRONTEND="noninteractive" \
BINSTALL_DISABLE_TELEMETRY=true \
CARGO_TERM_COLOR=always \
CARGO_NET_RETRY=10 \
CARGO_NET_GIT_FETCH_WITH_CLI=true \
LANG=C.UTF-8 \
TZ=UTC \
TERM=xterm-256color \
AWS_LC_SYS_PREBUILT_NASM=1
# With zig, we only need libclang and make. ca-certificates is required for curl
# to verify HTTPS downloads (zig tarballs, FoundationDB debs, crates.io, etc).
RUN \
--mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
rm -f /etc/apt/apt.conf.d/docker-clean && \
echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' >/etc/apt/apt.conf.d/keep-cache && \
apt-get update && \
apt-get install -yq --no-install-recommends ca-certificates curl git jq xz-utils make libclang-19-dev
# Zig is pinned to 0.13.0. Zig 0.14+ changed how its default target ABI is
# resolved for `x86_64-linux-gnu`-style triples, and on Stalwart's dep tree
# (rocksdb, jemalloc, aws-lc-sys, libfdb_c.so, rust-std 1.87+) this surfaces
# as ld.lld undefined references to pthread_*/stat/pow@GLIBC_2.33+. An
# explicit glibc suffix (`.2.17`) is NOT a workaround — C deps compiled via
# zig cc against Zig 0.16 headers still emit @2.33+ symbols even when the
# linker target is 2.17. Zig 0.13 produces binaries with a max glibc ref of
# ~2.28 which stays compatible with Debian 11 / RHEL 8. Revisit this pin if
# Zig upstream stabilizes a backwards-compatible default or if Stalwart drops
# its hard C deps.
RUN \
ZIG_VERSION=0.13.0 && \
curl --retry 5 -fsSL "https://ziglang.org/download/${ZIG_VERSION}/zig-linux-$(uname -m)-${ZIG_VERSION}.tar.xz" | tar -J -x -C /usr/local && \
ln -s "/usr/local/zig-linux-$(uname -m)-${ZIG_VERSION}/zig" /usr/local/bin/zig && \
zig version
# Install cargo-binstall
RUN curl --retry 5 -fL --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/cargo-bins/cargo-binstall/main/install-from-binstall-release.sh | bash
# Install cargo-chef & sccache & cargo-zigbuild
RUN cargo binstall --no-confirm [email protected] [email protected] [email protected]
# *****************
# Planner
# *****************
FROM base AS planner
WORKDIR /app
COPY . .
# Generate recipe file
RUN cargo chef prepare --recipe-path recipe.json
# *****************
# Builder
# *****************
FROM base AS builder
WORKDIR /app
COPY --from=planner /app/recipe.json recipe.json
ARG TARGET
ARG BUILD_ENV
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
# Install toolchain and specify some env variables
RUN \
rustup set profile minimal && \
rustup target add ${TARGET} && \
mkdir -p artifact && \
touch /env-cargo && \
if [ ! -z "${BUILD_ENV}" ]; then \
echo "export ${BUILD_ENV}" >> /env-cargo; \
echo "Setting up ${BUILD_ENV}"; \
fi && \
if [[ "${TARGET}" == *gnu ]]; then \
base_arch="${TARGET%%-*}"; \
case "$base_arch" in \
x86_64) \
echo "export FDB_ARCH=amd64" >> /env-cargo; \
;; \
aarch64) \
echo "export FDB_ARCH=aarch64" >> /env-cargo; \
;; \
*) \
exit 1; \
;; \
esac; \
fi
# Install FoundationDB (pinned to latest 7.4.x; Apple publishes 7.4 as
# prereleases on GitHub, so the release list is fetched without the
# prerelease filter and narrowed by tag name).
ARG FDB_VERSION_RANGE="7.4"
RUN \
source /env-cargo && \
if [ ! -z "${FDB_ARCH}" ]; then \
curl --retry 5 -fLso fdb-client.deb "$(curl --retry 5 -fLs 'https://api.github.com/repos/apple/foundationdb/releases?per_page=100' | jq --arg FDB_ARCH "$FDB_ARCH" --arg RANGE "${FDB_VERSION_RANGE}" -r '[.[] | select(.tag_name | startswith($RANGE + "."))] | sort_by(.tag_name | split(".") | map(tonumber)) | reverse | .[0].assets[] | select(.name | test("foundationdb-clients.*" + $FDB_ARCH + ".deb$")) | .browser_download_url')" && \
mkdir -p /fdb && \
dpkg -x fdb-client.deb /fdb && \
mv /fdb/usr/include/foundationdb /usr/include && \
mv /fdb/usr/lib/libfdb_c.so /usr/lib && \
rm -rf fdb-client.deb /fdb; \
fi
# Cargo-chef Cache layer
RUN \
--mount=type=secret,id=ACTIONS_RESULTS_URL,env=ACTIONS_RESULTS_URL \
--mount=type=secret,id=ACTIONS_RUNTIME_TOKEN,env=ACTIONS_RUNTIME_TOKEN \
--mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/usr/local/cargo/git \
source /env-cargo && \
if [ ! -z "${FDB_ARCH}" ]; then \
RUSTFLAGS="-L /usr/lib" cargo chef cook --recipe-path recipe.json --zigbuild --release --target ${TARGET} -p stalwart --no-default-features --features "foundationdb s3 redis nats enterprise"; \
fi
RUN \
--mount=type=secret,id=ACTIONS_RESULTS_URL,env=ACTIONS_RESULTS_URL \
--mount=type=secret,id=ACTIONS_RUNTIME_TOKEN,env=ACTIONS_RUNTIME_TOKEN \
--mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/usr/local/cargo/git \
source /env-cargo && \
cargo chef cook --recipe-path recipe.json --zigbuild --release --target ${TARGET} -p stalwart --no-default-features --features "sqlite postgres mysql rocks s3 redis azure nats enterprise"
# Copy the source code
COPY . .
ENV RUSTC_WRAPPER="sccache" \
SCCACHE_GHA_ENABLED=true
# Build FoundationDB version
RUN \
--mount=type=secret,id=ACTIONS_RESULTS_URL,env=ACTIONS_RESULTS_URL \
--mount=type=secret,id=ACTIONS_RUNTIME_TOKEN,env=ACTIONS_RUNTIME_TOKEN \
--mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/usr/local/cargo/git \
source /env-cargo && \
if [ ! -z "${FDB_ARCH}" ]; then \
RUSTFLAGS="-L /usr/lib" cargo zigbuild --release --target ${TARGET} -p stalwart --no-default-features --features "foundationdb s3 redis nats enterprise" && \
mv /app/target/${TARGET}/release/stalwart /app/artifact/stalwart-foundationdb; \
fi
# Build generic version
RUN \
--mount=type=secret,id=ACTIONS_RESULTS_URL,env=ACTIONS_RESULTS_URL \
--mount=type=secret,id=ACTIONS_RUNTIME_TOKEN,env=ACTIONS_RUNTIME_TOKEN \
--mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/usr/local/cargo/git \
source /env-cargo && \
cargo zigbuild --release --target ${TARGET} -p stalwart --no-default-features --features "sqlite postgres mysql rocks s3 redis azure nats enterprise" && \
mv /app/target/${TARGET}/release/stalwart /app/artifact/stalwart
# *****************
# Binary stage
# *****************
FROM scratch AS binaries
COPY --from=builder /app/artifact /
# *****************
# Runtime image for GNU targets
# *****************
FROM --platform=$TARGETPLATFORM docker.io/library/debian:trixie-slim AS gnu
RUN export DEBIAN_FRONTEND=noninteractive && \
apt-get update && \
apt-get install -yq --no-install-recommends ca-certificates curl tzdata libcap2-bin && \
rm -rf /var/lib/apt/lists/* && \
groupadd -r -g 2000 stalwart && \
useradd -r -u 2000 -g 2000 -s /usr/sbin/nologin -M stalwart && \
mkdir -p /etc/stalwart /var/lib/stalwart && \
chown stalwart:stalwart /etc/stalwart /var/lib/stalwart
COPY --from=builder --chmod=0755 /app/artifact/stalwart /usr/local/bin/stalwart
RUN setcap 'cap_net_bind_service=+ep' /usr/local/bin/stalwart
USER stalwart
WORKDIR /var/lib/stalwart
VOLUME ["/etc/stalwart", "/var/lib/stalwart"]
EXPOSE 443 25 110 587 465 143 993 995 4190 8080
ENV STALWART_HEALTHCHECK_URL=https://127.0.0.1:443/healthz/live
HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
CMD curl -fsSk -H "X-Forwarded-For: 127.0.0.1" "$STALWART_HEALTHCHECK_URL" || curl -fsS -H "X-Forwarded-For: 127.0.0.1" http://127.0.0.1:8080/healthz/live || exit 1
ENTRYPOINT ["/usr/local/bin/stalwart"]
CMD ["--config", "/etc/stalwart/config.json"]
# *****************
# Runtime image for musl targets
# *****************
FROM --platform=$TARGETPLATFORM alpine AS musl
RUN apk add --update --no-cache ca-certificates curl tzdata libcap && \
rm -rf /var/cache/apk/* && \
addgroup -S -g 2000 stalwart && \
adduser -S -D -H -u 2000 -G stalwart -s /sbin/nologin stalwart && \
mkdir -p /etc/stalwart /var/lib/stalwart && \
chown stalwart:stalwart /etc/stalwart /var/lib/stalwart
COPY --from=builder --chmod=0755 /app/artifact/stalwart /usr/local/bin/stalwart
RUN setcap 'cap_net_bind_service=+ep' /usr/local/bin/stalwart
USER stalwart
WORKDIR /var/lib/stalwart
VOLUME ["/etc/stalwart", "/var/lib/stalwart"]
EXPOSE 443 25 110 587 465 143 993 995 4190 8080
ENV STALWART_HEALTHCHECK_URL=https://127.0.0.1:443/healthz/live
HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
CMD curl -fsSk -H "X-Forwarded-For: 127.0.0.1" "$STALWART_HEALTHCHECK_URL" || curl -fsS -H "X-Forwarded-For: 127.0.0.1" http://127.0.0.1:8080/healthz/live || exit 1
ENTRYPOINT ["/usr/local/bin/stalwart"]
CMD ["--config", "/etc/stalwart/config.json"]
Dockerfile.fdb
+80
View File
@@ -0,0 +1,80 @@
# syntax=docker/dockerfile:1
FROM debian:trixie-slim AS chef
ARG TARGETARCH
ARG FDB_VERSION_RANGE="7.4"
RUN apt-get update && \
export DEBIAN_FRONTEND=noninteractive && \
apt-get install -yq --no-install-recommends \
build-essential \
ca-certificates \
cmake \
clang \
curl \
jq \
protobuf-compiler
ENV RUSTUP_HOME=/opt/rust/rustup \
PATH=/home/root/.cargo/bin:/opt/rust/cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
RUN curl https://sh.rustup.rs -sSf | \
env CARGO_HOME=/opt/rust/cargo \
sh -s -- -y --default-toolchain stable --profile minimal --no-modify-path && \
env CARGO_HOME=/opt/rust/cargo \
rustup component add rustfmt
RUN \
ARCH="${TARGETARCH:-$(dpkg --print-architecture)}" && \
case "$ARCH" in \
amd64) FDB_ARCH=amd64 ;; \
arm64) FDB_ARCH=aarch64 ;; \
*) echo "Unsupported architecture: $ARCH" >&2; exit 1 ;; \
esac && \
curl --retry 5 -fLso fdb-client.deb "$(curl --retry 5 -fLs 'https://api.github.com/repos/apple/foundationdb/releases?per_page=100' | jq --arg FDB_ARCH "$FDB_ARCH" --arg RANGE "${FDB_VERSION_RANGE}" -r '[.[] | select(.tag_name | startswith($RANGE + "."))] | sort_by(.tag_name | split(".") | map(tonumber)) | reverse | .[0].assets[] | select(.name | test("foundationdb-clients.*" + $FDB_ARCH + ".deb$")) | .browser_download_url')" && \
mkdir -p /fdb && \
dpkg -x fdb-client.deb /fdb && \
mv /fdb/usr/include/foundationdb /usr/include && \
mv /fdb/usr/lib/libfdb_c.so /usr/lib && \
rm -rf fdb-client.deb /fdb
RUN env CARGO_HOME=/opt/rust/cargo cargo install cargo-chef && \
rm -rf /opt/rust/cargo/registry/
WORKDIR /app
FROM chef AS planner
COPY Cargo.toml .
COPY Cargo.lock .
COPY crates/ crates/
COPY resources/ resources/
COPY tests/ tests/
RUN cargo chef prepare --recipe-path recipe.json
FROM chef AS builder
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json
COPY Cargo.toml .
COPY Cargo.lock .
COPY crates/ crates/
COPY resources/ resources/
COPY tests/ tests/
RUN cargo build -p stalwart --no-default-features --features "foundationdb s3 redis azure nats enterprise" --release
FROM debian:trixie-slim AS runtime
COPY --from=builder --chmod=0755 /app/target/release/stalwart /usr/local/bin/stalwart
COPY --from=builder /usr/lib/libfdb_c.so /usr/lib/libfdb_c.so
RUN export DEBIAN_FRONTEND=noninteractive && \
apt-get update && \
apt-get install -yq --no-install-recommends ca-certificates curl libcap2-bin && \
rm -rf /var/lib/apt/lists/* && \
groupadd -r -g 2000 stalwart && \
useradd -r -u 2000 -g 2000 -s /usr/sbin/nologin -M stalwart && \
mkdir -p /etc/stalwart /var/lib/stalwart && \
chown stalwart:stalwart /etc/stalwart /var/lib/stalwart && \
setcap 'cap_net_bind_service=+ep' /usr/local/bin/stalwart
USER stalwart
WORKDIR /var/lib/stalwart
VOLUME ["/etc/stalwart", "/var/lib/stalwart"]
EXPOSE 443 25 110 587 465 143 993 995 4190 8080
ENV STALWART_HEALTHCHECK_URL=https://127.0.0.1:443/healthz/live
HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
CMD curl -fsSk -H "X-Forwarded-For: 127.0.0.1" "$STALWART_HEALTHCHECK_URL" || curl -fsS -H "X-Forwarded-For: 127.0.0.1" http://127.0.0.1:8080/healthz/live || exit 1
ENTRYPOINT ["/usr/local/bin/stalwart"]
CMD ["--config", "/etc/stalwart/config.json"]
LICENSES/AGPL-3.0-only.txt
+235
View File
@@ -0,0 +1,235 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
Preamble
The GNU Affero General Public License is a free, copyleft license for software and other kinds of works, specifically designed to ensure cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, our General Public Licenses are intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users.
When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License which gives you legal permission to copy, distribute and/or modify the software.
A secondary benefit of defending all users' freedom is that improvements made in alternate versions of the program, if they receive widespread use, become available for other developers to incorporate. Many developers of free software are heartened and encouraged by the resulting cooperation. However, in the case of software used on network servers, this result may fail to come about. The GNU General Public License permits making a modified version and letting the public access it on a server without ever releasing its source code to the public.
The GNU Affero General Public License is designed specifically to ensure that, in such cases, the modified source code becomes available to the community. It requires the operator of a network server to provide the source code of the modified version running there to the users of that server. Therefore, public use of a modified version, on a publicly accessible server, gives the public access to the source code of the modified version.
An older license, called the Affero General Public License and published by Affero, was designed to accomplish similar goals. This is a different license, not a version of the Affero GPL, but Affero has released a new version of the Affero GPL which permits relicensing under this license.
The precise terms and conditions for copying, distribution and modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based on the Program.
To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work.
A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source.
The Corresponding Source for a work in source code form is that same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures.
When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified it, and giving a relevant date.
b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices".
c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so.
A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways:
a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b.
d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d.
A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product.
"Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made.
If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM).
The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or authors of the material; or
e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors.
All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11).
However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice.
Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party.
If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it.
A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software. This Corresponding Source shall include the Corresponding Source for any work covered by version 3 of the GNU General Public License that is incorporated pursuant to the following paragraph.
Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the work with which it is combined will remain governed by version 3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of the GNU Affero General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU Affero General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU Affero General Public License, you may choose any version ever published by the Free Software Foundation.
If the Program specifies that a proxy can decide which future versions of the GNU Affero General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program.
Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer network, you should also make sure that it provides a way for users to get its source. For example, if your program is a web application, its interface could display a "Source" link that leads users to an archive of the code. There are many ways you could offer source, and different solutions will be better for different programs; see section 13 for the specific requirements.
You should also get your employer (if you work as a programmer) or school, if any, to sign a "copyright disclaimer" for the program, if necessary. For more information on this, and how to apply and follow the GNU AGPL, see <http://www.gnu.org/licenses/>.
LICENSES/LicenseRef-SEL.txt
+213
View File
@@ -0,0 +1,213 @@
# Stalwart Enterprise License 2.0 (SELv2) Agreement
*Last Update: March 29, 2026*
PLEASE CAREFULLY READ THIS STALWART ENTERPRISE LICENSE AGREEMENT ("AGREEMENT"). THIS AGREEMENT CONSTITUTES A LEGALLY BINDING AGREEMENT BETWEEN YOU AND STALWART LABS LLC AND GOVERNS YOUR USE OF THE SOFTWARE (DEFINED BELOW). IF YOU DO NOT AGREE WITH THIS AGREEMENT, YOU MAY NOT USE THE SOFTWARE. IF YOU ARE USING THE SOFTWARE ON BEHALF OF A LEGAL ENTITY, YOU REPRESENT AND WARRANT THAT YOU HAVE AUTHORITY TO AGREE TO THIS AGREEMENT ON BEHALF OF SUCH ENTITY. IF YOU DO NOT HAVE SUCH AUTHORITY, DO NOT USE THE SOFTWARE IN ANY MANNER.
This Agreement is entered into by and between Stalwart Labs LLC and you, or the legal entity on behalf of whom you are acting.
---
## 1. DEFINITIONS
1.1. "Software" refers to the Stalwart Server Enterprise Edition software, including all its versions, updates, modifications, accompanying documentation, and related materials. The Software is self-hosted by Licensee on its own infrastructure.
1.2. "Subscription" refers to the paid access to the Software provided by Licensor to Licensee, billed on a monthly or annual basis.
1.3. "Licensor" refers to Stalwart Labs LLC, the entity providing the Software.
1.4. "Licensee" refers to the individual or entity installing, accessing, or using the Software with a valid Subscription.
1.5. "License Key" refers to the unique code provided by Licensor upon purchasing a Subscription which activates the full features of the Software. Each License Key is bound to the domain name (including all subdomains) designated by Licensee at the time of purchase.
1.6. "Source Code" refers to the human-readable version of the Software's code, as opposed to the compiled machine-readable version.
1.7. "Mailbox" refers to each individual user account or group account provisioned within the Software. The total number of Mailboxes across all domains and tenants hosted by Licensee determines the applicable Subscription tier.
1.8. "Confidential Information" refers to any non-public information disclosed by either party to the other in connection with this Agreement, whether in written, oral, electronic, or other form, that is designated as confidential or that a reasonable person would understand to be confidential given the nature of the information and circumstances of disclosure.
## 2. GRANT OF LICENSE
2.1. Licensor grants Licensee a non-exclusive, non-transferable, non-sublicensable, limited license to download, install, and use the Software during the Subscription term, subject to the terms and conditions of this Agreement.
2.2. The use of the Software is conditioned upon Licensee maintaining an active and valid paid Subscription with Licensor. The paid Subscription covers all versions of the Software and all updates and modifications released during the Subscription term.
2.3. This license grants Licensee the right to use the Software for both personal and commercial purposes. Licensee may install and operate the Software on an unlimited number of servers within its organization, host an unlimited number of domains, and host data for an unlimited number of external organizations (tenants) using the Software's multi-tenancy features. The Subscription tier is determined solely by the total number of Mailboxes provisioned. However, Licensee is expressly prohibited from reselling, leasing, sublicensing, or otherwise redistributing the Software itself.
2.4. This license is further governed by the terms and conditions set forth in any licensing agreements separately executed between Licensor and Licensee. In the event of any conflict between the terms of this Agreement and the terms of a signed licensing agreement, the terms of the signed licensing agreement shall control.
2.5. You are not granted any other rights beyond what is expressly stated herein.
## 3. LICENSE KEYS
3.1. The Software shall not be used without a valid License Key issued by Licensor.
3.2. Licensee is required to use valid License Keys issued by Licensor to run the Software, including any modified versions. Any attempts to bypass the License Key requirement is a violation of this Agreement.
3.3. Distribution or sharing of License Keys to third parties, not associated with Licensee, is strictly prohibited.
3.4. License Keys are bound to the Subscription period. Should your Subscription expire or be cancelled, all License Keys will become invalid after fifteen (15) days from the Subscription expiration or cancellation date.
3.5. Any instance of the Software using such an expired key will revert to the Community Edition functionality after the aforementioned fifteen (15) day period.
## 4. SOURCE CODE USAGE
4.1. Licensee is permitted to view, copy, and modify the Software's Source Code, as made available by Licensor, solely for Licensee's internal business use and in compliance with this Agreement's terms.
4.2. Any modifications to the Source Code do not grant Licensee any ownership rights to the original Software or any modifications. All rights, title, and interest to the Software and its Source Code remain exclusively with Licensor.
4.3. Licensee is strictly prohibited from altering, removing, or in any way tampering with the License Key validation system within the Software. Any such unauthorized modifications will be considered a material breach of this Agreement and may result in legal action.
4.4. Notwithstanding the availability of the Software's Source Code for review and limited modification, the Software and its Source Code are not open source and remain proprietary to Licensor. The provision of access to the Source Code does not confer any rights typically associated with open source software, including but not limited to the right to freely sublicense, or create derivative works for public distribution. All rights not expressly granted herein are reserved by Licensor.
4.5. Notwithstanding the foregoing, you may copy the Source Code for development and testing purposes, without requiring a Subscription.
## 5. INTELLECTUAL PROPERTY RIGHTS
5.1. The Licensor retains all rights, title, and interest in and to the Software, including all intellectual property rights therein. This Agreement does not transfer any ownership rights to the Licensee.
5.2. The Licensee must not remove, alter, or obscure any proprietary notices (including copyright and trademark notices) on the Software.
## 6. SUBSCRIPTION TERMS, RENEWAL, AND CANCELLATION
6.1. Subscriptions are available on a monthly or annual basis. The applicable fees, Mailbox tier, and billing cycle will be as set forth at the time of purchase or as subsequently agreed in writing between the parties.
6.2. Where Licensee has provided a valid payment method (such as a credit card) on file, the Subscription will automatically renew at the end of each billing cycle at the then-current rate, unless Licensee removes the payment method or cancels the Subscription prior to the renewal date. No advance cancellation notice period is required; Licensee may cancel at any time by removing the payment method on file or by notifying Licensor.
6.3. Where Licensee pays by invoice (bank transfer), the Subscription will not automatically renew. Licensor will issue an invoice notification prior to the end of the billing cycle, and the Subscription will renew only upon receipt of payment.
6.4. Upon cancellation of a Subscription by Licensee prior to the end of a paid billing cycle, Licensee is entitled to a prorated refund for the unused portion of the then-current billing period. Refunds will be calculated from the effective date of cancellation through the end of the billing cycle and will be issued within thirty (30) days of the cancellation date.
6.5. Licensor reserves the right to modify Subscription fees upon renewal. Any fee changes will be communicated to Licensee at least thirty (30) days prior to the start of the next billing cycle.
## 7. SUPPORT AND SERVICE LEVELS
7.1. All Licensees with an active Subscription have access to standard community support resources, including documentation and community forums, as made available by Licensor.
7.2. Priority email support is available exclusively to Licensees whose Subscription covers one hundred fifty (150) or more Mailboxes. Priority email support inquiries will receive an initial response within forty-eight (48) hours of receipt during Licensor's standard business hours.
7.3. The forty-eight (48) hour response time set forth in Section 7.2 constitutes a service level commitment. In the event Licensor consistently fails to meet this commitment over a period of thirty (30) consecutive days, the affected Licensee's sole remedy shall be the right to terminate the Subscription and receive a prorated refund for the unused portion of the billing cycle.
7.4. The Software is self-hosted by Licensee on Licensee's own infrastructure. Licensor does not provide hosting services and makes no guarantees regarding uptime, availability, or performance of Licensee's self-hosted deployment.
## 8. TERMINATION
8.1. Licensor may terminate this Agreement immediately upon written notice if Licensee commits a material breach of any term of this Agreement and fails to cure such breach within thirty (30) days of receiving written notice specifying the breach.
8.2. Licensor may terminate this Agreement for convenience upon thirty (30) days' written notice to Licensee. In such event, Licensee shall receive a prorated refund for the unused portion of any prepaid Subscription fees.
8.3. In the event of a termination, Licensee will be provided with written notice, sent to the email address used during Subscription registration, outlining the reasons for the termination.
8.4. Upon termination, all rights granted to Licensee under this Agreement will cease, and Licensee must promptly cease all use of the Software and destroy or delete all copies in its possession, except that Licensee may retain copies of the Source Code obtained prior to termination solely for archival purposes, subject to the continuing obligations of confidentiality and intellectual property protection set forth herein.
## 9. CONFIDENTIALITY
9.1. Each party agrees to hold the other party's Confidential Information in strict confidence and not to disclose such information to any third party, except to employees, contractors, or agents who have a need to know and are bound by confidentiality obligations no less protective than those contained herein.
9.2. Confidential Information does not include information that: (a) is or becomes publicly available through no fault of the receiving party; (b) was rightfully in the receiving party's possession prior to disclosure; (c) is independently developed by the receiving party without use of the disclosing party's Confidential Information; or (d) is rightfully obtained from a third party without restriction on disclosure.
9.3. A receiving party may disclose Confidential Information to the extent required by applicable law, regulation, or court order, provided that the receiving party gives the disclosing party prompt written notice (where legally permissible) and cooperates with the disclosing party's efforts to seek protective treatment of such information.
9.4. The obligations of confidentiality set forth in this Section shall survive the termination or expiration of this Agreement for a period of three (3) years.
## 10. LIMITATION OF LIABILITY
10.1. In no event will the Licensor be liable for any indirect, incidental, special, consequential, or punitive damages, or any loss of profits or revenues, whether incurred directly or indirectly, or any loss of data, use, goodwill, or other intangible losses, resulting from (i) your use or inability to use the Software; (ii) any unauthorized access to or use of your servers and/or any personal information stored therein.
10.2. Except for liability arising from death or personal injury caused by negligence, fraud, willful misconduct, or a party's indemnification obligations under this Agreement, Licensor's total aggregate liability for any and all claims under this Agreement shall be limited to the total Subscription fees paid by Licensee to Licensor in the twelve (12) months immediately preceding the event giving rise to the claim.
## 11. INDEMNIFICATION
11.1. Licensee agrees to indemnify, defend, and hold harmless Licensor, its officers, directors, employees, agents, licensors, suppliers, and any third-party information providers from and against all claims, losses, expenses, damages, and costs, including reasonable attorneys' fees, resulting from any violation of this Agreement or any activity related to Licensee's use or misuse of the Software (including negligent or wrongful conduct).
11.2. Licensor agrees to indemnify, defend, and hold harmless Licensee from and against any third-party claim that the Software, as provided by Licensor, infringes or misappropriates any patent, copyright, trademark, or trade secret of a third party, provided that Licensee: (a) gives Licensor prompt written notice of such claim; (b) grants Licensor sole control of the defense and settlement of such claim; and (c) provides reasonable cooperation at Licensor's expense.
11.3. If the Software becomes, or in Licensor's opinion is likely to become, the subject of an infringement claim, Licensor may at its option and expense: (a) procure for Licensee the right to continue using the Software; (b) modify or replace the Software to make it non-infringing while maintaining substantially equivalent functionality; or (c) if neither (a) nor (b) is commercially practicable, terminate this Agreement and provide Licensee with a prorated refund of any prepaid Subscription fees.
11.4. Licensor shall have no obligation under this Section for any claim arising from: (a) modifications to the Software made by Licensee; (b) use of the Software in combination with products, services, or technologies not provided by Licensor, where the infringement would not have occurred but for such combination; or (c) Licensee's continued use of a version of the Software after being notified of the availability of a non-infringing update.
## 12. DATA PROTECTION AND PRIVACY
12.1. The Software is self-hosted by Licensee, and Licensee retains sole responsibility for all data stored and processed within its deployment of the Software, including any personal data of its users or tenants.
12.2. To the extent that Licensor processes any personal data on behalf of Licensee (for example, in connection with support services or license management), such processing shall be conducted in accordance with applicable data protection laws, including but not limited to the General Data Protection Regulation (GDPR) where applicable, the California Consumer Privacy Act (CCPA) where applicable, and any other relevant data protection legislation.
12.3. Where required by applicable data protection law, the parties shall enter into a separate Data Processing Agreement ("DPA") that sets forth the terms and conditions governing Licensor's processing of personal data on behalf of Licensee.
12.4. In the event of a data breach affecting personal data processed by Licensor in connection with this Agreement, Licensor shall notify Licensee without undue delay and in any event within seventy-two (72) hours of becoming aware of the breach, and shall cooperate with Licensee in investigating and remediating the breach.
12.5. Additional details regarding Licensor's data handling practices are outlined in Licensor's Privacy Policy, which can be accessed on Licensor's website.
## 13. EXPORT COMPLIANCE
13.1. The Software may be subject to export control and sanctions laws of the United States and other jurisdictions. Licensee agrees to comply with all applicable export control laws, including without limitation the U.S. Export Administration Regulations (EAR) and the regulations administered by the U.S. Department of the Treasury's Office of Foreign Assets Control (OFAC).
13.2. Licensee represents and warrants that: (a) Licensee is not located in, organized under the laws of, or a resident of any country or territory subject to comprehensive U.S. sanctions (currently including Cuba, Iran, North Korea, Syria, and the Crimea, Donetsk, and Luhansk regions of Ukraine); (b) Licensee is not listed on any U.S. government restricted party list; and (c) Licensee will not export, re-export, or transfer the Software to any prohibited destination, entity, or individual without the required governmental authorizations.
## 14. ANTI-CORRUPTION
14.1. Each party represents and warrants that it has not and will not, in connection with this Agreement, directly or indirectly offer, pay, promise to pay, or authorize the payment of any money or anything of value to any government official, political party, or candidate for political office for the purpose of influencing any act or decision, or securing any improper advantage.
14.2. Each party shall comply with all applicable anti-corruption and anti-bribery laws, including without limitation the U.S. Foreign Corrupt Practices Act (FCPA) and the UK Bribery Act 2010.
## 15. GOVERNING LAW AND DISPUTE RESOLUTION
15.1. This Agreement shall be governed by and construed under the laws of the State of Wyoming, United States of America, without regard to its conflict of laws principles.
15.2. Any dispute, controversy, or claim arising out of or relating to this Agreement, or the breach, termination, or invalidity thereof, shall first be attempted to be resolved through good faith negotiation between the parties for a period of thirty (30) days following written notice of the dispute.
15.3. If the dispute is not resolved through negotiation within the thirty (30) day period, it shall be finally resolved by binding arbitration administered by the American Arbitration Association ("AAA") in accordance with its Commercial Arbitration Rules. The arbitration shall be conducted in Sheridan, Wyoming, before a single arbitrator. The language of the arbitration shall be English.
15.4. The arbitrator's award shall be final and binding and may be entered as a judgment in any court of competent jurisdiction. Each party shall bear its own costs and attorneys' fees in connection with the arbitration, unless the arbitrator determines otherwise.
15.5. Notwithstanding the foregoing, either party may seek injunctive or other equitable relief in any court of competent jurisdiction to protect its intellectual property rights or Confidential Information without first submitting to arbitration.
## 16. NOTICES
16.1. All notices required or permitted under this Agreement shall be in writing and shall be deemed effectively given: (a) upon personal delivery; (b) upon confirmed transmission by email; or (c) one (1) business day after deposit with a nationally recognized overnight courier service.
16.2. Notices to Licensor shall be sent to the address and email set forth in Section 21 (Contact Information) of this Agreement. Notices to Licensee shall be sent to the email address provided during Subscription registration or as subsequently updated by Licensee in writing.
## 17. ASSIGNMENT
17.1. Licensee may not transfer or assign this Agreement or any rights or obligations hereunder without the prior written consent of Licensor, except that Licensee may assign this Agreement without consent in connection with a merger, acquisition, corporate reorganization, or sale of all or substantially all of its assets, provided that the assignee agrees in writing to be bound by the terms of this Agreement.
17.2. Licensor may assign this Agreement without restriction. Any assignment in violation of this Section shall be null and void.
## 18. DISCLAIMERS AND WARRANTIES
18.1. The Software is provided "AS IS" and "AS AVAILABLE", without warranty of any kind, either express or implied, including, without limitation, warranties of merchantability, fitness for a particular purpose, and non-infringement.
18.2. Licensor does not warrant that the Software will be error-free, that access thereto will be uninterrupted, or that defects will be corrected.
18.3. Licensor warrants that, as of the date of delivery, the Software will perform substantially in accordance with the accompanying documentation for a period of ninety (90) days. Licensee's sole remedy for breach of this warranty shall be, at Licensor's option, repair or replacement of the non-conforming Software, or a refund of the Subscription fees paid for the period during which the Software was non-conforming.
## 19. FORCE MAJEURE
Neither party shall be in default or otherwise liable for any delay in or failure of its performance under this Agreement if such delay or failure arises by any reason of any event beyond the reasonable control of a party, including acts of God, the elements, earthquakes, floods, fires, epidemics, riots, failures or delays in transportation or communications, or any act or failure to act by the other party or such other party's officers, employees, agents, or contractors. The affected party shall give prompt notice to the other party and shall use commercially reasonable efforts to mitigate the effects of the force majeure event. If a force majeure event continues for more than ninety (90) days, either party may terminate this Agreement upon written notice, and Licensee shall receive a prorated refund of any prepaid Subscription fees.
## 20. SURVIVAL
The following Sections shall survive the termination or expiration of this Agreement: Section 1 (Definitions), Section 4.2 (Ownership of Modifications), Section 4.4 (Proprietary Nature of Software), Section 5 (Intellectual Property Rights), Section 9 (Confidentiality), Section 10 (Limitation of Liability), Section 11 (Indemnification), Section 12 (Data Protection and Privacy), Section 13 (Export Compliance), Section 15 (Governing Law and Dispute Resolution), and Section 20 (Survival).
## 21. SEVERABILITY
If any provision of this Agreement is held to be unenforceable or invalid for any reason, that provision shall be reformed to the extent necessary to make it enforceable and consistent with the intent of the parties, and the remaining provisions shall remain in full force and effect.
## 22. ENTIRE AGREEMENT
This Agreement constitutes the entire agreement between the Licensor and the Licensee with respect to the subject matter hereof and supersedes all prior or contemporaneous understandings regarding such subject matter. No amendment to or modification of this Agreement will be binding unless in writing and signed by the Licensor.
## 23. ACCEPTANCE
By downloading, installing, or using the Software, even without explicitly clicking on an "I Agree" button or a similar mechanism, you acknowledge that you have read, understood, and agreed to be bound by the terms and conditions of this Agreement.
## 24. CONTACT INFORMATION
If you have any questions about this Agreement, please contact Stalwart Labs LLC at:
Stalwart Labs LLC
1309 Coffeen Avenue STE 1200
Sheridan, Wyoming 82801
USA
[email protected]
README.md
+186
View File
@@ -0,0 +1,186 @@
<p align="center">
<a href="https://stalw.art">
<img src="./img/logo-red.svg" height="150">
</a>
</p>
<h3 align="center">
Secure, scalable mail & collaboration server with comprehensive protocol support 🛡️ <br/>(IMAP, JMAP, SMTP, CalDAV, CardDAV, WebDAV)
</h3>
<br>
<p align="center">
<a href="https://github.com/stalwartlabs/stalwart/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/stalwartlabs/stalwart/ci.yml?style=flat-square" alt="continuous integration"></a>
&nbsp;
<a href="https://www.gnu.org/licenses/agpl-3.0"><img src="https://img.shields.io/badge/License-AGPL_v3-blue.svg?label=license&style=flat-square" alt="License: AGPL v3"></a>
&nbsp;
<a href="https://stalw.art/docs/install/get-started"><img src="https://img.shields.io/badge/read_the-docs-red?style=flat-square" alt="Documentation"></a>
&nbsp;
<a href="https://github.com/stalwartlabs/stalwart/releases"><img src="https://img.shields.io/github/downloads/stalwartlabs/stalwart/total?style=flat-square" alt="downloads"></a
</p>
<p align="center">
<a href="https://mastodon.social/@stalwartlabs"><img src="https://img.shields.io/mastodon/follow/109929667531941122?style=flat-square&logo=mastodon&color=%236364ff&label=Mastodon" alt="Mastodon"></a>
&nbsp;
<a href="https://twitter.com/stalwartlabs"><img src="https://img.shields.io/twitter/follow/stalwartlabs?style=flat-square&logo=x&label=Twitter" alt="Twitter"></a>
<a href="https://discord.gg/vhqRgdhguq"><img src="https://img.shields.io/discord/923615863037390889?label=Discord&logo=discord&style=flat-square" alt="Discord"></a>
&nbsp;
<a href="https://www.reddit.com/r/stalwartlabs/"><img src="https://img.shields.io/reddit/subreddit-subscribers/stalwartlabs?label=%2Fr%2Fstalwartlabs&logo=reddit&style=flat-square" alt="Reddit"></a>
</p>
## Features
**Stalwart** is an open-source mail & collaboration server with JMAP, IMAP4, POP3, SMTP, CalDAV, CardDAV and WebDAV support and a wide range of modern features. It is written in Rust and designed to be secure, fast, robust and scalable.
Key features:
- **Email** server with complete protocol support:
- JMAP:
* [JMAP for Mail](https://datatracker.ietf.org/doc/html/rfc8621) server.
* [JMAP for Sieve Scripts](https://www.ietf.org/archive/id/draft-ietf-jmap-sieve-22.html).
* [WebSocket](https://datatracker.ietf.org/doc/html/rfc8887), [Blob Management](https://www.rfc-editor.org/rfc/rfc9404.html) and [Quotas](https://www.rfc-editor.org/rfc/rfc9425.html) extensions.
- IMAP:
* [IMAP4rev2](https://datatracker.ietf.org/doc/html/rfc9051) and [IMAP4rev1](https://datatracker.ietf.org/doc/html/rfc3501) server.
* [ManageSieve](https://datatracker.ietf.org/doc/html/rfc5804) server.
* Numerous [extensions](https://stalw.art/docs/development/rfcs#imap4-and-extensions) supported.
- POP3:
- [POP3](https://datatracker.ietf.org/doc/html/rfc1939) server.
- [STLS](https://datatracker.ietf.org/doc/html/rfc2595) and [SASL](https://datatracker.ietf.org/doc/html/rfc5034) support as well as other [extensions](https://datatracker.ietf.org/doc/html/rfc2449).
- SMTP:
* SMTP server with built-in [DMARC](https://datatracker.ietf.org/doc/html/rfc7489), [DKIMv2](https://datatracker.ietf.org/doc/draft-ietf-dkim-dkim2-spec/), [DKIMv1](https://datatracker.ietf.org/doc/html/rfc6376), [SPF](https://datatracker.ietf.org/doc/html/rfc7208) and [ARC](https://datatracker.ietf.org/doc/html/rfc8617) support for message authentication.
* Strong transport security through [DANE](https://datatracker.ietf.org/doc/html/rfc6698), [MTA-STS](https://datatracker.ietf.org/doc/html/rfc8461) and [SMTP TLS](https://datatracker.ietf.org/doc/html/rfc8460) reporting.
* Automated DKIM key rotation and management.
* Inbound throttling and filtering with granular configuration rules, sieve scripting, MTA hooks and milter integration.
* Distributed virtual queues with delayed delivery, priority delivery, quotas, routing rules and throttling support.
* Envelope rewriting and message modification.
- **Collaboration** server:
- Calendaring and scheduling:
- [CalDAV](https://datatracker.ietf.org/doc/html/rfc4791) and [CalDAV Scheduling](https://datatracker.ietf.org/doc/html/rfc6638) support.
- [JMAP for Calendars](https://datatracker.ietf.org/doc/html/draft-ietf-jmap-calendars-24) support.
- Contact management:
- [CardDAV](https://datatracker.ietf.org/doc/html/rfc6352) support.
- [JMAP for Contacts](https://datatracker.ietf.org/doc/html/rfc9610) support.
- File storage:
- [WebDAV](https://datatracker.ietf.org/doc/html/rfc4918) support.
- [JMAP for File Storage](https://datatracker.ietf.org/doc/html/draft-ietf-jmap-filenode-03) support.
- Sharing with fine-grained access controls:
- [WebDAV ACL](https://datatracker.ietf.org/doc/html/rfc3744) support.
- [JMAP Sharing](https://datatracker.ietf.org/doc/html/rfc9670) support.
- **Spam** and **Phishing** built-in filter:
- Comprehensive set of filtering **rules** on par with popular solutions.
- LLM-driven spam filtering and message analysis.
- Statistical **spam classifier** with collaborative filtering, automatic training capabilities and address book integration.
- DNS Blocklists (**DNSBLs**) checking of IP addresses, domains, and hashes.
- Collaborative digest-based spam filtering with **Pyzor**.
- **Phishing** protection against homographic URL attacks, sender spoofing and other techniques.
- Trusted **reply** tracking to recognize and prioritize genuine e-mail replies.
- Sender **reputation** monitoring by IP address, ASN, domain and email address.
- **Greylisting** to temporarily defer unknown senders.
- **Spam traps** to set up decoy email addresses that catch and analyze spam.
- **Flexible**:
- Pluggable storage backends with **RocksDB**, **FoundationDB**, **PostgreSQL**, **mySQL**, **SQLite**, **S3-Compatible**, **Azure** and **Redis** support.
- Full-text search available in 17 languages using the built-in search engine or via **Meilisearch**, **ElasticSearch**, **OpenSearch**, **PostgreSQL** or **mySQL** backends.
- Sieve scripting language with support for all [registered extensions](https://www.iana.org/assignments/sieve-extensions/sieve-extensions.xhtml).
- Email aliases, mailing lists, subaddressing and catch-all addresses support.
- Automated DNS management.
- Automatic account configuration and discovery with [PACC](https://datatracker.ietf.org/doc/draft-ietf-mailmaint-pacc/), [autoconfig](https://datatracker.ietf.org/doc/draft-ietf-mailmaint-autoconfig/) and [autodiscover](https://learn.microsoft.com/en-us/exchange/architecture/client-access/autodiscover?view=exchserver-2019).
- Multi-tenancy support with domain and tenant isolation.
- Disk quotas per user and tenant.
- **Secure and robust**:
- Encryption at rest with **S/MIME** or **OpenPGP**.
- Automatic TLS certificate provisioning with [ACME](https://datatracker.ietf.org/doc/html/rfc8555) using `TLS-ALPN-01`, `DNS-01`, `DNS-PERSIST-01` or `HTTP-01` challenges.
- Automated blocking of IP addresses that attack, abuse or scan the server for exploits.
- Rate limiting.
- Security audited (read the [report](https://stalw.art/blog/security-audit)).
- Memory safe (thanks to Rust).
- **Scalable and fault-tolerant**:
- Designed to handle growth seamlessly, from small setups to large-scale deployments of thousands of nodes.
- Built with **fault tolerance** and **high availability** in mind, recovers from hardware or software failures with minimal operational impact.
- Peer-to-peer cluster coordination or with **Kafka**, **Redpanda**, **NATS** or **Redis**.
- **Kubernetes**, **Apache Mesos** and **Docker Swarm** support for automated scaling and container orchestration.
- Read replicas, sharded blob storage and in-memory data stores for high performance and low latency.
- **Authentication and Authorization**:
- **OpenID Connect** authentication.
- OAuth 2.0 authorization with [authorization code](https://www.rfc-editor.org/rfc/rfc8628) and [device authorization](https://www.rfc-editor.org/rfc/rfc8628) flows.
- **LDAP**, **OIDC**, **SQL** or built-in authentication backend support.
- System for Cross-domain Identity Management ([SCIM](https://www.rfc-editor.org/info/rfc7643/)) v2 for automated provisioning.
- Two-factor authentication with Time-based One-Time Passwords (`2FA-TOTP`)
- Application passwords (App Passwords).
- Roles and permissions.
- Access Control Lists (ACLs).
- **Observability**:
- Logging and tracing with **OpenTelemetry**, journald, log files and console support.
- Metrics with **OpenTelemetry** and **Prometheus** integration.
- Webhooks for event-driven automation.
- Alerts with email and webhook notifications.
- Live tracing and metrics.
- **Web-based administration**:
- Dashboard with real-time statistics and monitoring.
- Account, domain, group and mailing list management.
- SMTP queue management for messages and outbound DMARC and TLS reports.
- Report visualization interface for received DMARC, TLS-RPT and Failure (ARF) reports.
- Configuration of every aspect of the mail server.
- Log viewer with search and filtering capabilities.
- Self-service portal for password reset and encryption-at-rest key management.
## Screenshots
<img src="./img/demo.gif">
## Presentation
**Want a deeper dive?** Need to explain to your boss why Stalwart is the perfect fit? Whether you're evaluating options, making a case to your team, or simply curious about how it all works under the hood, these slides walk you through the key features, architecture, and benefits of Stalwart. Browse the [slides](https://stalw.art/slides) to see what makes it stand out.
## Get Started
Install Stalwart on your server by following the instructions for your platform:
- [Linux / MacOS / FreeBSD](https://stalw.art/docs/install/platform/linux)
- [Windows](https://stalw.art/docs/install/platform/windows)
- [Docker](https://stalw.art/docs/install/platform/docker)
All documentation is available at [stalw.art/docs](https://stalw.art/docs/install/get-started).
## Support
If you are having problems running Stalwart, found a bug, or just have a question, please head to the [Stalwart Support Portal](https://support.stalw.art) at [support.stalw.art](https://support.stalw.art).
Additionally, you may purchase an [Enterprise License](https://stalw.art/enterprise) to obtain priority support from Stalwart Labs LLC, including response-time commitments and a private Priority Support area on the portal.
## Contributing
We welcome contributions, but to keep the project maintainable there are a few things to know before opening a pull request. Because of the high volume of low-quality, AI-generated submissions, pull requests are limited to a list of vouched contributors; to be added, post at [support.stalw.art](https://support.stalw.art) describing the change you would like to submit, together with a link to the proposed change. At this stage only bug fixes and translations are accepted, and new features are not, unless they involve just a few lines of code.
For the full guidelines, please read [CONTRIBUTING.md](CONTRIBUTING.md).
## Roadmap
Stalwart has reached an exciting point in its journey, it’s now **feature complete**. All the core functionality and open standard email and collaboration protocols that we set out to support are in place. In other words, Stalwart already does everything you’d expect from a modern, standards-compliant mail and collaboration platform.
The next major milestone is all about refinement: finalizing the database schema and focusing on performance optimizations to ensure everything runs as efficiently and reliably as possible. Once that’s done, we’ll be ready to roll out version **1.0**.
Of course, development doesn’t stop there. The community has contributed hundreds of great ideas for improvements and new features, everything from subtle usability tweaks to entirely new integrations. You can see the full list of proposals over on our [GitHub issues](https://github.com/stalwartlabs/stalwart/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc+label%3Aenhancement). If there’s something you’d like to see prioritized, just give it a thumbs up as we plan to implement enhancements based on the community’s votes.
## Sponsorship
Your support is crucial in helping us continue to improve the project, add new features, and maintain the highest level of quality. By [becoming a sponsor](https://opencollective.com/stalwart), you help fund the development and future of Stalwart. As a thank-you, sponsors who contribute $5 per month or more will automatically receive a [Enterprise edition](https://stalw.art/enterprise/) license. And, sponsors who contribute $30 per month or more, also have access to [Premium Support](https://stalw.art/support) from Stalwart Labs.
## Funding
Part of the development of this project was funded through:
- [NGI0 Entrust Fund](https://nlnet.nl/entrust), a fund established by [NLnet](https://nlnet.nl/) with financial support from the European Commission's [Next Generation Internet](https://ngi.eu/) programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 101069594.
- [NGI Zero Core](https://nlnet.nl/NGI0/), a fund established by [NLnet](https://nlnet.nl/) with financial support from the European Commission's programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 101092990.
If you find the project useful you can help by [becoming a sponsor](https://opencollective.com/stalwart). Thank you!
## License
This project is dual-licensed under the **GNU Affero General Public License v3.0** (AGPL-3.0; as published by the Free Software Foundation) and the **Stalwart Enterprise License v2 (SELv2)**:
- The [GNU Affero General Public License v3.0](./LICENSES/AGPL-3.0-only.txt) is a free software license that ensures your freedom to use, modify, and distribute the software, with the condition that any modified versions of the software must also be distributed under the same license.
- The [Stalwart Enterprise License v2 (SELv2)](./LICENSES/LicenseRef-SEL.txt) is a proprietary license designed for commercial use. It offers additional features and greater flexibility for businesses that do not wish to comply with the AGPL-3.0 license requirements.
Each file in this project contains a license notice at the top, indicating the applicable license(s). The license notice follows the [REUSE guidelines](https://reuse.software/) to ensure clarity and consistency. The full text of each license is available in the [LICENSES](./LICENSES/) directory.
## Copyright
Copyright (C) 2020, Stalwart Labs LLC
SECURITY.md
+154
View File
@@ -0,0 +1,154 @@
# Security Policy for Stalwart
## Supported Versions
We provide security updates for the following versions of Stalwart:
| Version | Supported | End of Support |
| ------- | ------------------ | -------------- |
| 0.16.x | :white_check_mark: | TBD |
| 0.15.x | :white_check_mark: | 2026-12-01 |
| < 0.14 | :x: | Ended |
**Note**: We typically support the current major version and one previous major version. Users are strongly encouraged to upgrade to the latest version for the best security posture.
## Reporting a Vulnerability
We take the security of Stalwart very seriously. If you believe you've found a security vulnerability, we encourage you to inform us responsibly through coordinated disclosure.
### How to Report
**Do not report security vulnerabilities through public GitHub issues, discussions, or social media.**
Instead, please use one of these secure channels:
1. **Email** (preferred): Send details to `[email protected]`
2. **GitHub Security Advisories**: Use the "Report a vulnerability" button in the Security tab
3. **Backup contact**: If no response within 48 hours, email `[email protected]`
### What to Include
To help us understand and address the issue quickly, please include:
**Required Information:**
- Brief description of the vulnerability type
- Affected version(s) and components
- Steps to reproduce the issue
- Impact assessment (what could an attacker achieve?)
**Helpful Additional Details:**
- Full paths of affected source files
- Specific commit/branch where the issue exists
- Required configuration to reproduce
- Proof-of-concept code (if available)
- Suggested mitigation or fix (if you have ideas)
### Our Response Process
**Timeline Commitments:**
- **Initial acknowledgment**: Within 24 hours
- **Detailed response**: Within 72 hours
- **Status updates**: Every 7 days until resolved
- **Resolution target**: 90 days for most issues
**What We'll Do:**
1. Acknowledge your report and assign a tracking ID
2. Assess the vulnerability and determine severity
3. Develop and test a fix
4. Coordinate disclosure timeline with you
5. Release security update and publish advisory
6. Credit you in our security advisory (if desired)
## Disclosure Policy
We follow responsible disclosure principles:
- **Coordinated disclosure**: We'll work with you to determine appropriate disclosure timing
- **Typical timeline**: 90 days from report to public disclosure
- **Early disclosure**: May occur if issue is being actively exploited
- **Delayed disclosure**: May be necessary for complex issues requiring significant changes
## Scope
This security policy applies to:
**In Scope:**
- Stalwart (all supported versions)
- Official Docker images
- Documentation that could lead to insecure configurations
- Dependencies with security implications
**Out of Scope:**
- Third-party integrations or plugins
- Issues requiring physical access to the server
- Social engineering attacks
- Attacks requiring compromised credentials (unless the vulnerability enables credential compromise)
- Theoretical vulnerabilities without practical exploitation
## Security Measures
**Our Commitments:**
- Regular security audits of dependencies using `cargo audit`
- Automated security scanning in CI/CD pipeline
- Following Rust security best practices
- Prompt security updates for critical dependencies
- Security-focused code review process
**User Responsibilities:**
- Keep Stalwart updated to supported versions
- Follow security configuration guidelines
- Implement proper network security (firewalls, TLS, etc.)
- Regular security monitoring and logging
- Secure credential management
## Legal Safe Harbor
We support security research conducted in good faith. If you follow these guidelines:
**We will NOT:**
- Initiate legal action against you
- Contact law enforcement about your research
- Suspend or terminate your access to Stalwart services
**You must:**
- Only test against your own Stalwart installations
- Not access, modify, or delete user data
- Not perform testing that could degrade service availability
- Not publicly disclose the issue before coordinated disclosure
- Act in good faith and not for malicious purposes
## Recognition
We believe in recognizing security researchers who help keep Stalwart secure:
- **Security Advisory Credits**: We'll credit you in our GitHub Security Advisories (unless you prefer to remain anonymous)
- **Hall of Fame**: Significant contributors may be listed in our security acknowledgments
- **Swag**: We may send Stalwart merchandise for notable contributions
## Security Updates
**Stay Informed:**
- Subscribe to our [GitHub releases](https://github.com/stalwartlabs/stalwart/releases) for security updates
- Join our community channels for security announcements
- Enable GitHub notifications for security advisories
**Update Process:**
- Security updates are published as patch releases (e.g., 0.12.1 → 0.12.2)
- Critical vulnerabilities may receive out-of-band releases
- Docker images are updated simultaneously with releases
- Security advisories are published through GitHub Security Advisories
## Contact Information
- **Security reports**: security@stalw.art
- **General inquiries**: hello@stalw.art
- **PGP Key**: Available upon request for sensitive communications
## Additional Resources
- [Stalwart Security Incident Response Process](SECURITY_PROCESS.md)
- [Security Configuration Guide](https://stalw.art/docs/install/security)
- [Rust Security Advisory Database](https://rustsec.org/)
*This security policy is effective as of June 20, 2025 and may be updated periodically. Check back regularly for updates.*
SECURITY_PROCESS.md
+173
View File
@@ -0,0 +1,173 @@
# Stalwart Security Incident Response Checklist
## Phase 1 : Initial Assessment & Validation
### Updates
<< Use this section to detail the report received, initial assessment, and validation results >>
Example:
I've reviewed the security report and confirmed this vulnerability exists in Stalwart version X.Y.Z.
Assessment of exploitability:
- Attack complexity: [High/Medium/Low]
- Prerequisites: [Authentication required/Network access/Specific configuration/etc.]
- User interaction required: [Yes/No]
Potential impact:
- Email data confidentiality: [At risk/Not affected]
- Server integrity: [At risk/Not affected]
- Service availability: [At risk/Not affected]
- Estimated affected installations: [Number/Percentage]
### Resources
- [Stalwart Security Policy](https://github.com/stalwartlabs/stalwart/blob/main/SECURITY.md)
- [CVE Scoring Calculator](https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator)
- [Rust Security Advisory Database](https://rustsec.org/)
### Tasks
- [ ] Reproduce the vulnerability in test environment
- [ ] Assess CVSS score and severity level
- [ ] Check if vulnerability affects current stable version
- [ ] Check if vulnerability affects LTS versions (if applicable)
- [ ] Determine if this requires immediate action or can wait for next release cycle
- [ ] Document technical details and root cause
### Assessment Summary
- **Severity Level**: `Critical|High|Medium|Low`
- **CVSS Score**: `X.X`
- **Affects versions**: `X.Y.Z to X.Y.Z`
- **Root cause**: Brief technical explanation
- **Introduced in commit/version**: `commit-hash` or `vX.Y.Z`
- **Attack vector**: `Network|Local|Physical`
- **Estimated timeline for fix**: `X days/weeks`
## Phase 2: Immediate Response & Mitigation
### Updates
<< Document immediate actions taken and mitigation strategies >>
Example:
Working on hotfix for version X.Y.Z. Temporary workaround available by disabling [feature] in configuration.
### Tasks
- [ ] Implement immediate workaround if possible
- [ ] Update security advisory draft
- [ ] Prepare patch/hotfix
- [ ] Test fix thoroughly in development environment
- [ ] Prepare updated Docker images and binaries
- [ ] Draft security advisory for GitHub Security Advisories
- [ ] Consider if coordinated disclosure timeline needs adjustment
### Mitigation Details
- **Workaround available**: `Yes|No` - If yes, describe briefly
- **Fix implemented on**: `YYYY-MM-DD`
- **Patch/hotfix version**: `vX.Y.Z`
- **GitHub Security Advisory ID**: `GHSA-XXXX-XXXX-XXXX`
## Phase 3: Impact Assessment & User Analysis
### Updates
<< Analysis of potential impact on the Stalwart deployments >>
Based on telemetry data and version statistics, approximately X installations may be affected.
### Tasks
- [ ] Analyze version adoption from update checks (if available)
- [ ] Estimate number of vulnerable installations
- [ ] Assess if default configurations are vulnerable
- [ ] Review if vulnerability has been exploited (check logs, reports)
- [ ] Determine if any user data may have been compromised
- [ ] Check for indicators of active exploitation in the wild
### Analysis Notes
_Document your impact assessment process and findings_
### Impact Summary
- **Estimated vulnerable installations**: `~X out of Y`
- **Default configuration vulnerable**: `Yes|No`
- **Evidence of exploitation**: `Found|Not found|Unknown`
- **User data potentially at risk**: `Email content|Credentials|Configuration|None`
- **Confidence in assessment**: `High|Medium|Low`
## Phase 4: Communication & Release
### Updates
<< Communication strategy and release timeline >>
Security release vX.Y.Z will be published on YYYY-MM-DD with coordinated disclosure.
### Tasks
**Pre-release preparation:**
- [ ] Finalize security patch
- [ ] Prepare release notes with security details
- [ ] Update documentation if needed
- [ ] Test automated update mechanisms
- [ ] Prepare GitHub Security Advisory
**Communication channels:**
- [ ] Draft announcement for Stalwart community forum/Discord
- [ ] Prepare release announcement for GitHub
- [ ] Draft security advisory content
- [ ] Consider notification to major distributors/packagers
**Release execution:**
- [ ] Publish patched version to GitHub releases
- [ ] Update Docker images on Docker Hub
- [ ] Publish GitHub Security Advisory
- [ ] Post to community channels (Discord/forum)
- [ ] Update project website/documentation
- [ ] Submit CVE request if warranted (CVSS ≥ 4.0)
**Post-release:**
- [ ] Monitor community channels for questions
- [ ] Track adoption of security update
- [ ] Follow up on any additional reports
- [ ] Document lessons learned
### Communication Record
- **Security release published**: `YYYY-MM-DD HH:MM UTC`
- **GitHub Security Advisory**: `GHSA-XXXX-XXXX-XXXX`
- **CVE ID** (if applicable): `CVE-YYYY-XXXXX`
- **Community announcement**: [Link to forum/Discord post]
- **Estimated time to 50% adoption**: `X days/weeks`
## Post-Incident Review
### What went well?
-
### What could be improved?
-
### Action items for future incidents:
- [ ]
- [ ]
- [ ]
### Process improvements:
- [ ]
- [ ]
## Emergency Contacts
- **Primary maintainer**: hello@stalw.art
SECURITY_TEMPLATE.md
+135
View File
@@ -0,0 +1,135 @@
# Stalwart Security Advisory
**CVE ID:** CVE-YYYY-NNNNN
**Publication Date:** YYYY-MM-DD
**Last Updated:** YYYY-MM-DD
## Summary
[Provide a brief, non-technical summary of the vulnerability in 1-2 sentences]
## Affected Products and Versions
**Product:** Stalwart Mail and Collaboration Server
**Affected Versions:**
- Version X.X.X through Y.Y.Y
- [List specific affected version ranges]
**Fixed Versions:**
- Version Z.Z.Z and later
- [List all versions that include the fix]
## Vulnerability Details
### Description
[Detailed technical description of the vulnerability, including how it can be exploited]
### Impact
[Describe the potential impact if this vulnerability is exploited]
### CVSS Score
**CVSS v3.1 Base Score:** X.X ([SEVERITY])
**Vector String:** CVSS:3.1/AV:X/AC:X/PR:X/UI:X/S:X/C:X/I:X/A:X
**Severity Breakdown:**
- **Attack Vector:** [Network/Adjacent/Local/Physical]
- **Attack Complexity:** [Low/High]
- **Privileges Required:** [None/Low/High]
- **User Interaction:** [None/Required]
- **Scope:** [Unchanged/Changed]
- **Confidentiality Impact:** [None/Low/High]
- **Integrity Impact:** [None/Low/High]
- **Availability Impact:** [None/Low/High]
### CWE Classification
**CWE-XXX:** [Weakness Name]
## Technical Details
### Root Cause
[Explain the underlying cause of the vulnerability]
### Attack Scenario
[Describe a realistic attack scenario or proof of concept, without providing exploit code]
### Prerequisites
[List any conditions that must be met for successful exploitation]
## Remediation
### Recommended Actions
1. **Immediate:** Upgrade to version Z.Z.Z or later
2. **Short-term:** [Any temporary mitigation measures]
3. **Long-term:** [Any additional security hardening recommendations]
### Upgrade Instructions
```bash
# Example upgrade commands
[Provide specific upgrade instructions for Stalwart]
```
### Workarounds
[If applicable, describe any temporary workarounds for systems that cannot be immediately upgraded]
**Note:** Workarounds are temporary measures and do not fully resolve the vulnerability. Upgrading is strongly recommended.
## Detection
### Indicators of Compromise
[List any logs, patterns, or indicators that may suggest exploitation attempts]
### Log Entries
```
[Example log entries that administrators should look for]
```
## Timeline
- **YYYY-MM-DD:** Vulnerability discovered [by researcher/team name]
- **YYYY-MM-DD:** Vendor notified
- **YYYY-MM-DD:** Vendor acknowledged issue
- **YYYY-MM-DD:** Fix developed and tested
- **YYYY-MM-DD:** Fixed version released
- **YYYY-MM-DD:** Public disclosure
## Credits
This vulnerability was discovered by [Researcher Name / Organization].
## References
- Stalwart Mail Server: https://stalw.art/
- CVE Entry: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-YYYY-NNNNN
- GitHub Advisory: [Link to GitHub Security Advisory if applicable]
- Release Notes: [Link to release notes with fix]
## Contact Information
For questions or concerns regarding this advisory, please contact:
**Security Team:** hello@stalw.art
**Website:** https://stalw.art
To report security vulnerabilities in Stalwart, please follow our [responsible disclosure policy](https://github.com/stalwartlabs/stalwart/security/policy).
## Disclaimer
This advisory is provided "as is" without warranty of any kind. The information contained in this advisory is subject to change without notice.
---
**Document Version:** 1.0
**Classification:** Public
UPGRADING/v0_04.md
+49
View File
@@ -0,0 +1,49 @@
# Upgrading from `v0.4.0` to `v0.4.x`
- Replace the binary with the new version.
- Restart the service.
# Upgrading from `v0.3.x` to `v0.4.0`
## What's changed
- **Configuration File Split:** While the `config.toml` configuration file format hasn't changed much, the new version has divided it into multiple sub-files. These sub-files are now included from the new `config.toml`. This division was implemented because the config file had grown significantly, and splitting it improves organization.
- **Changes in the Sieve Interpreter Attribute Names:**
- The configuration key prefix `jmap.sieve` (JMAP Sieve Interpreter) has been renamed to `sieve.untrusted`.
- The configuration key prefix `sieve` (SMTP Sieve Interpreter) has been renamed to `sieve.trusted`.
## What's been added
- **SPAM Filter Module:** The most notable addition in this version is the SPAM filter module. It comprises:
- A TOML configuration file located at `etc/smtp/spamfilter.toml`.
- A set of Sieve scripts in `etc/spamfilter/scripts`.
- Lookup maps in `etc/spamfilter/maps`.
- **New Configuration Key:** A new key `resolver.public-suffix` has been added. This specifies the URL of the list of public suffixes.
## Migration Steps
1. **Backup:** Ensure you have a backup of your current `config.toml` file.
2. **Download Configuration Bundle:** Fetch the new configuration bundle from [this link](https://get.stalw.art/resources/config.zip). Unpack it under `BASE_DIR/etc` (for example `/opt/stalwart-mail/etc`).
3. **Update Configuration Files:** Modify the following files with your domain name, host name, certificate paths, DKIM signatures, and so on:
- `etc/config.toml`
- `etc/jmap/store.toml`
- `etc/jmap/oauth.toml`
- `etc/smtp/signature.toml`
- `etc/common/tls.toml`
4. **Adjust included files:** If you are using an LDAP directory for authentication, edit `etc/config.toml` and replace the `etc/directory/sql.toml` include with `etc/directory/ldap.toml`.
5. **Configure the SPAM Filter Database:** Set up and configure the SPAM filter database. More details can be found [here](https://stalw.art/docs/spamfilter/settings/database).
6. **Review All TOML Files:** Navigate to every TOML file under the `etc/` directory and make necessary changes.
7. **Update Binary:** Download and substitute the v0.4.0 binary suitable for your platform from [here](https://github.com/stalwartlabs/mail-server/releases/tag/v0.4.0).
8. **Restart Service:** Conclude by restarting the Stalwart service.
### Alternative Method:
1. **Separate Installation:** Install v0.4.0 in a distinct directory. This will auto-update all configuration files and establish the spam filter database in SQLite format.
2. **Move Configuration Files:** Transfer the configuration files from `etc/` and the SQLite spam filter database from `data/` to your current installation's directory.
3. **Replace Binary:** Move the binary from the `bin/` directory to your current installation's `data/` directory.
4. **Restart Service:** Finally, restart the Stalwart service.
We apologize for the lack of an automated migration tool for this upgrade. However, we are planning on introducing an automated migration tool in the near future. Thank you for your understanding and patience.
UPGRADING/v0_05.md
+63
View File
@@ -0,0 +1,63 @@
# Upgrading from `v0.5.2` to `v0.5.3`
- The following configuration attributes have been renamed, see [store.toml](https://github.com/stalwartlabs/mail-server/blob/main/resources/config/common/store.toml) for an example:
- `jmap.store.data` -> `storage.data`
- `jmap.store.fts` -> `storage.fts`
- `jmap.store.blob` -> `storage.blob`
- `jmap.encryption.*` -> `storage.encryption.*`
- `jmap.spam.header` -> `storage.spam.header`
- `jmap.fts.default-language` -> `storage.fts.default-language`
- `jmap.cluster.node-id` -> `storage.cluster.node-id`
- `management.directory` and `sieve.trusted.default.directory` -> `storage.directory`
- `sieve.trusted.default.store` -> `storage.lookup`
- Proxy networks are now configured under `server.proxy.trusted-networks` rather than `server.proxy-trusted-networks`. IP addresses/masks have to be defined within a set (`{}`) rather than a list (`[]`), see [server.toml](https://github.com/stalwartlabs/mail-server/blob/main/resources/config/common/server.toml) for an example.
# Upgrading from `v0.5.1` to `v0.5.2`
- Make sure that implicit TLS is enabled for the JMAP [listener](https://stalw.art/docs/server/listener) configured under `ets/jmap/listener.toml`:
```toml
[server.listener."jmap".tls]
implicit = true
```
- Optional: Enable automatic TLS with [ACME](https://stalw.art/docs/server/tls/acme).
- Replace the binary with the new version.
- Restart the service.
# Upgrading from `v0.5.0` to `v0.5.1`
- Replace the binary with the new version.
- Restart the service.
# Upgrading from `v0.4.x` to `v0.5.0`
## What's changed
- **Database Layout**: Version 0.5.0 utilizes a different database layout which is more efficient and allows multiple backends to be supported. For this reason, the database must be migrated to the new layout.
- **Configuration file changes**: The configuration file has been updated to support multiple stores, most configuration attributes starting with `store.*` and `directory.*` need to be reviewed.
- **SPAM filter**: Sieve scripts that interact with databases need to be updated. The functions `lookup` and `lookup_map` has been renamed to `key_exists` and `key_get`. It is recommended to replace all scripts with the new versions rather than updating them manually. Additionally, the SPAM database no longer requires an SQL server, it can now be stored in Redis or any of the supported databases.
- **Directory superusers**: Due to problems and confusion with the `superuser-group` attribute, the concept of a superuser group has been removed. Instead, a new attribute `type` has been added to external directories. The value of this attribute can be `individual`, `group` or `admin`. The `admin` type is equivalent to the old superuser group. The `type` attribute is required for all principals in the directory, it defaults to `individual` if not specified.
- **Purge schedules**: The attributes `jmap.purge.schedule.db` and `jmap.purge.schedule.blobs` have been removed. Instead, the purge frequency is now specified per store in `store.<name>.purge.frequency`. The attribute `jmap.purge.schedule.sessions` has been renamed to `jmap.purge.sessions.frequency`.
## What's been added
- **Multiple stores**: The server now supports multiple stores to be defined in the configuration file under `store.<name>`. Which store to use is defined in the `jmap.store.data`, `jmap.store.fts` and `jmap.store.blob` settings.
- **More backend options**: It is now possible to use `RocksDB`, `PostgreSQL` and `MySQL` as data stores. It is also now possible to store blobs in any of the supported databases instead of being limited to the filesystem or an S3-compatible storage. Full-text indexing can now be done using `Elasticsearch` and the Spam database stored in `Redis`.
- **Internal Directory**: The server now has an internal directory that can be used to store user accounts, passwords and group membership. This directory can be used instead of an external directory such as LDAP or SQL.
- **New settings**: When running Stalwart in a cluster, `jmap.cluster.node-id` allows to specify a unique identifier for each node. Messages containing the SPAM headers defined in `jmap.spam.header` are moved automatically to the user's Junk Mail folder.
- **Default Sieve stores**: For Sieve scripts such as the Spam filter that require access to a directory and a lookup store, it is now possible to configure the default lookup store and directory using the `sieve.trusted.default.directory` and `sieve.trusted.default.store` settings.
## Migration Steps
Rather than manually updating the configuration file, it is recommended to start with a fresh configuration file and update it with the necessary settings:
- Install `v0.5.0` in a distinct directory. You now have the option to use an [internal directory](https://stalw.art/docs/directory/types/internal), which will allow you to manage users and groups directly from Stalwart server. Alternatively, you can continue to use an external directory such as LDAP or SQL.
- Update the configuration files with your previous settings. All configuration attributes are backward compatible, except those starting with `store.*`, `directory.*` and `jmap.purge.*`.
- Export each account following the procedure described in the [migration guide](https://stalw.art/docs/management/database/migrate).
- Stop the old `v0.4.x` server.
- If there are messages pending to be delivered in the SMTP queue, move the `queue` directory to the new installation.
- Start the new `v0.5.0` server.
- Import each account following the procedure described in the [migration guide](https://stalw.art/docs/management/database/migrate).
Once again, we apologize for the lack of an automated migration tool for this upgrade. However, we are planning on introducing an automated migration tool once the web-admin is released in Q1 2024. Thank you for your understanding and patience.
UPGRADING/v0_06.md
+7
View File
@@ -0,0 +1,7 @@
# Upgrading from `v0.5.3` to `v0.6.0`
- In order to support [expressions](https://stalw.art/docs/configuration/expressions/overview), version `0.6.0` introduces multiple breaking changes in the SMTP server configuration file. It is recommended to download the new SMTP configuration files from the [repository](https://github.com/stalwartlabs/mail-server/tree/main/resources/config/smtp), make any necessary changes and replace the old files under `INSTALL_DIR/etc/smtp` with the new ones.
- If you are using custom subaddressing of catch-all rules, you'll need to replace these rules with expressions. Check out the updated [syntax](https://stalw.art/docs/directory/addresses).
- Message queues are now distributed and stored in the backend specified by the `storage.data` and `storage.blob` settings. Make sure to flush your SMTP message queue before upgrading to `0.6.0` to avoid losing any outgoing messages pending delivery.
- Replace the binary with the new version.
- Restart the service.
UPGRADING/v0_07.md
+33
View File
@@ -0,0 +1,33 @@
# Upgrading from `v0.6.0` to `v0.7.0`
Version `0.7.0` of Stalwart introduces significant improvements and features that enhance performance and functionality. However, it also comes with multiple breaking changes in the configuration files and a revamped database layout optimized for accessing large mailboxes. Additionally, Stalwart now supports compression for binaries stored in the blob store, further increasing efficiency.
Due to these extensive changes, the recommended approach for upgrading is to perform a clean reinstallation of Stalwart and manually migrate your accounts to the new version.
## Pre-Upgrade Steps
- Download the `v0.7.0` mail-server and CLI binaries for your platform from the [releases page](https://github.com/stalwartlabs/mail-server/releases/latest/).
- Initialize the setup on a distinct directory using the command `sudo ./stalwart-mail --init /path/to/new-install`. This command will print the administrator password required to access the web-admin.
- Create the `bin` directory using `mkdir /path/to/new-install/bin`.
- Move the downloaded binaries to the `bin` directory using the command `mv stalwart-mail stalwart-cli /path/to/new-install/bin`.
- Open `/path/to/new-install/etc/config.toml` in a text editor and comment out all listeners except the HTTP listener for port `8080`.
- Start the new installation from the terminal using the command `sudo /path/to/new-install/bin/stalwart-mail --config /path/to/new-install/etc/config.toml`.
- Point your browser to the web-admin at `http://yourserver.org:8080` and login using the auto-generated administrator password.
- Configure the new installation with your domain, hostname, certificates, and other settings following the instructions at [stalw.art/docs/get-started](https://stalw.art/docs/get-started). Ignore the part about using the installation script, we are performing a manual installation.
- Add your user accounts.
- Configure Stalwart to run as the `stalwart-mail` user and `stalwart-mail` group from `Settings` > `Server` > `System`. This is not necessary if you are using Docker.
- Stop the new installation by pressing `Ctrl+C` in the terminal.
## Upgrade Steps
- On your `v0.6.0` installation, open in a text editor the `smtp/listener.toml`, `imap/listener.toml` files and comment out all listeners except the JMAP/HTTP listener (we are going to need it to export the user accounts) and then restart the service.
- If you are using an external store, backup the database using the appropriate method for your database system.
- Create the `~/exports` directory, here we will store the exported accounts.
- Using the existing CLI tool (not the one you just downloaded as it is not compatible), export each user account using the command `./stalwart-cli -u https://your-old-server.org -c <ADMIN_PASSWORD> export account <ACCOUNT_NAME> ~/exports`.
- Stop the `v0.6.0` installation using the command `sudo systemctl stop stalwart-mail`.
- Move the old `v0.6.0` installation to a backup directory, for example `mv /opt/stalwart-mail /opt/stalwart-mail-backup`.
- Move the new `v0.7.0` installation to the old installation directory, for example `mv /path/to/new-install /opt/stalwart-mail`.
- Set the right permissions for the new installation using the command `sudo chown -R stalwart-mail:stalwart-mail /opt/stalwart-mail`.
- Start the new installation using the command `sudo systemctl start stalwart-mail`.
- Import the accounts using the new CLI tool with the command `./stalwart-cli -u http://yourserver.org:8080 -c <ADMIN_PASSWORD> import account <ACCOUNT> ~/exports/<ACCOUNT>`.
- Using the admin tool, reactivate all the necessary listener (SMTP, IMAP, etc.)
- Restart the service using the command `sudo systemctl restart stalwart-mail`.
We apologize for the complexity of the upgrade process associated with this version of Stalwart. We understand the challenges and inconveniences that the requirement for a clean reinstallation and manual account migration poses. Moving forward, an automated migration tool will be included in any future releases that necessitate changes to the database layout, aiming to streamline the upgrade process for you. Furthermore, as we approach the milestone of version 1.0.0, we anticipate that such foundational changes will become increasingly infrequent, leading to more straightforward updates. We appreciate your patience and commitment to Stalwart during this upgrade.
UPGRADING/v0_08.md
+86
View File
@@ -0,0 +1,86 @@
# Upgrading from `v0.7.3` to `v0.8.0`
Version `0.8.0` includes both performance and security enhancements that require your data to be migrated to a new database layout. Luckily version `0.7.3` includes a migration tool which should make this process much easier than previous upgrades. In addition to the new layout, you will have to change the systemd service file to use the `CAP_NET_BIND_SERVICE` capability.
## Preparation
- Upgrade to version `0.7.3` if you haven't already. If you are on a version previous to `0.7.0`, you will have to do a manual migration of your data using the Command-line Interface.
- Create a directory where your data will be exported to, for example `/opt/stalwart-mail/export`.
## Systemd service upgrade (Linux only)
- Stop the `v0.7.3` installation:
```bash
$ sudo systemctl stop stalwart-mail
```
- Update your systemd file to include the `CAP_NET_BIND_SERVICE` capability. Open the file `/etc/systemd/system/stalwart-mail.service` in a text editor and add the following lines under the `[Service]` section:
```
User=stalwart-mail
Group=stalwart-mail
AmbientCapabilities=CAP_NET_BIND_SERVICE
```
- Reload the daemon:
```bash
$ systemctl daemon-reload
```
- Do not start the service yet.
## Data migration
- Stop Stalwart and export your data:
```bash
$ sudo systemctl stop stalwart-mail
$ sudo /opt/stalwart-mail/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --export /opt/stalwart-mail/export
$ sudo chown -R stalwart-mail:stalwart-mail /opt/stalwart-mail/export
```
or, if you are using the Docker image:
```bash
$ docker stop stalwart-mail
$ docker run --rm -v <STALWART_DIR>:/opt/stalwart-mail -it stalwart-mail /opt/stalwart-mail/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --export /opt/stalwart-mail/export
```
- Backup your `v0.7.3` installation:
- If you are using RocksDB or SQLite, simply rename the `data` directory to `data-backup`, for example:
```bash
$ mv /opt/stalwart-mail/data /opt/stalwart-mail/data-backup
$ mkdir /opt/stalwart-mail/data
$ chown stalwart-mail:stalwart-mail /opt/stalwart-mail/data
```
- If you are using PostgreSQL, rename the database and create a blank database with the same name, for example:
```sql
ALTER DATABASE stalwart RENAME TO stalwart_old;
CREATE database stalwart;
```
- If you are using MySQL, rename the database and create a blank database with the same name, for example:
```sql
CREATE DATABASE stalwart_old;
RENAME TABLE stalwart.b TO stalwart_old.b;
RENAME TABLE stalwart.v TO stalwart_old.v;
RENAME TABLE stalwart.l TO stalwart_old.l;
RENAME TABLE stalwart.i TO stalwart_old.i;
RENAME TABLE stalwart.t TO stalwart_old.t;
RENAME TABLE stalwart.c TO stalwart_old.c;
DROP DATABASE stalwart;
CREATE database stalwart;
```
- If you are using FoundationDB, backup your database and clean the entire key range.
- Download the `v0.8.0` mail-server for your platform from the [releases page](https://github.com/stalwartlabs/mail-server/releases/latest/) and replace the binary in `/opt/stalwart-mail/bin`. If you are using the Docker image, pull the latest image.
- Import your data:
```bash
$ sudo -u stalwart-mail /opt/stalwart-mail/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --import /opt/stalwart-mail/export
```
or, if you are using the Docker image:
```bash
$ docker run --rm -v <STALWART_DIR>:/opt/stalwart-mail -it stalwart-mail /opt/stalwart-mail/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --import /opt/stalwart-mail/export
```
- Start the service:
```bash
$ sudo systemctl start stalwart-mail
```
Or, if you are using the Docker image:
```bash
$ docker start stalwart-mail
```
UPGRADING/v0_09.md
+9
View File
@@ -0,0 +1,9 @@
# Upgrading from `v0.8.x` to `v0.9.0`
Version `0.9.0` introduces significant internal improvements while maintaining compatibility with existing database layouts and configuration file formats from version `0.8.0`. As a result, no data or configuration migration is necessary. This release focuses on enhancing performance and functionality, particularly in logging and tracing capabilities.
To upgrade to Stalwart version `0.9.0` from `0.8.x`, begin by downloading the latest version of the `stalwart-mail` binary. Once downloaded, replace the existing binary with the new version. Additionally, it's important to update the WebAdmin interface to the latest version to ensure compatibility and to access new features introduced in this release.
In terms of breaking changes, this release brings significant updates to webhooks. All webhook event names have been modified, requiring a thorough review and adjustment of existing webhook configurations. Furthermore, the update introduces hundreds of new event types, enhancing the granularity and specificity of event handling capabilities. Users should familiarize themselves with these changes to effectively integrate them into their systems.
The reason for this release being classified as a major version, despite the absence of changes to the database or configuration formats, is the complete rewrite of the logging and tracing layer. This overhaul substantially improves the efficiency and speed of generating detailed tracing and logging events, making the system more robust and facilitating easier debugging and monitoring.
UPGRADING/v0_10.md
+37
View File
@@ -0,0 +1,37 @@
# Upgrading from `v0.9.x` to `v0.10.0`
## Important Notes
- In version `0.10.0` accounts are associated with roles and permissions, which define what resources they can access. The concept of administrator or super user accounts no longer exists, now there is a single account type (the `individual` principal) which can be assigned the `admin` role or custom permissions to have administrator access.
- Due to the changes in the database layout in order to support roles and permissions, the database must be migrated to the new layout. The migration is automatic and should not require any manual intervention.
- While the database migration is automatic, it's recommended to **back up your data** before upgrading.
- The webadmin must be upgraded **before** the mail server to maintain access post-upgrade. This is true even if you run Stalwart in Docker.
## Step-by-Step Upgrade Process
- Upgrade the webadmin by clicking on `Manage` > `Maintenance` > `Update Webadmin`.
- Stop Stalwart and backup your data:
```bash
$ sudo systemctl stop stalwart-mail
$ sudo /opt/stalwart-mail/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --export /opt/stalwart-mail/export
$ sudo chown -R stalwart-mail:stalwart-mail /opt/stalwart-mail/export
```
or, if you are using the Docker image:
```bash
$ docker stop stalwart-mail
$ docker run --rm -v <STALWART_DIR>:/opt/stalwart-mail -it stalwart-mail /usr/local/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --export /opt/stalwart-mail/export
```
- Download the `v0.10.0` mail-server for your platform from the [releases page](https://github.com/stalwartlabs/mail-server/releases/latest/) and replace the binary in `/opt/stalwart-mail/bin`. If you are using the Docker image, pull the latest image.
- Start the service:
```bash
$ sudo systemctl start stalwart-mail
```
Or, if you are using the Docker image:
```bash
$ docker start stalwart-mail
```
UPGRADING/v0_11.md
+13
View File
@@ -0,0 +1,13 @@
# Upgrading from `v0.10.x` to `v0.11.0`
Version `0.11.0` introduces breaking changes to the spam filter configuration. Although no data migration is required, if changes were made to the previous spam filter, the configuration of the new spam filter should be reviewed. In particular:
- `lookup.spam-*` settings are no longer used, these have been replaced by `spam-filter.*` settings. Review the [updated documentation](http://stalw.art/docs/spamfilter/overview).
- Previous `spam-filter` and `track-replies` Sieve scripts cannot be used with the new version. They have been replaced by a built-in spam filter written in Rust.
- Cache settings have changed, see the [documentation](https://stalw.art/docs/server/cache) for details.
- Support for Pipes was removed in favor of MTA hooks and Milter.
- `config.resource.spam-filter` is now `spam-filter.resource`.
- `config.resource.webadmin` is now `webadmin.resource`.
- `authentication.rate-limit` was removed as security is handled by fail2ban.
UPGRADING/v0_12.md
+68
View File
@@ -0,0 +1,68 @@
# Upgrading from `v0.11.x` to `v0.12.x`
## Important Notes
Version `0.12.x` introduces significant improvements such as zero-copy deserialization which make the new database layout incompatible with the previous version. As a result, the database must be migrated to the new layout. The migration is done automatically on startup and should not require any manual intervention. However, it is highly recommended to **back up your data** before upgrading since it is not possible to downgrade the database once it has been migrated. You may also want to run a mock migration before upgrading to ensure that everything works as expected.
In addition to the database layout changes, multiple settings were renamed:
- `server.http.*` to `http.*`.
- `jmap.folders.*` to `email.folders.*`.
- `jmap.account.purge.frequency` to `account.purge.frequency`.
- `jmap.email.auto-expunge` to `email.auto-expunge`.
- `jmap.protocol.changes.max-history` to `changes.max-history`.
- `storage.encryption.*` to `email.encryption.*`.
## Step-by-Step Upgrade Process
- Stop Stalwart in **every single node of your cluster**. If you are using the systemd service, you can do this with the following command:
```bash
$ sudo systemctl stop stalwart-mail
```
- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database. If your database does not support backups, you can use the [built-in migration utility](https://stalw.art/docs/management/migration) to export your data to a file. For example:
```bash
$ sudo /opt/stalwart-mail/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --export /opt/stalwart-mail/export
$ sudo chown -R stalwart-mail:stalwart-mail /opt/stalwart-mail/export
```
- Download the `v0.12.x` binary for your platform (which is now called `stalwart` rather than `mail-server`) from the [releases page](https://github.com/stalwartlabs/stalwart/releases/latest/) and replace the binary in `/opt/stalwart-mail/bin`. If you rename the binary from `stalwart` to `stalwart-mail`, you can keep the same systemd service file, otherwise you will need to update the service file to point to the new binary name.
- Start the service. In a cluster, you can speed up the migration process by starting all nodes at once.
```bash
$ sudo systemctl start stalwart-mail
```
- Upgrade the webadmin by clicking on `Manage` > `Maintenance` > `Update Webadmin`.
## Step-by-Step Upgrade Process (Docker)
- Stop the Stalwart container in **every single node of your cluster**. If you are using Docker, you can do this with the following command:
```bash
$ docker stop stalwart-mail
```
- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database. If your database does not support backups, you can use the `--export` command to export your data to a file. For example:
```bash
$ docker run --rm -v <STALWART_DIR>:/opt/stalwart-mail -it stalwart-mail /usr/local/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --export /opt/stalwart-mail/export
```
- The Docker image location has now changed to `stalwartlabs/stalwart` instead of `stalwartlabs/mail-server`. Pull the latest image and configure it to use your existing data directory:
```bash
$ docker run -d -ti -p 443:443 -p 8080:8080 \
-p 25:25 -p 587:587 -p 465:465 \
-p 143:143 -p 993:993 -p 4190:4190 \
-p 110:110 -p 995:995 \
-v <STALWART_DIR>:/opt/stalwart \
--name stalwart stalwartlabs/stalwart:latest
```
- Since the mount point has changed from `/opt/stalwart-mail` to `/opt/stalwart`, you will need to update your Stalwart's configuration file to reflect this change. Open the file `/opt/stalwart/etc/config.toml` and update the paths accordingly.
- Upgrade the webadmin by clicking on `Manage` > `Maintenance` > `Update Webadmin`.
UPGRADING/v0_13.md
+59
View File
@@ -0,0 +1,59 @@
# Upgrading from `v0.12.x` (and `v0.11.x`) to `v0.13.x`
## Important Notes
Version `0.13.x` introduces a significant redesign of the MTA’s delivery and queueing subsystem. This includes a transition to a new message queue serialization format and a move to a strategy-based configuration model for routing, scheduling, and delivery control. Upon first launch of version `0.13.0`, any messages currently in the outbound queue will be automatically migrated to the new format. This migration is handled internally and does not require manual intervention.
However, if your deployment includes custom routing rules or queueing logic, it is important to manually reconfigure those settings using the new strategy framework. The previous configuration format for routing is no longer compatible and will need to be updated. For systems that rely solely on the default configuration, no changes are required and the upgrade should proceed without issue.
Even if your system uses the default settings, it is strongly recommended to read the accompanying [blog announcement](https://stalw.art/blog/virtual-queues) and consult the [updated documentation](https://stalw.art/docs/mta/outbound/overview). These resources provide a full overview of the new delivery architecture and can help you determine whether any adjustments are needed for your environment.
Before applying the upgrade to a production system, take time to familiarize yourself with the new configuration structure and validate that your delivery behavior aligns with the new model.
## Step-by-Step Upgrade Process
- Stop Stalwart in **every single node of your cluster**. If you are using the systemd service, you can do this with the following command:
```bash
$ sudo systemctl stop stalwart
```
- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database. If your database does not support backups, you can use the [built-in migration utility](https://stalw.art/docs/management/migration) to export your data to a file. For example:
```bash
$ sudo /opt/stalwart/bin/stalwart --config /opt/stalwart/etc/config.toml --export /opt/stalwart/export
$ sudo chown -R stalwart:stalwart /opt/stalwart/export
```
- Download the `v0.13.x` binary for your platform from the [releases page](https://github.com/stalwartlabs/stalwart/releases/latest/) and replace the binary in `/opt/stalwart/bin`.
- Start the service. In a cluster, you can speed up the migration process by starting all nodes at once.
```bash
$ sudo systemctl start stalwart
```
- Upgrade the webadmin by clicking on `Manage` > `Maintenance` > `Update Webadmin`.
## Step-by-Step Upgrade Process (Docker)
- Stop the Stalwart container in **every single node of your cluster**. If you are using Docker, you can do this with the following command:
```bash
$ docker stop stalwart
```
- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database. If your database does not support backups, you can use the `--export` command to export your data to a file. For example:
```bash
$ docker run --rm -v <STALWART_DIR>:/opt/stalwart -it stalwart /usr/local/bin/stalwart --config /opt/stalwart/etc/config.toml --export /opt/stalwart/export
```
- Pull the latest image and restart the container:
```bash
$ docker pull stalwartlabs/stalwart:latest
$ docker start stalwart
```
- Upgrade the webadmin by clicking on `Manage` > `Maintenance` > `Update Webadmin`.
UPGRADING/v0_14.md
+48
View File
@@ -0,0 +1,48 @@
# Upgrading from `v0.13.x` to `v0.14.x`
## Binary installation
- Stop Stalwart in **every single node of your cluster**. If you are using the systemd service, you can do this with the following command:
```bash
$ sudo systemctl stop stalwart
```
- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database. If your database does not support backups, you can use the [built-in migration utility](https://stalw.art/docs/management/migration) to export your data to a file. For example:
```bash
$ sudo /opt/stalwart/bin/stalwart --config /opt/stalwart/etc/config.toml --export /opt/stalwart/export
$ sudo chown -R stalwart:stalwart /opt/stalwart/export
```
- Download the latest binary for your platform from the [releases page](https://github.com/stalwartlabs/stalwart/releases/latest/) and replace the binary in `/opt/stalwart/bin`.
- Start the service. In a cluster, you can speed up the migration process by starting all nodes at once.
```bash
$ sudo systemctl start stalwart
```
- Upgrade the webadmin by clicking on `Manage` > `Maintenance` > `Update Webadmin`.
## Containerized
- Stop the Stalwart container in **every single node of your cluster**. If you are using Docker, you can do this with the following command:
```bash
$ docker stop stalwart
```
- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database. If your database does not support backups, you can use the `--export` command to export your data to a file. For example:
```bash
$ docker run --rm -v <STALWART_DIR>:/opt/stalwart -it stalwart /usr/local/bin/stalwart --config /opt/stalwart/etc/config.toml --export /opt/stalwart/export
```
- Pull the latest image and restart the container:
```bash
$ docker pull stalwartlabs/stalwart:latest
$ docker start stalwart
```
- Upgrade the webadmin by clicking on `Manage` > `Maintenance` > `Update Webadmin`.
UPGRADING/v0_15.md
+156
View File
@@ -0,0 +1,156 @@
# Upgrading from `v0.14.x` to `v0.15.x`
Stalwart `v0.15.x` introduces **breaking changes** to both the **database schema** and some **configuration options**.
Upgrading to this version **requires a schema migration**, which is performed **automatically when Stalwart starts** for the first time on `v0.15.x`.
Because this migration modifies how data is stored and indexed, it is important to understand what will change, what will be migrated, and how the upgrade may impact your deployment—especially for larger installations.
## What's changed
Version `0.15.x` introduces significant internal improvements focused on performance, storage efficiency, and accuracy:
- **Optimized database schema**: The database schema has been redesigned to use less storage space and significantly reduce the number of read and write operations required for common tasks.
- **Rewritten search layer**: The search subsystem has been completely rewritten to use a more efficient and scalable indexing strategy.
- **Native full-text search for SQL backends**: When using **PostgreSQL** or **MySQL** as the backend, Stalwart now leverages the database’s **native full-text search capabilities**, replacing the previous custom full-text search implementation.
- **New spam classifier engine** : The spam classifier has been rewritten to use the **FTRL-Proximal** algorithm instead of the previous **Naive Bayes** implementation. This change improves classification accuracy, reduces memory usage, and reduces storage requirements for training data.
## What will be migrated
The migration process runs automatically at startup and will migrate the following data:
- **E-mail metadata**, including flags, folders, and parsed message representations. *(The raw e-mail content stored in the blob store is not migrated.)*
- **Encryption-at-rest settings**, which now also include a **spam training privacy option**
- **MTA message queue metadata** *(The actual message contents are not migrated.)*
- **Maintenance tasks**
- **Blob links** *(The underlying blobs themselves are not migrated.)*
- **Search indexes**, which will be **rebuilt** using the new indexing strategy
## Important considerations
- For deployments with **1,000 or more mailboxes**, the migration may take a **considerable amount of time**, depending on the volume of stored data.
- During migration, **Stalwart runs in read-only mode**:
- No new e-mail can be received
- No outbound e-mail can be sent
- It is **strongly recommended** to perform this upgrade during a **maintenance window**.
- By default, the migration process is **multithreaded** and uses two threads for each available CPUs. You can control the number of threads by setting the following environment variable ``NUM_THREADS=<number>``
> **Note:** If you do **not** require any of the features introduced in `v0.15.x`, consider **waiting for the next major release**, which will introduce a proxy-based architecture allowing **zero-downtime upgrades**.
## Upgrading steps
### Binary installation
- Stop Stalwart in **every single node of your cluster**. If you are using the systemd service, you can do this with the following command:
```bash
$ sudo systemctl stop stalwart
```
- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database.
- Download the latest binary for your platform from the [releases page](https://github.com/stalwartlabs/stalwart/releases/latest/) and replace the binary in `/opt/stalwart/bin`.
- Start the service. In a cluster, you can speed up the migration process by starting all nodes at once.
```bash
$ sudo systemctl start stalwart
```
### Containerized
- Stop the Stalwart container in **every single node of your cluster**. If you are using Docker, you can do this with the following command:
```bash
$ docker stop stalwart
```
- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database.
- Pull the latest image and restart the container:
```bash
$ docker pull stalwartlabs/stalwart:latest
$ docker start stalwart
```
## Post-upgrade steps
After the upgrade and migration complete, several follow-up steps are required or recommended:
- **Upgrade the webadmin**: Upgrade the webadmin interface by navigating to ``Manage → Maintenance → Update Webadmin``
- **Update the spam rules**: Download and apply the latest spam rules from the webadmin ``Manage → Maintenance → Update Spam rules``
- **Update search settings**: Review the updated documentation for search settings, as some configuration options have changed. In particular, the Elasticsearch backend now uses **different authentication settings** than previous versions.
- **Rebuild search indexes**: All search indexes must be rebuilt to take advantage of the new indexing strategy. This can be done from the webadmin interface ``Manage → Maintenance``.
- **Recalculate disk quotas for all accounts**: This step is **not required immediately**, but it is recommended to perform it at some point after the upgrade. The new version includes additional metadata in quota calculations, so recalculating ensures accurate disk usage reporting.
```bash
$ curl -X DELETE https://myserver.org/api/store/quota/<account_name> -u <admin_user>:<admin_pass> -k
```
- **Delete deprecated spam classifier keys**: Remove deprecated spam classifier keys from the memory store. These are the keys starting with the integer prefixes `12` to `16` and `17` to `18`:
- If you are using Redis:
```bash
$ for code in {12..18}; do
char=$(printf "\\x$(printf '%02x' $code)")
redis-cli --scan --pattern "${char}*" | xargs -r redis-cli DEL
done
```
- If you are using your database as the in-memory store:
```bash
$ /opt/stalwart/bin/stalwart --config /opt/stalwart/etc/config.toml --console
Stalwart Server v0.15.2 Data Store CLI
> delete y\x0c\x00 y\x12\xff
> delete m\x0c\x00 m\x12\xff
> exit
```
- If you are using your database as the in-memory store with Docker:
```bash
$ docker stop stalwart
$ docker run -it --rm \
-v <STALWART_DIR>:/opt/stalwart \
--entrypoint /usr/local/bin/stalwart \
stalwartlabs/stalwart:latest \
--config /opt/stalwart/etc/config.toml --console
Stalwart Server v0.15.2 Data Store CLI
> delete y\x0c\x00 y\x12\xff
> delete m\x0c\x00 m\x12\xff
> exit
$ docker start stalwart
```
## Troubleshooting
### Interrupted or stopped migration
If the migration process is interrupted or stopped, it can be **resumed automatically** by simply restarting Stalwart.
### `Data corruption detected` error
If you see an error message similar to: ``Data corruption detected``. This indicates that **another node wrote data using the old format while the migration was in progress**. This usually happens when the cluster was **not fully stopped** before starting the upgrade.
In order to resolve this issue, follow these steps:
1. Stop **all** Stalwart nodes.
2. Ensure **all nodes are upgraded** to `v0.15.x`.
3. Start the nodes again.
### Forcing a migration
If the migration does not resume because the node responsible for it already marked it as completed, you can force migration using environment variables:
- **Force re-migration of MTA queue metadata**: ``FORCE_MIGRATE_QUEUE=4``
- **Force re-migration of blob links**: ``FORCE_MIGRATE_BLOBS=4``
- **Force re-migration of a specific account**: ``FORCE_MIGRATE_ACCOUNT=<account-id>``
- **Force re-migration of all data**: ``FORCE_MIGRATE=4``
Use these options with care and only when necessary.
UPGRADING/v0_16.md
+605
View File
@@ -0,0 +1,605 @@
# Upgrading from `v0.15.x` to `v0.16.x`
Stalwart `v0.16.x` introduces **significant breaking changes** that make its configuration and management layer **completely incompatible** with every previous release. The database layout used to store user data (emails, calendars, contacts, files, blobs, search indexes) is **not** affected by this change, so message bodies, mailboxes, calendar events, and shared files remain on disk unchanged. What does change is **how the server is configured and managed**, and because those records live inside the same database, a multi-step migration is required.
Before continuing, please read this document in full. Skipping steps will leave the server in an unrecoverable state and will require restoring from a backup.
If any step below raises questions, a dedicated discussion thread for the `v0.16` upgrade is open at https://support.stalw.art. The earlier design discussion that led to these changes is also public at https://github.com/stalwartlabs/stalwart/discussions/2892 and describes the user-reported problems that motivated each breaking change.
## A note on downtime
Email is a critical service, and we understand that a forced maintenance window is disruptive: in some environments it is simply not an option. The breaking changes in `v0.16` are not cosmetic. Stalwart has been under continuous development for close to five years; in that time the feature set and the user base have both grown well beyond what the original configuration and management layer was designed for. The gap between what users need and what the old architecture can cleanly support has widened to the point where a redesign was unavoidable: and the redesign itself unlocks a long list of frequently-requested features that were simply not implementable under the previous model. The storage layer is untouched by all of this: emails, calendars, contacts, files, and every other piece of user data stay exactly where they are. The migration is about configuration, not about data.
Operators who cannot accept downtime should **wait**. In the next two to three weeks we plan to release two tools that work together:
- A **zero-downtime migration utility** that moves data (accounts, mailboxes, calendars, contacts, files) from an existing (`v0.15.x` or below) deployment to a freshly-installed `v0.16.x` deployment one account at a time, while both servers are running.
- A **proxy** that sits in front of both deployments and routes each incoming connection to the server that currently owns that account. As accounts are migrated one by one, the proxy transparently shifts their traffic from the old deployment to the new one, so end users never notice a cutover.
Together, these let operators migrate a live production deployment on an account-by-account basis with no scheduled maintenance window. When those tools are available, the instructions in this document will be superseded for most deployments. Everyone else can follow the manual steps below during a scheduled maintenance window.
## What has changed
### No more TOML configuration files
The previous server used one or more TOML files, with some settings living on local disk and others living in the database. In `v0.16` there is a single small `config.json` on disk that describes **only** the datastore (the database Stalwart uses to keep everything else). Every other configuration and management setting: domains, accounts, mail routing, DKIM signatures, storage backends, rate limits, spam rules, and so on: is now stored inside that datastore as a **JMAP object**. JMAP ("JSON Meta Application Protocol") is the JSON-based API Stalwart uses to expose its data; treating configuration as JMAP objects means the same API that serves email metadata also serves server configuration.
This change is driven by two real problems with the old model. First, in a **clustered deployment** every node had to carry its own copy of the configuration file and stay in lockstep with every other node. Divergence was easy to introduce and hard to debug, and it made distributed deployments unnecessarily fragile. Centralising everything in the database means configuration is consistent across the cluster by definition. Second, the **split between "settings in the file" and "settings in the database"** was a persistent source of user confusion: the same conceptual setting had to be documented in two places depending on where it happened to live, and administrators routinely edited the wrong one. A single unified model removes that entire category of mistake, and it gives management tooling (the WebUI and the CLI) a complete view of the system.
**For Ansible, NixOS, Terraform, and other declarative tooling:** the small `config.json` is still a plain file and can be managed with existing tooling exactly as before. Everything that used to live in TOML is now managed through [`stalwart-cli apply`](https://stalw.art/docs/management/cli/apply), which accepts a declarative plan file and idempotently reconciles the live server state to match it, creating what is missing, updating what has changed, and removing what the plan no longer declares. This is the same pattern used by CockroachDB (cluster settings via SQL/CLI), Consul (KV store), Elasticsearch (`PUT /_cluster/settings`), and HashiCorp Vault (CLI/API for policies and secrets); infrastructure-as-code tooling targets the API rather than a file. The workflow becomes: commit the declarative plan to version control, deploy `config.json` through existing tooling, and invoke `stalwart-cli apply` as an idempotent step in a playbook or activation script.
### REST API replaced by JMAP
The `/api/...` endpoints from previous releases no longer exist. All management operations happen through **JMAP objects** reachable at `/jmap`. JMAP (RFC 8620) is a well-specified, transport-efficient protocol with first-class support for batch operations, push notifications, and fine-grained change tracking. Stalwart already speaks JMAP for email: extending it to administration gives operators and integrators a single consistent protocol for interacting with the entire server. In practice this means dozens of configuration changes can be applied in a single round-trip (the `apply` command uses this), any JMAP client library works against the management surface, and the same authentication flow covers both mail access and administration. Existing scripts and integrations that called the old REST endpoints must be updated; the new CLI is the straightforward replacement for most of them.
### Account names must be email addresses
Every user and group principal now has a **local part** (the name) and an associated **domain**. In previous releases an account could be a bare string such as `alice`; in `v0.16` it must be `[email protected]`. The migration script handles this automatically: accounts without a domain are assigned the default domain of the deployment (chosen by scanning existing principals for the most common domain), so no users are lost during conversion.
To avoid locking existing users out of their mail clients on the first login after the upgrade, `v0.16` **automatically appends the default domain** when a client authenticates with a bare username. Administrators running an **external directory** (LDAP, SQL, etc.), however, do need to update their directory filters to query by full email address rather than by bare account name; the old filters will no longer match.
**CalDAV, CardDAV, and WebDAV clients need one manual adjustment.** These protocols use the account name as part of the URL path (for example `/dav/cal/alice`), and because the account name is now a full email address, that path changes. The `@` character is reserved in URLs and must be encoded as `%40`, so the equivalent path in `v0.16` becomes `/dav/cal/alice%40example.com`. Authentication itself still works (the server accepts the bare username and appends the default domain, as described above), but calendar, contact, and file sync will stop working until each client is reconfigured to point at the new path. It is a good idea to notify users before the upgrade so that they can update their calendar and contacts accounts in Apple Calendar, Thunderbird, DAVx⁵, and similar clients.
Two reasons drove this requirement. The first is **support for multiple external directories simultaneously**: when account names are bare strings there is no reliable way to tell which directory owns a given username, whereas email addresses are naturally namespaced by domain and make that mapping unambiguous. The second, and more consequential, reason is the **PACC specification** ([draft-ietf-mailmaint-pacc](https://datatracker.ietf.org/doc/draft-ietf-mailmaint-pacc/)): the IETF's replacement for the fragmented collection of autoconfig / autodiscover / SRV-record mechanisms that mail clients use today to discover server settings. PACC expects login names shaped like email addresses; when they are not, the server has to reveal whether a given account exists just to disambiguate the login, which is exactly the privacy leak the spec is designed to prevent. Aligning account names with email addresses is what lets Stalwart implement PACC correctly.
PACC also brings OAuth into the autodiscovery flow, and because the draft originates from Apple, a correct PACC implementation is the path to supporting Apple Mail clients with OIDC and MFA: a long-standing user request that only becomes possible once this groundwork is in place.
## What has been added
- **A brand-new WebUI**, rewritten from scratch on top of the new JMAP-based management API.
- **A brand-new CLI** (`stalwart-cli`) that also uses the JMAP API and can be used for day-to-day administration, scripted deployments, and infrastructure-as-code workflows. Full documentation is available at https://stalw.art/docs/management/cli.
- **[Over one hundred feature requests and bug fixes](https://github.com/stalwartlabs/stalwart/blob/main/CHANGELOG.md#0160---2026-xx-xx)** across every subsystem.
## Evaluate `v0.16` before migrating
Because so much has changed, `v0.16` will feel like a different product at first contact. Concepts have been renamed, some have been removed, and several new ones have been introduced.
It is **strongly recommended** that operators first install a fresh `v0.16` instance in a Docker container or a throwaway virtual machine, log into the new WebUI, and spend time becoming familiar with how configuration works in the new release. This avoids the situation where a critical production upgrade is the first time an operator sees the new interface.
A second, equally important benefit: any settings created in the test deployment (directory integrations, SMTP listeners, spam rules, rate limits, TLS providers, etc.) can be exported using the [`snapshot`](https://stalw.art/docs/management/cli/overview/snapshot) command. The resulting JSON file is an `apply` plan that can be fed directly into the production instance after the migration completes. Time spent on a test deployment is not thrown away.
## How the migration works
The migration is a **multi-step, offline** process. At a high level:
1. If the server is still on a version older than `v0.15.x`, it must first be upgraded to `v0.15.x`. The `v0.16` migration tooling does not support anything older. Operators who cannot upgrade to `v0.15.x` now should wait for the zero-downtime proxy described above, which will perform a direct migration from older releases.
2. A Python helper script is run against the live `v0.15.x` server. It downloads the current settings and principals, converts them to the new format, and produces two files: `config.json` (the new on-disk datastore configuration) and `export.json` (a snapshot of everything else, in a format that the new CLI can replay).
3. The `v0.15.x` server is stopped and its database is backed up.
4. The `v0.16` binary (or Docker image) is started in **recovery mode**. On first start it detects the old data, wipes the pieces that are no longer compatible, migrates the spam classifier model, and comes up listening on a single HTTP port (`8080`) exposing the management API.
5. `stalwart-cli apply` replays `export.json` (and, optionally, any snapshots from the test deployment) against the recovery-mode server.
6. Recovery mode is disabled, the service manager (systemd / init.d / Docker) is reconfigured to use the new `config.json`, and the server is restarted normally.
7. Post-migration tasks are triggered from the WebUI to recalculate disk quotas.
The following sections describe each step in detail.
> **Note for clustered deployments.** Before starting the migration, **every node in the cluster must be stopped**. If even one node is left running on `v0.15.x` while another is being upgraded, it will write records in the old format and cause data corruption that can only be repaired by manually deleting the offending keys. This requirement is repeated in the binary and Docker sections below, but it applies globally.
## Step 1: Convert existing settings into a configuration snapshot
This step is **independent of how Stalwart is deployed** and **does not require stopping the server**. The migration script talks to the running `v0.15.x` server over its management API and produces two JSON files on the machine where it is run. Running this step early is encouraged: it gives the operator a chance to review the generated files before touching the server, and to rerun the conversion with different options if needed.
### Download the migration script
Download the script from the Stalwart repository:
```bash
$ curl -fLO https://raw.githubusercontent.com/stalwartlabs/stalwart/refs/heads/main/resources/scripts/migrate_v016.py
```
Review the script before running it. It is a single self-contained Python file and makes no changes to the running server: it only reads configuration and principal data.
### Create a Python virtual environment
A virtual environment (`venv`) is a self-contained Python setup that keeps installed libraries out of the system-wide Python install. This avoids polluting the host Python and lets the script run on systems where `pip` installs are restricted.
```bash
$ python3 -m venv .venv
$ source .venv/bin/activate
(.venv) $ pip install requests urllib3
```
The first command creates the environment in a `.venv/` directory. The second activates it (the shell prompt usually gains a `(.venv)` prefix). The third installs the only two libraries the script needs.
### Dump the live `v0.15.x` settings
The script has two subcommands. The first, `dump`, connects to the running server and downloads its settings and principals into two files on disk:
```bash
(.venv) $ python migrate_v016.py dump \
--url https://mail.example.com \
--username admin \
--password adminPassword \
--settings settings.json \
--principals principals.json
```
Replace the URL and credentials with those of the `v0.15.x` server. The admin account must have permission to read all settings and principals. Output files default to `settings.json` and `principals.json` in the current directory. These files are plain JSON: opening them in a text editor to inspect their contents is encouraged.
### Convert the dump to the new format
The second subcommand, `convert`, reads the two dump files and produces the two files that the new server will consume:
```bash
(.venv) $ python migrate_v016.py convert \
--settings settings.json \
--principals principals.json \
--config config.json \
--output export.json
```
This produces:
- `config.json`: the new on-disk datastore configuration. This is the file the `v0.16` server will be pointed at on startup. It is small, because it describes only the datastore (data store, blob store, search store, in-memory store).
- `export.json`: a snapshot of every other piece of state the script could convert, in the format consumed by `stalwart-cli apply`. This file will be replayed against the `v0.16` server once it is running in recovery mode.
### What the script converts: and what it does not
The conversion is intentionally conservative. Only the following settings are migrated, because the rest have changed enough that automatic mapping would do more harm than good:
- **User accounts, groups, and mailing lists** (with their aliases and memberships)
- **Tenants** (in multi-tenant deployments)
- **Domain names** (including those discovered by scanning the addresses of existing accounts)
- **Data store, blob store, full-text search store, and in-memory store** settings
- **DKIM signatures** (except `rsa-sha1`, which is obsolete and not supported in `v0.16`)
- **TLS certificates** (including those issued by the built-in ACME client)
**Everything else**: SMTP listeners, mail routing rules, rate limits, connection limits, spam filter settings, logging and telemetry configuration, authentication backends other than the ones listed above, session scripts, Sieve preludes, milter/MTA hook configuration, etc.: must be recreated on the new server.
This is the reason the test deployment recommended above is so useful: recreating the remaining settings on a test `v0.16` instance, then using [`stalwart-cli snapshot`](https://stalw.art/docs/management/cli/snapshot) to export them, turns what would otherwise be manual post-migration work into a second `apply` run. If the production deployment is close to the defaults, this is straightforward. If it has extensive customisation, plan for the time this takes.
## Step 2: Back up the database
When the `v0.16` server starts for the first time, it will **wipe** the parts of the database that are no longer compatible with the new schema. No user mail is touched, but everything below is **deleted unconditionally**:
- **Directory records.** Every user, group, tenant, OAuth client, domain, and mailing list record is removed. The mail itself stays in place on disk: it is the *directory entries* describing who owns it that are deleted. After the first `apply`, the new directory entries produced from `export.json` recreate these records with the same identities.
- **All settings.** Every stored setting is deleted. `export.json` replays what the script was able to convert; anything the script could not convert needs to be recreated manually (or via a snapshot from the test deployment).
- **DMARC, TLS, and ARF reports** (both incoming and outgoing). Report records are incompatible with the new schema.
- **Pending tasks.** Maintenance tasks queued for background execution are discarded. `v0.16` exposes a task panel in the WebUI, and the equivalent tasks can be triggered manually from there.
- **Telemetry data.** Metrics and trace spans are deleted.
- **Spam training samples** (but **not** the spam classifier model itself: the model is preserved and migrated).
- **Disk quotas.** All per-account disk-usage counters are reset to zero and must be recalculated after the migration (see Step 4).
Because the wipe is irreversible, a full backup of the existing data must exist **before** the new server is started.
### Embedded databases (RocksDB, SQLite)
These store everything in a single directory on disk (typically `/var/lib/stalwart/data` or `/opt/stalwart/data`). A file-level copy while the server is stopped is sufficient:
```bash
$ sudo systemctl stop stalwart # or the equivalent for the service manager in use
$ sudo cp -a /var/lib/stalwart /var/lib/stalwart.v015-backup
```
Record the path of the backup somewhere safe. If the migration fails, restoring this directory and starting the old binary returns the system to its previous state.
### PostgreSQL / MySQL backends
The database holds many tables, but only a subset needs to be captured to be able to undo the migration. Each table is a single ASCII character that corresponds to an internal Stalwart subspace. The destructive part of the migration touches the following tables:
| Table | Purpose | Priority |
|:---:|---|---|
| `s` | Settings | **Critical**: contains all server configuration |
| `d` | Directory | **Critical**: users, groups, domains, tenants, mailing lists, OAuth clients |
| `r` | Incoming reports (DMARC, TLS, ARF) | Recommended |
| `h` | Outgoing reports | Recommended |
| `b` | Legacy bitmap index | Recommended |
| `g` | Legacy full-text-search index | Recommended |
| `j` | Legacy blob-extra metadata | Recommended |
| `f` | Pending task queue | Recommended |
| `u` | Quotas (partially reset) | Recommended |
| `o` | Telemetry spans (traces) | Optional: can be very large |
| `x` | Telemetry metrics | Optional: can be very large |
| `w` | Legacy telemetry/spam-sample index | Optional: can be very large |
The telemetry tables (`o`, `x`, `w`) can grow into tens of gigabytes on busy servers. Skipping them from the backup is reasonable unless there is a specific need to preserve historical metrics or traces.
For PostgreSQL, a per-table dump looks like this:
```bash
$ pg_dump -U stalwart -d stalwart \
-t s -t d -t r -t h -t b -t g -t j -t f -t u \
-f /var/backups/stalwart-v015-critical.sql
```
The equivalent with `mysqldump`:
```bash
$ mysqldump -u stalwart -p stalwart \
s d r h b g j f u \
> /var/backups/stalwart-v015-critical.sql
```
A full database dump (`pg_dump` / `mysqldump` without the `-t` flags, or `pg_dumpall`) is the safest option if disk space allows.
## Step 3: Perform the migration
This step is the only one that requires downtime. The sequence has moving parts, and each moving part must complete before the next begins. Reading this entire section before starting is strongly encouraged.
> **Clustered deployments:** stop **every** node before beginning. Leaving a single `v0.15.x` node running while the migration is in progress will corrupt the database.
### Option A: Binary deployments (systemd / init.d)
The following instructions assume the standard FHS layout (`/usr/local/bin/stalwart`, `/etc/stalwart/config.toml`, `/var/lib/stalwart`). Operators using a custom prefix (for example `/opt/stalwart`) should substitute their paths accordingly.
**1. Download the `v0.16` binary.** Grab the release matching the target platform from https://github.com/stalwartlabs/stalwart/releases/latest. Do **not** replace the running binary yet.
**2. Stop the old service.**
- On systems with systemd:
```bash
$ sudo systemctl stop stalwart
```
- On SysV-style systems with init.d:
```bash
$ sudo service stalwart stop
```
Verify the process is gone with `ps` before continuing. In a cluster, repeat this on every node.
**3. Back up the old binary and install the new one.**
```bash
$ sudo mv /usr/local/bin/stalwart /usr/local/bin/stalwart.v015
$ sudo mv /path/to/downloaded/stalwart /usr/local/bin/stalwart
$ sudo chmod 0755 /usr/local/bin/stalwart
$ sudo chown root:root /usr/local/bin/stalwart
```
**4. Install the new `config.json`.** The file produced by the migration script in Step 1 goes where the old TOML configuration used to live:
```bash
$ sudo mv /path/to/config.json /etc/stalwart/config.json
$ sudo chown stalwart:stalwart /etc/stalwart/config.json
$ sudo chmod 0640 /etc/stalwart/config.json
```
The old `config.toml` can be kept as a reference but is no longer read by the server.
**5. Start the new binary in recovery mode from the foreground.** Running the initial migration under the service manager is discouraged: if something goes wrong, the output scrolls past in `journalctl` and the restart loop masks the cause. Instead, run it directly as the `stalwart` user so that stdout and stderr are visible in the current terminal:
```bash
$ sudo -u stalwart env \
STALWART_RECOVERY_MODE=1 \
STALWART_RECOVERY_ADMIN=admin:someTemporaryPassword \
/usr/local/bin/stalwart --config=/etc/stalwart/config.json
```
`STALWART_RECOVERY_MODE=1` tells the server to enter the one-shot migration path: wipe the incompatible subspaces listed above, migrate the spam classifier model, and then bring up **only** the management HTTP endpoint on port `8080`. Mail ports stay closed. `STALWART_RECOVERY_ADMIN=admin:someTemporaryPassword` provisions a temporary admin credential that the CLI can authenticate against: this is needed because the converted `export.json` does not grant admin rights to any user (that is deliberate; admin assignment is a deployment decision). Replace `someTemporaryPassword` with a strong value; this account exists only until a real admin is created.
The migration output will scroll past. When it finishes, the process stays in the foreground, listening on port `8080`. Leave this terminal open.
**6. Apply the exported snapshot.** From a **second terminal** (on the same host or any machine that can reach the server on port `8080`), install the new CLI (*make sure to install v1.0.2 or later*): instructions at https://stalw.art/docs/management/cli/overview: and run:
```bash
$ export STALWART_URL=http://127.0.0.1:8080
$ export STALWART_USER=admin
$ export STALWART_PASSWORD=someTemporaryPassword
$ stalwart-cli apply --file /path/to/export.json
```
A summary similar to the following should appear:
```
Plan: 0 destroy, 5 update, 6 create (…)
✓ created Tenant (…)
✓ created Domain (…)
✓ created Account (…)
…
Done: 0 destroyed, 5 updated, … created (0 failed)
```
If any operation fails, the CLI stops immediately and prints the error. Fix the root cause (usually a conflict with an object created in an earlier attempt) and rerun. `apply` is re-entrant with `--continue-on-error` when needed.
At this point, snapshots exported from the test deployment with `stalwart-cli snapshot` can also be applied, in order:
```bash
$ stalwart-cli apply --file /path/to/test-deployment-snapshot.json
```
**7. Shut down recovery mode.** Return to the terminal running the foreground server and press `Ctrl+C`. The process will exit cleanly.
**8. Reconfigure the service manager.** The systemd unit or init.d script still references the old TOML path. Update it to point at the new JSON file:
- For systemd (typically `/etc/systemd/system/stalwart.service`), locate the `ExecStart=` line and change the `--config=` argument:
```ini
ExecStart=/usr/local/bin/stalwart --config=/etc/stalwart/config.json
```
Then reload the unit:
```bash
$ sudo systemctl daemon-reload
```
- For init.d (typically `/etc/init.d/stalwart`), update the `DAEMON_ARGS` line similarly.
**9. Decide how to handle the recovery admin.** The recovery admin credential must be available the first time a real administrator logs in to create a proper admin account. Two options:
- **Preferred, if a test deployment was used:** the test-deployment snapshot applied in step 6 can already include an administrator account, in which case no further action is needed. Start the service normally.
- **Otherwise:** leave `STALWART_RECOVERY_ADMIN` in place until a real admin is created through the WebUI, then remove it. For systemd, set it via the environment file referenced by `EnvironmentFile=` in the service unit (the default Stalwart install creates `/etc/stalwart/stalwart.env` for exactly this purpose: uncomment the `STALWART_RECOVERY_ADMIN` line and set the value). For init.d, export the variable in `/etc/default/stalwart` or the distribution's equivalent. Do **not** set `STALWART_RECOVERY_MODE=1`: that is for the migration only and would put the server back into recovery mode at every restart.
**10. Start the service.**
```bash
$ sudo systemctl start stalwart # or: sudo service stalwart start
```
Verify it comes up cleanly and is listening on its normal ports. The deployment is now on `v0.16`. For reference on how a fresh `v0.16` Linux install is expected to look, see https://stalw.art/docs/install/platform/linux.
### Option B: Docker deployments
The new Docker image uses **different mount points** than the old one. Where the previous image mounted a single `/opt/stalwart` volume, the new image mounts two:
| Volume | Purpose |
|:---|---|
| `/etc/stalwart` | Configuration directory (contains `config.json`) |
| `/var/lib/stalwart` | Persistent application data (RocksDB, local blobs, bootstrap registry) |
The Docker migration uses the same recovery-mode pattern as the binary case: stop the old container, run a throwaway container in recovery mode, apply the snapshot, stop the throwaway, then start the real container.
> **Clustered deployments:** stop every container running `v0.15.x` before starting the migration on any node.
**1. Stop the old container.**
```bash
$ docker stop stalwart
```
**2. Prepare the new volumes.** Two named volumes (or two host directories, if bind-mounting) are required:
```bash
$ docker volume create stalwart-etc
$ docker volume create stalwart-data
```
For deployments where the embedded database holds user mail (RocksDB / SQLite), the contents of the old `/opt/stalwart/data` directory must be placed in the new `stalwart-data` volume before starting the recovery-mode container. The simplest way is a helper container:
```bash
$ docker run --rm \
-v <OLD_STALWART_DIR>:/old \
-v stalwart-data:/new \
alpine sh -c 'cp -a /old/data/. /new/ && chown -R 2000:2000 /new'
```
Replace `<OLD_STALWART_DIR>` with the host path that the previous container had mounted at `/opt/stalwart`. The `chown` step is required because the new image runs as UID `2000`. For deployments that use external databases (PostgreSQL, MySQL, FoundationDB, S3, Azure, Redis, NATS), skip the copy: the data already lives outside the container.
**3. Install `config.json` in the new config volume.**
```bash
$ docker run --rm \
-v /path/to/local/config.json:/src/config.json:ro \
-v stalwart-etc:/dst \
alpine sh -c 'cp /src/config.json /dst/config.json && chown 2000:2000 /dst/config.json'
```
> **Update embedded paths inside `config.json` and `export.json` for the new mount points.** The migration script writes the on-disk paths it found in the v0.15 deployment, which on the previous Docker image typically pointed at `/opt/stalwart/data` (and `/opt/stalwart/data/blobs` for the filesystem [BlobStore](https://stalw.art/docs/ref/object/blob-store)). The new image mounts persistent data at `/var/lib/stalwart` instead, so any path referencing the old location must be rewritten before the recovery container is started; otherwise the container exits with `Permission denied: /opt/stalwart/data` because UID `2000` cannot create that directory inside the container's filesystem.
>
> The migration script ships with a `--patch-paths` flag that handles the rewrite during `convert`:
>
> ```bash
> $ python migrate_v016.py convert \
> --settings settings.json --principals principals.json \
> --config config.json --output export.json \
> --patch-paths /opt/stalwart=/var/lib/stalwart
> ```
>
> `--patch-paths SOURCE=DEST` walks both emitted files and rewrites any string value beginning with the source prefix. The flag may be supplied multiple times for deployments that mount data under several legacy paths. When the script detects `/opt/stalwart` in the source settings and the flag was not passed, it prints a notice with the exact command to rerun.
>
> For deployments that already produced `config.json` and `export.json` without the flag, the equivalent in-place edit is:
>
> ```bash
> $ sed -i.bak \
> -e 's|/opt/stalwart/data/blobs|/var/lib/stalwart/blobs|g' \
> -e 's|/opt/stalwart/data|/var/lib/stalwart|g' \
> config.json export.json
> $ grep -n /opt/stalwart config.json export.json # verify clean
> ```
>
> The blob-path substitution runs first so the more general data-path rewrite does not double-rewrite it. The `.bak` files left behind by `-i.bak` are the rollback if the substitution went wrong.
>
> Skip this paragraph entirely on deployments that use external databases (PostgreSQL, MySQL, FoundationDB) and external blob backends; those deployments have no on-disk paths to rewrite.
**4. Start a temporary container in recovery mode.** This container exists only for the duration of the migration:
```bash
$ docker run -d --name stalwart-recovery \
-e STALWART_RECOVERY_MODE=1 \
-e STALWART_RECOVERY_ADMIN=admin:someTemporaryPassword \
-p 8080:8080 \
-v stalwart-etc:/etc/stalwart \
-v stalwart-data:/var/lib/stalwart \
stalwartlabs/stalwart:v0.16
```
Only port `8080` (management API) is published: mail ports stay closed in recovery mode. Watch the logs to confirm the migration completes successfully:
```bash
$ docker logs -f stalwart-recovery
```
Wait until the logs stop scrolling and settle on the message indicating the HTTP endpoint is listening.
**5. Apply the exported snapshot.** From the host (or any machine that can reach `http://<docker-host>:8080`):
```bash
$ export STALWART_URL=http://127.0.0.1:8080
$ export STALWART_USER=admin
$ export STALWART_PASSWORD=someTemporaryPassword
$ stalwart-cli apply --file /path/to/export.json
```
Follow with any snapshots captured from the test deployment:
```bash
$ stalwart-cli apply --file /path/to/test-deployment-snapshot.json
```
**6. Stop and remove the temporary container.**
```bash
$ docker stop stalwart-recovery
$ docker rm stalwart-recovery
```
**7. Start the production container.** Same image, without `STALWART_RECOVERY_MODE`, with all mail ports published:
```bash
$ docker run -d --name stalwart \
--restart unless-stopped \
-e STALWART_RECOVERY_ADMIN=admin:someTemporaryPassword \
-p 443:443 -p 8080:8080 \
-p 25:25 -p 587:587 -p 465:465 \
-p 143:143 -p 993:993 \
-p 110:110 -p 995:995 \
-p 4190:4190 \
-v stalwart-etc:/etc/stalwart \
-v stalwart-data:/var/lib/stalwart \
stalwartlabs/stalwart:v0.16
```
The `STALWART_RECOVERY_ADMIN` variable is retained deliberately so that a real administrator account can still be created through the WebUI after the first login. Once a permanent admin exists, restart the container without that environment variable to remove the back-door credential. If the test deployment snapshot applied in step 5 already includes an administrator account, the variable can be omitted from this step entirely.
For reference on the standard Docker deployment, see https://stalw.art/docs/install/platform/docker.
## Step 4: Post-migration tasks
With the server running on `v0.16`, a few follow-up actions are required to complete the upgrade.
### Log in to the admin panel
Open a browser and navigate to:
```
https://mail.example.org/admin
```
Replace `mail.example.org` with the server's hostname. Log in either with the recovery admin credential (if it is still active) or with the administrator account created via the test-deployment snapshot.
A few behavioural changes from `v0.15.x` are worth flagging before the first sign-in:
- **The WebUI is reached over HTTPS on the configured hostname only.** The OAuth, OIDC, and JMAP discovery documents `v0.16` publishes use `https://<defaultHostname>/...` exclusively in normal mode. Loading the WebUI by IP address, by container name, or over plain HTTP (for example `http://192.168.1.10:8080/admin`) will appear to load the sign-in page but will fail at the OAuth callback. Use the same hostname that was entered in Step 1 of the wizard, or that already lives on `defaultHostname` from the migrated settings.
- **`http://...:8080` is no longer the right URL for day-to-day administration.** Port `8080` carries the recovery / bootstrap HTTP listener and is intended for the migration window; once the server is running normally it stops being a valid sign-in entry point.
- **When the public HTTPS port is not `443`** (for example a reverse proxy on `:8443`, or a Docker host port mapping that diverges from the container's `443`), set the [`STALWART_HTTPS_PORT`](https://stalw.art/docs/configuration/environment-variables#public-urls) environment variable to that port and restart the server. Without it, the discovery documents will publish `https://<host>/...` (port `443` implied) and clients will be sent to a port the proxy is not listening on.
- **Plain-text mail listeners (port `587` submission, port `143` IMAP) are no longer added by default.** This is required for compliance with the PACC autoconfig draft, which only advertises implicit-TLS ports. Mail clients that were configured to connect over `587` STARTTLS will silently stop working until either the listener is recreated through the WebUI / CLI or the clients are pointed at the implicit-TLS ports (`465` for submission, `993` for IMAPS).
#### Reverse-proxy deployments
If the deployment sits behind a reverse proxy (NGINX, Traefik, Caddy, HAProxy, or similar), this is the part of the migration where proxy-related issues most often surface. The migrated `defaultHostname`, the proxy's public hostname, the proxy's listening port, and the proxy's TLS configuration all have to line up before the first sign-in completes; if any of them is off, the OAuth flow fails partway through with errors that are hard to relate back to the proxy.
The most reliable way through this step is to **bypass the proxy temporarily** for the duration of the recovery-mode `apply`, the first sign-in, and the creation of a permanent administrator. Concretely:
1. While running `stalwart-cli apply` and creating the permanent admin, point the CLI and the browser at Stalwart directly: `http://<stalwart-host>:8080` for the recovery-mode CLI session, then `https://<stalwart-host>/admin` (accepting any self-signed certificate warning) for the first WebUI sign-in.
2. Once a permanent administrator account exists and the WebUI is confirmed working, restore the reverse-proxy configuration. From this point forward, end users reach Stalwart through the proxy and the discovery documents already point at the public hostname over HTTPS.
A full description of how `v0.16` composes the published URLs, how the proxy can talk to Stalwart on either HTTP or HTTPS, and where to set `STALWART_HTTPS_PORT` for non-standard public ports lives at https://stalw.art/docs/server/reverse-proxy/overview.
### Recalculate disk quotas
Disk quotas were reset to zero during the wipe and need to be rebuilt from the actual mailbox contents. Navigate to the **Tasks** section of the admin panel and trigger the **"Recalculate disk quotas"** task. This spawns one subtask per user account, each of which scans that user's storage and updates the counter. On large deployments this may take a while to complete: progress is visible in the Tasks panel.
### Recalculate tenant quotas (multi-tenant deployments)
Only applicable when per-tenant disk quotas are in use. After the per-account recalculation has finished for every user, trigger a second task from the Tasks panel: **"Recalculate tenant quotas"**. This rolls the per-account totals up into the tenant-level counters.
### Create a permanent administrator
If the migration was performed without a snapshot from a test deployment, the only administrative credential at this point is the recovery admin defined by `STALWART_RECOVERY_ADMIN`. This credential is a back door: as long as the environment variable is set, the username and password it specifies can log in regardless of directory state. Create a real administrator account through **Management → Accounts**, verify the new account can log in, and then remove `STALWART_RECOVERY_ADMIN` from:
- the systemd environment file (for example `/etc/stalwart/stalwart.env`) and restart the service, or
- the init.d defaults file, and restart the service, or
- the Docker container's `-e` flag (redeploy the container without it).
### Review the rest of the configuration
The migration script converts directory, domain, storage, DKIM, and certificate state. Everything else: SMTP listeners, mail routing, spam rules, rate limits, retention policies, ACME, authentication backends other than those listed above: must be reviewed in the WebUI and either recreated by hand, applied from a test-deployment snapshot, or validated against the defaults that `v0.16` ships with.
## Troubleshooting
### The server fails to start after installing `config.json`
Confirm the file is valid JSON (`python -m json.tool config.json` or `jq . config.json`) and that the datastore described in it is reachable with the provided credentials. The daemon's logs will name the offending field when a field is missing or malformed.
### `stalwart-cli apply` fails partway through
Most failures come from trying to create an object whose parent does not exist yet (for example, a `DkimSignature` referencing a `Domain` that is missing from the plan). The error message names the object and the missing reference. Either edit the plan to include the missing parent, or split the `apply` into two runs using the individual snapshot files produced by the script and the test deployment.
### Recovering from a partial `apply`
`apply` runs operations in plan order and stops on the first error. When a `create` fails halfway through, every prior `create` in the same run has already been committed to the database. Re-running the same plan now fails with `primaryKeyViolation` (the objects exist) or `invalidForeignKey` (a parent that did not get created the first time is still missing).
> **Do not bulk-delete `Account` objects to recover.** The migration plan creates each account with its original v0.15 account id (the `restore-<id>` mechanism), so a migrated `Account` points at the existing v0.15 mailbox data in the data store. Deleting that `Account` schedules account destruction, which unlinks and erases all mail, calendars, and contacts stored under that id. On the community edition this runs immediately, with no retention window. Never run `delete Account` against a data store that already contains v0.15 mail.
Recovery does not require deleting accounts. An account that a partial `apply` already created is correct and is reused as-is on the next run; the only objects that need clearing are the registry-only ones that carry no mailbox data and whose re-creation would otherwise raise `primaryKeyViolation`. While the server is still in recovery mode:
```bash
$ stalwart-cli query DkimSignature --json | jq -r '.[].id' \
| stalwart-cli delete DkimSignature --stdin
$ stalwart-cli query Certificate --json | jq -r '.[].id' \
| stalwart-cli delete Certificate --stdin
$ stalwart-cli query Domain --json | jq -r '.[].id' \
| stalwart-cli delete Domain --stdin
$ stalwart-cli query Tenant --json | jq -r '.[].id' \
| stalwart-cli delete Tenant --stdin
```
`Domain` and `Tenant` hold only directory metadata and are safe to delete and recreate; `Account` is deliberately omitted. Then fix the underlying cause in `export.json` (most often a domain that fails the v0.16 hostname check, an account whose local-part contains `@`, or a stale `/opt/stalwart` path embedded by the migration script), remove from `export.json` the `create` operation for `Account` (and any other object that already committed before the failure, so re-applying it does not raise `primaryKeyViolation`), and rerun:
```bash
$ stalwart-cli apply --file export.json
```
If you must start over with the accounts as well, do not delete them: point the new deployment at an empty data store (or restore the v0.15 data-store backup) before re-running `apply`, so that destroying and recreating accounts cannot reach live mail.
If the failure was caused by data that the migration script itself produced incorrectly, also rerun the script with the latest version from `main` before applying. Fixes during the v0.16.0 / v0.16.1 window addressed several edge cases (group names containing `@`, ACME base64 padding, single-URL Redis stores, paths embedded in custom storage backends, and `%{file:...}%` / `%{env:...}%` macros in DKIM private keys and certificates, which are now expanded by the script instead of being passed through verbatim and aborting the `apply`).
For deployments where individual objects are easier to identify than to wipe wholesale, use `stalwart-cli query <type>` to list ids and `stalwart-cli delete <type> --ids <id>` to remove a specific one. The same warning applies: deleting an `Account` destroys the mail stored under it. Only `Domain`, `Tenant`, `DkimSignature`, and `Certificate` are safe to delete and recreate during recovery.
### Bootstrapping a real administrator from the CLI
When the WebUI is unreachable for any reason (TLS not yet in place, reverse proxy misconfigured, OAuth callback failing), the CLI is the supported escape hatch for promoting the first real administrator. Authenticate as the recovery admin and run:
```bash
$ export STALWART_URL='http://127.0.0.1:8080'
$ export STALWART_USER='admin'
$ export STALWART_PASSWORD='someTemporaryPassword'
$ stalwart-cli query Domain --fields id,name
$ stalwart-cli create account/user \
--field name=admin \
--field domainId=<domain-id>
$ stalwart-cli query Account --where name=admin --fields id
$ stalwart-cli update Account <account-id> \
--field 'credentials={"0":{"@type":"Password","secret":"<NEW-PASSWORD>"}}'
$ stalwart-cli update Account <account-id> \
--field 'roles={"@type":"Admin"}'
```
Once the new account can sign in to the WebUI, remove `STALWART_RECOVERY_ADMIN` from the service environment and restart the service.
### Common questions
- **`primaryKeyViolation` on a rerun of `apply`**: see *Recovering from a partial `apply`* above.
- **`Domain: create failed for create-N: invalidPatch | Invalid domain name`**: the domain in `export.json` does not pass the v0.16 hostname check (typically a missing or non-public TLD). Either correct the domain in v0.15 before redumping, or hand-edit the offending block in `export.json`.
- **`/admin` redirects to `http://<random>:8080/`**: fixed in v0.16.x; upgrade to the latest patch release.
- **"Recalculate disk quotas" not visible in the WebUI**: open *Tasks → Scheduled → Create task*, choose the *Quota recalculation* maintenance type at the per-account scope, and pick a near-future timestamp.
### `Data corruption detected` after migration
This error means one node in a cluster was left running on `v0.15.x` while another was being migrated, and the old node wrote records in the obsolete format into the shared database. Stop every node in the cluster, ensure every binary is on `v0.16`, and restart. If corruption persists, the logs name the offending keys and they can be removed with the `stalwart-cli delete` command.
### `/admin` (or `/account`) returns `404 Not Found`
The WebUI is delivered as a downloadable [Application](https://stalw.art/docs/applications/overview) bundle that the server fetches from `https://github.com/stalwartlabs/webui/releases/latest/` on first start, and then refreshes on a schedule. When the very first download fails, no bundle has been unpacked locally and every request to a WebUI mount path returns `404 Not Found`. This is the most common cause of "the server is running, port `8080` answers, but `/admin` returns 404" reports during the migration.
The fix is to make sure outbound HTTPS from the Stalwart host can reach GitHub's release storage (`github.com` and `objects.githubusercontent.com`). On a host that genuinely cannot reach the public internet, stage the WebUI bundle on an internal HTTPS server and update the [`resourceUrl`](https://stalw.art/docs/ref/object/application#resourceurl) field on the WebUI's [Application](https://stalw.art/docs/ref/object/application) record to point at the internal location. After the first successful download, subsequent failures are non-fatal: the previously installed bundle stays in service and `/admin` keeps working until the next successful refresh. The full description, including the precise hosts involved, is at https://stalw.art/docs/management/webui/overview#outbound-network-requirement.
### Rolling back
If the migration cannot be completed within the available maintenance window, the database backup captured in Step 2 can be restored and the old binary (preserved as `/usr/local/bin/stalwart.v015` in the example) started again. The `v0.16` binary will refuse to start a second time against a database that has already been migrated, so restoring the pre-migration backup is the only path back to `v0.15.x`.
## Questions
If any part of this migration is unclear, or if something does not behave as documented, please post in the dedicated upgrade discussion at https://support.stalw.art. Include:
- The deployment type (binary / Docker / clustered)
- The datastore backend (RocksDB / SQLite / PostgreSQL / MySQL / FoundationDB)
- The exact version being upgraded from (`stalwart --version`)
- Any error messages from the server log or the CLI, verbatim
We would rather answer a question than watch a deployment break. There is no such thing as an obvious question for a migration of this size.
api/v1/openapi.yml
+586
View File
@@ -0,0 +1,586 @@
openapi: 3.0.3
info:
title: Stalwart Management API
description: |
REST Management API for Stalwart server. These endpoints are helpers
that complement the JMAP API — most of the server's configuration and data
is managed via JMAP (see `POST /jmap/`). The endpoints documented here cover
interactive login, account introspection, configuration schema retrieval and
live (Server-Sent Events) telemetry streams.
version: "1.0"
license:
name: AGPL-3.0-only OR LicenseRef-SEL
servers:
- url: https://{host}
description: Stalwart server
variables:
host:
default: mail.example.com
description: The hostname of Stalwart server
security:
- bearerAuth: []
- basicAuth: []
paths:
/api/auth:
post:
operationId: login
summary: Authenticate a user and obtain an authorization code
description: |
Anonymous endpoint used by the web UI and device-flow clients to exchange
user credentials (plus optional MFA token and PKCE challenge) for an
OAuth authorization `client_code` that can then be exchanged for an
access token via `POST /auth/token`. Rate-limited as an anonymous request.
security: []
tags: [Authentication]
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/LoginRequest'
examples:
authCode:
summary: Authorization-code flow (web UI)
value:
type: authCode
accountName: [email protected]
accountSecret: s3cret
clientId: webadmin
redirectUri: https://mail.example.com/login
codeChallenge: E9Melhoa2OwvFrEMTJguCHaoeK1t8URWbuGJSstw-cM
codeChallengeMethod: S256
authDevice:
summary: Device-flow completion
value:
type: authDevice
accountName: [email protected]
accountSecret: s3cret
code: BDWP-HQPK
responses:
'200':
description: Result of the authentication attempt
content:
application/json:
schema:
$ref: '#/components/schemas/LoginResponse'
examples:
authenticated:
value:
type: authenticated
clientCode: 3F7A9C1E4B2D8E6F
mfaRequired:
value:
type: mfaRequired
failure:
value:
type: failure
'400':
$ref: '#/components/responses/BadRequest'
'429':
$ref: '#/components/responses/TooManyRequests'
/api/discover/{email}:
get:
operationId: discoverOidc
summary: Discover the OpenID Connect provider for an email address
description: |
Returns the OpenID Connect discovery document for the directory that
owns the domain part of `email`. If the domain is not bound to an
external OIDC directory, the server's own OIDC discovery document
(equivalent to `/.well-known/openid-configuration`) is returned.
Anonymous endpoint, rate-limited.
security: []
tags: [Authentication]
parameters:
- name: email
in: path
required: true
description: Email address or account name
schema:
type: string
format: email
responses:
'200':
description: OpenID Connect discovery document
content:
application/json:
schema:
type: object
description: OIDC discovery metadata (RFC 8414)
additionalProperties: true
'404':
$ref: '#/components/responses/NotFound'
'429':
$ref: '#/components/responses/TooManyRequests'
/api/account:
get:
operationId: getAccount
summary: Return the authenticated account's permissions, edition and locale
tags: [Account]
responses:
'200':
description: Account descriptor
content:
application/json:
schema:
$ref: '#/components/schemas/Account'
'401':
$ref: '#/components/responses/Unauthorized'
/api/schema:
get:
operationId: getSchemaRedirect
summary: Redirect to the versioned configuration schema URL
description: |
Redirects (302) to `/api/schema/{hash}` where `{hash}` is the SHA-256
of the current configuration schema. Use this when you do not yet know
the hash; once you have cached a schema at a given hash the immutable
cache will never require re-download.
tags: [Schema]
responses:
'302':
description: Redirect to the hashed schema URL
headers:
Location:
schema:
type: string
example: /api/schema/a1b2c3d4e5f6...
'401':
$ref: '#/components/responses/Unauthorized'
/api/schema/{hash}:
get:
operationId: getSchema
summary: Return the configuration schema at a specific hash
description: |
Returns the JSON Schema describing the full Stalwart configuration tree.
The response is always gzip-encoded (`Content-Encoding: gzip`) and served
with an immutable cache policy — the schema for a given hash never
changes. If the hash does not match the server's current schema, the
server redirects to the correct URL.
tags: [Schema]
parameters:
- name: hash
in: path
required: true
description: SHA-256 hex digest of the configuration schema
schema:
type: string
responses:
'200':
description: Gzipped JSON Schema document
headers:
Content-Encoding:
schema:
type: string
example: gzip
Cache-Control:
schema:
type: string
example: public, max-age=31536000, immutable
content:
application/json:
schema:
type: object
description: JSON Schema document describing Stalwart config
additionalProperties: true
'302':
description: Redirect to the current schema URL when the hash is stale
'401':
$ref: '#/components/responses/Unauthorized'
/api/token/delivery:
get:
operationId: issueDeliveryToken
summary: Issue a short-lived token for live delivery diagnostics
description: |
Returns a plain-text bearer token, valid for 60 seconds, that authorises
connecting to `/api/live/delivery/{target}` as a query parameter
(`?token=...`). Useful for EventSource clients that cannot send
`Authorization` headers. Requires `LiveDeliveryTest` permission.
tags: [Live Telemetry]
responses:
'200':
description: Short-lived delivery token
content:
text/plain:
schema:
type: string
example: eyJhbGciOi...
'401':
$ref: '#/components/responses/Unauthorized'
'403':
$ref: '#/components/responses/Forbidden'
/api/token/tracing:
get:
operationId: issueTracingToken
summary: Issue a short-lived token for live tracing (Enterprise)
description: |
Returns a plain-text bearer token, valid for 60 seconds, that authorises
connecting to `/api/live/tracing` as a query parameter. Requires the
`LiveTracing` permission. Available only in the Enterprise edition.
tags: [Live Telemetry]
responses:
'200':
description: Short-lived tracing token
content:
text/plain:
schema:
type: string
'401':
$ref: '#/components/responses/Unauthorized'
'403':
$ref: '#/components/responses/Forbidden'
'404':
description: Enterprise feature not available in this edition
/api/token/metrics:
get:
operationId: issueMetricsToken
summary: Issue a short-lived token for live metrics (Enterprise)
description: |
Returns a plain-text bearer token, valid for 60 seconds, that authorises
connecting to `/api/live/metrics` as a query parameter. Requires the
`LiveMetrics` permission. Available only in the Enterprise edition.
tags: [Live Telemetry]
responses:
'200':
description: Short-lived metrics token
content:
text/plain:
schema:
type: string
'401':
$ref: '#/components/responses/Unauthorized'
'403':
$ref: '#/components/responses/Forbidden'
'404':
description: Enterprise feature not available in this edition
/api/live/delivery/{target}:
get:
operationId: liveDelivery
summary: Stream outbound-delivery diagnostics as Server-Sent Events
description: |
Opens a `text/event-stream` connection that streams each stage of an
outbound delivery attempt to `target` (a domain or email address): MX
lookup, MTA-STS fetch, TLSA/DANE validation, SMTP conversation, and so
on. Each SSE frame carries a JSON-encoded `DeliveryStage` wrapped in a
single-element array (`data: [{...}]`). The stream ends with a final
`completed` event. Requires `LiveDeliveryTest` permission; may also be
authenticated via the `?token=` query parameter obtained from
`/api/token/delivery`.
tags: [Live Telemetry]
security:
- bearerAuth: []
- basicAuth: []
- liveToken: []
parameters:
- name: target
in: path
required: true
description: Target domain or email address to diagnose
schema:
type: string
- name: timeout
in: query
required: false
description: Maximum stream lifetime in seconds (minimum 1, default 30)
schema:
type: integer
minimum: 1
default: 30
responses:
'200':
description: Server-Sent Events stream of delivery-diagnose stages
content:
text/event-stream:
schema:
type: string
description: |
Series of SSE frames. Each `event: event` frame carries
`data: [<DeliveryStage JSON>]`; the final frame's stage is
`{"type":"completed"}`.
'401':
$ref: '#/components/responses/Unauthorized'
'403':
$ref: '#/components/responses/Forbidden'
/api/live/tracing:
get:
operationId: liveTracing
summary: Stream live tracing events (Enterprise)
description: |
Opens a `text/event-stream` connection streaming server trace events in
real time. Requires `LiveTracing` permission; may be authenticated via
the `?token=` query parameter from `/api/token/tracing`. Enterprise only.
tags: [Live Telemetry]
security:
- bearerAuth: []
- basicAuth: []
- liveToken: []
responses:
'200':
description: Server-Sent Events stream of tracing events
content:
text/event-stream:
schema:
type: string
'401':
$ref: '#/components/responses/Unauthorized'
'403':
$ref: '#/components/responses/Forbidden'
'404':
description: Enterprise feature not available in this edition
/api/live/metrics:
get:
operationId: liveMetrics
summary: Stream live metrics events (Enterprise)
description: |
Opens a `text/event-stream` connection streaming server metrics in real
time. Requires `LiveMetrics` permission; may be authenticated via the
`?token=` query parameter from `/api/token/metrics`. Enterprise only.
tags: [Live Telemetry]
security:
- bearerAuth: []
- basicAuth: []
- liveToken: []
responses:
'200':
description: Server-Sent Events stream of metric events
content:
text/event-stream:
schema:
type: string
'401':
$ref: '#/components/responses/Unauthorized'
'403':
$ref: '#/components/responses/Forbidden'
'404':
description: Enterprise feature not available in this edition
components:
securitySchemes:
bearerAuth:
type: http
scheme: bearer
bearerFormat: JWT
description: OAuth2 access token issued by `POST /auth/token`.
basicAuth:
type: http
scheme: basic
description: HTTP Basic authentication using account name and secret.
liveToken:
type: apiKey
in: query
name: token
description: |
Short-lived token (60s lifetime) issued by `/api/token/{kind}` and used
to authorise Server-Sent Events streams where an `Authorization` header
cannot be set (e.g. browser `EventSource`).
responses:
BadRequest:
description: Request payload is malformed or fails validation
content:
application/problem+json:
schema:
$ref: '#/components/schemas/ProblemDetails'
Unauthorized:
description: Missing or invalid credentials
headers:
WWW-Authenticate:
schema:
type: string
example: Bearer realm="Stalwart Server"
content:
application/problem+json:
schema:
$ref: '#/components/schemas/ProblemDetails'
Forbidden:
description: Authenticated principal lacks the required permission
content:
application/problem+json:
schema:
$ref: '#/components/schemas/ProblemDetails'
NotFound:
description: Resource not found
content:
application/problem+json:
schema:
$ref: '#/components/schemas/ProblemDetails'
TooManyRequests:
description: Anonymous-request rate limit exceeded
content:
application/problem+json:
schema:
$ref: '#/components/schemas/ProblemDetails'
schemas:
LoginRequest:
type: object
description: |
Tagged union discriminated by `type`. Use `authCode` for the standard
OAuth authorization-code flow (optionally with PKCE) and `authDevice`
to complete an OAuth device-authorization flow.
oneOf:
- $ref: '#/components/schemas/LoginRequestAuthCode'
- $ref: '#/components/schemas/LoginRequestAuthDevice'
discriminator:
propertyName: type
mapping:
authCode: '#/components/schemas/LoginRequestAuthCode'
authDevice: '#/components/schemas/LoginRequestAuthDevice'
LoginRequestAuthCode:
type: object
required: [type, accountName, accountSecret, clientId]
properties:
type:
type: string
enum: [authCode]
accountName:
type: string
accountSecret:
type: string
format: password
mfaToken:
type: string
nullable: true
description: MFA token returned by a previous `mfaRequired` response
clientId:
type: string
description: OAuth client identifier
redirectUri:
type: string
format: uri
nullable: true
description: Must use `https://` unless the server is in recovery or dev mode
nonce:
type: string
nullable: true
scope:
type: string
nullable: true
codeChallenge:
type: string
nullable: true
description: PKCE code challenge (RFC 7636)
codeChallengeMethod:
type: string
enum: [plain, S256]
nullable: true
description: Defaults to `plain` when a `codeChallenge` is present
state:
type: string
nullable: true
LoginRequestAuthDevice:
type: object
required: [type, accountName, accountSecret, code]
properties:
type:
type: string
enum: [authDevice]
accountName:
type: string
accountSecret:
type: string
format: password
mfaToken:
type: string
nullable: true
code:
type: string
description: User-facing device code issued by `POST /auth/device`
LoginResponse:
type: object
description: Tagged union discriminated by `type`.
oneOf:
- $ref: '#/components/schemas/LoginResponseAuthenticated'
- $ref: '#/components/schemas/LoginResponseVerified'
- $ref: '#/components/schemas/LoginResponseMfaRequired'
- $ref: '#/components/schemas/LoginResponseFailure'
discriminator:
propertyName: type
mapping:
authenticated: '#/components/schemas/LoginResponseAuthenticated'
verified: '#/components/schemas/LoginResponseVerified'
mfaRequired: '#/components/schemas/LoginResponseMfaRequired'
failure: '#/components/schemas/LoginResponseFailure'
LoginResponseAuthenticated:
type: object
required: [type, clientCode]
properties:
type:
type: string
enum: [authenticated]
clientCode:
type: string
description: Authorization code to exchange at `POST /auth/token`
LoginResponseVerified:
type: object
required: [type]
properties:
type:
type: string
enum: [verified]
LoginResponseMfaRequired:
type: object
required: [type]
properties:
type:
type: string
enum: [mfaRequired]
LoginResponseFailure:
type: object
required: [type]
properties:
type:
type: string
enum: [failure]
Account:
type: object
required: [permissions, edition, locale]
properties:
permissions:
type: array
description: |
Effective permissions for the authenticated principal, filtered to
exclude internal/system-only permissions. Values are from the
`Permission` enum (e.g. `authenticate`, `jmap-email-get`,
`sys-account-settings-get`).
items:
type: string
edition:
type: string
enum: [oss, community, enterprise]
description: Server edition
locale:
type: string
description: Preferred locale for the account (IETF BCP 47-style tag)
ProblemDetails:
type: object
description: RFC 7807 problem details document
properties:
type:
type: string
format: uri
title:
type: string
status:
type: integer
detail:
type: string
instance:
type: string
crates/common/Cargo.toml
+101
View File
@@ -0,0 +1,101 @@
[package]
name = "common"
version = "0.16.22"
edition = "2024"
build = "build.rs"
[dependencies]
utils = { path = "../utils" }
nlp = { path = "../nlp" }
store = { path = "../store" }
trc = { path = "../trc" }
directory = { path = "../directory" }
coordinator = { path = "../coordinator" }
types = { path = "../types" }
registry = { path = "../registry" }
jmap_proto = { path = "../jmap-proto" }
sieve-rs = { version = "0.7", features = ["rkyv", "serde"] }
mail-parser = { version = "0.11", features = ["full_encoding"] }
mail-builder = { version = "1.0" }
mail-auth = { version = "0.13", features = ["generate", "arc"] }
smtp-proto = { version = "0.2.3", features = ["rkyv"] }
dns-update = { version = "0.5" }
calcard = { version = "0.3", features = ["rkyv"] }
ahash = { version = "0.8.12", features = ["serde"] }
parking_lot = "0.12.5"
regex = "1.13.1"
proxy-header = { version = "0.1.2", features = ["tokio"] }
arc-swap = "1.9.2"
rustls = { version = "0.23.43", default-features = false, features = ["std", "aws_lc_rs", "tls12"] }
rustls-pemfile = "2.2"
rustls-pki-types = { version = "1" }
aws-lc-rs = { version = "1" }
tokio = { version = "1.53", features = ["net", "macros"] }
tokio-rustls = { version = "0.26", default-features = false, features = ["aws_lc_rs", "tls12"] }
futures = "0.3"
rcgen = { version = "0.14", default-features = false, features = ["crypto", "pem", "aws_lc_rs"] }
reqwest = { version = "0.13", default-features = false, features = ["rustls", "http2", "stream"]}
serde = { version = "1.0", features = ["derive"]}
serde_json = "1.0"
base64 = "0.23"
x509-parser = "0.18"
pem = "4.0"
chrono = { version = "0.4", features = ["serde"] }
hyper = { version = "1.11.1", features = ["server", "http1", "http2"] }
opentelemetry = { git = "https://github.com/stalwartlabs/opentelemetry-rust" }
opentelemetry_sdk = { git = "https://github.com/stalwartlabs/opentelemetry-rust" }
opentelemetry-otlp = { git = "https://github.com/stalwartlabs/opentelemetry-rust", default-features = false, features = ["reqwest-client", "http-proto", "trace", "metrics", "logs", "internal-logs", "grpc-tonic", "tls-aws-lc", "tls-roots", "reqwest-rustls"] }
opentelemetry-semantic-conventions = { git = "https://github.com/stalwartlabs/opentelemetry-rust" }
prometheus = { version = "0.14", default-features = false }
imagesize = "0.15"
sha1 = "0.11"
sha2 = "0.11"
md5 = "0.8.1"
whatlang = "0.18"
idna = "1.1"
decancer = "3.3.3"
unicode-security = "0.1.2"
infer = "0.22"
bincode = { version = "2.0.1", features = ["serde"] }
hostname = "0.4.2"
zip = "8.6"
xxhash-rust = { version = "0.8.18", features = ["xxh3"] }
psl = "2"
aes-gcm-siv = "0.12.1"
jsonwebtoken = { version = "11.0.0", features = ["aws_lc_rs"] }
rsa = "0.9.10"
p256 = { version = "0.13", features = ["ecdh", "ecdsa", "pkcs8", "pem"] }
p384 = { version = "0.13", features = ["ecdh"] }
hashify = "0.2"
rkyv = { version = "0.8.18", features = ["little_endian"] }
tinyvec = { version = "1.12.0", features = ["alloc"] }
compact_str = { version = "0.10.0", features = ["rkyv", "serde"] }
lz4_flex = { version = "0.14", features = ["frame"], default-features = false }
hickory-proto = "0.26.3"
nohash-hasher = "0.2.0"
quick_cache = "0.7"
rasn = "0.28"
rasn-pkix = "0.28"
sequoia-openpgp = { version = "2.4", default-features = false, features = ["crypto-rust", "allow-experimental-crypto", "allow-variable-time-crypto"] }
zxcvbn = "3.1.1"
pkcs8 = { version = "0.10.2", features = ["alloc", "std"] }
quick-xml = "0.41"
[target.'cfg(unix)'.dependencies]
privdrop = "0.5.6"
libc = "0.2.189"
[target.'cfg(windows)'.dependencies]
socket2 = "0.6"
[features]
test_mode = []
dev_mode = []
enterprise = []
foundation = []
[dev-dependencies]
tokio = { version = "1.53", features = ["full"] }
[lints]
workspace = true
crates/common/build.rs
+234
View File
@@ -0,0 +1,234 @@
use std::collections::{HashMap, HashSet};
use std::env;
use std::fs;
use std::path::Path;
fn main() {
let out_dir = env::var("OUT_DIR").unwrap();
let dest_path = Path::new(&out_dir).join("locales.rs");
// Read the YAML file
let manifest_dir = env::var("CARGO_MANIFEST_DIR").unwrap();
let repo_root = Path::new(&manifest_dir).parent().unwrap().parent().unwrap();
let yaml_path = repo_root.join("resources/locales/i18n.yml");
let yaml_content =
fs::read_to_string(&yaml_path).unwrap_or_else(|_| panic!("Failed to read {yaml_path:?}"));
let locales = parse_yaml(&yaml_content);
let generated_code = generate_locale_code(&locales);
fs::write(&dest_path, generated_code).expect("Failed to write generated locales.");
println!("cargo:rerun-if-changed={}", yaml_path.display());
}
fn parse_yaml(content: &str) -> HashMap<String, HashMap<String, String>> {
let mut result: HashMap<String, HashMap<String, String>> = HashMap::new();
let mut current_key = None;
for line in content.lines() {
if let Some((key, value)) = line.split_once(':') {
let is_translation = key
.as_bytes()
.first()
.is_some_and(|&b| b.is_ascii_whitespace());
let key = key.trim();
if !key.starts_with('#') && !key.is_empty() {
if !is_translation {
current_key = result.entry(key.replace('.', "_")).or_default().into();
} else {
current_key
.as_mut()
.unwrap()
.insert(key.to_string(), value.trim().trim_matches('"').to_string());
}
}
}
}
result
}
fn const_name(language: &str) -> String {
language.to_uppercase().replace('-', "_")
}
const PLURAL_CATEGORIES: [&str; 6] = ["zero", "one", "two", "few", "many", "other"];
const RTL_LANGUAGES: [&str; 10] = ["ar", "ckb", "dv", "fa", "he", "ps", "sd", "ug", "ur", "yi"];
fn direction(language: &str) -> &'static str {
let tag = language.split(['-', '_']).next().unwrap_or(language);
if RTL_LANGUAGES.contains(&tag) {
"rtl"
} else {
"ltr"
}
}
fn split_plural_forms(value: &str) -> Option<Vec<(&str, &str)>> {
value
.split(';')
.map(|segment| {
segment
.split_once('=')
.filter(|(name, _)| PLURAL_CATEGORIES.contains(name))
})
.collect()
}
fn plural_keys(locales: &HashMap<String, HashMap<String, String>>) -> HashSet<String> {
let mut keys = HashSet::new();
for (key, translations) in locales {
if !translations
.values()
.any(|value| split_plural_forms(value).is_some())
{
continue;
}
for (language, value) in translations {
let Some(forms) = split_plural_forms(value) else {
panic!(
"{key}: {language} has no plural categories while other languages do: {value:?}"
);
};
let mut seen = HashSet::new();
for (name, _) in &forms {
if !seen.insert(*name) {
panic!("{key}: {language} repeats the plural category {name:?}");
}
}
if !seen.contains("other") {
panic!("{key}: {language} is missing the required \"other\" plural category");
}
}
keys.insert(key.clone());
}
keys
}
fn plural_forms_literal(value: &str) -> String {
let forms = split_plural_forms(value).expect("validated above");
let other = forms
.iter()
.find(|(name, _)| *name == "other")
.map(|(_, text)| *text)
.expect("validated above");
let mut literal = String::from("PluralForms {");
for category in PLURAL_CATEGORIES {
let text = forms
.iter()
.find(|(name, _)| *name == category)
.map_or(other, |(_, text)| *text);
literal.push_str(&format!(" {category}: {text:?},"));
}
literal.push_str(" }");
literal
}
fn generate_locale_code(locales: &HashMap<String, HashMap<String, String>>) -> String {
let mut code = String::new();
let plural = plural_keys(locales);
code.push_str("#[derive(Debug, Clone, Copy)]\n");
code.push_str("pub struct PluralForms {\n");
for category in PLURAL_CATEGORIES {
code.push_str(&format!(" pub {category}: &'static str,\n"));
}
code.push_str("}\n\n");
code.push_str("#[derive(Debug, Clone)]\n");
code.push_str("pub struct Locale {\n");
code.push_str(" pub name: &'static str,\n");
code.push_str(" pub direction: &'static str,\n");
for key in locales.keys() {
let field_type = if plural.contains(key) {
"PluralForms"
} else {
"&'static str"
};
code.push_str(&format!(" pub {key}: {field_type},\n"));
}
code.push_str("}\n\n");
let mut languages = std::collections::HashSet::new();
for translations in locales.values() {
for lang in translations.keys() {
languages.insert(lang.clone());
}
}
for lang in &languages {
code.push_str(&format!(
"pub static {}_LOCALES: Locale = Locale {{\n",
const_name(lang)
));
code.push_str(&format!(" name: {lang:?},\n"));
code.push_str(&format!(" direction: {:?},\n", direction(lang)));
for (key, translations) in locales {
let value = translations
.get(lang)
.unwrap_or_else(|| panic!("Missing: {}", key));
if plural.contains(key) {
code.push_str(&format!(" {key}: {},\n", plural_forms_literal(value)));
} else {
code.push_str(&format!(" {key}: {value:?},\n"));
}
}
code.push_str("};\n\n");
}
let mut sorted: Vec<&String> = languages.iter().collect();
sorted.sort_unstable();
code.push_str(&format!(
"pub static ALL_LOCALES: [&Locale; {}] = [\n",
sorted.len()
));
for lang in &sorted {
code.push_str(&format!(" &{}_LOCALES,\n", const_name(lang)));
}
code.push_str("];\n\n");
code.push_str("pub fn locale(name: &str) -> Option<&'static Locale> {\n");
code.push_str(" hashify::tiny_map_ignore_case!(name.as_bytes(),\n");
for lang in &languages {
code.push_str(&format!(
" \"{}\" => &{}_LOCALES,\n",
lang,
const_name(lang)
));
}
code.push_str(" )\n");
code.push_str("}\n\n");
// Maps a bare language tag onto the regional locale shipped for it
let mut by_language: Vec<(&str, &str)> = languages
.iter()
.map(|lang| (lang.split('-').next().unwrap_or(lang), lang.as_str()))
.collect();
by_language.sort_unstable();
by_language.dedup_by_key(|(language, _)| *language);
code.push_str("pub fn locale_by_language(language: &str) -> Option<&'static Locale> {\n");
code.push_str(" hashify::tiny_map_ignore_case!(language.as_bytes(),\n");
for (language, lang) in by_language {
code.push_str(&format!(
" \"{}\" => &{}_LOCALES,\n",
language,
const_name(lang)
));
}
code.push_str(" )\n");
code.push_str("}\n");
code
}
crates/common/src/auth/access_token.rs
+940
View File
@@ -0,0 +1,940 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use super::AccessToken;
use crate::{
Server,
auth::{
AccessScope, AccessTo, AccessTokenInner, AccountTenantIds, Permissions, RECOVERY_ADMIN_ID,
permissions::{BuildPermissions, PermissionsListBuilder},
},
network::limiter::{ConcurrencyLimiter, LimiterResult},
};
use ahash::AHasher;
use registry::{
schema::{
enums::Permission,
structs::{self, Account, Roles, UserRoles},
},
types::EnumImpl,
};
use std::{
hash::{Hash, Hasher},
net::IpAddr,
sync::Arc,
};
use store::{query::acl::AclQuery, rand, write::now};
use tinyvec::TinyVec;
use trc::{AddContext, StoreEvent};
use types::{acl::Acl, collection::Collection};
use utils::map::bitmap::{Bitmap, BitmapItem};
use xxhash_rust::xxh3;
impl Server {
async fn build_access_token(
&self,
account: Account,
account_id: u32,
revision: u64,
revision_account: u64,
) -> trc::Result<AccessTokenInner> {
match account {
Account::User(account) => {
let tenant_id = account.member_tenant_id.map(|t| t.id() as u32);
let permissions = self
.effective_permissions(
&account.permissions,
match &account.roles {
UserRoles::User => {
self.core.network.security.default_role_ids_user.as_slice()
}
UserRoles::Admin => {
if tenant_id.is_none() {
self.core.network.security.default_role_ids_admin.as_slice()
} else {
self.core
.network
.security
.default_role_ids_tenant
.as_slice()
}
}
UserRoles::Custom(custom_roles) => custom_roles.role_ids.as_slice(),
},
tenant_id,
)
.await?;
let member_of = account
.member_group_ids
.iter()
.map(|m| m.id() as u32)
.collect::<TinyVec<[u32; 3]>>();
let mut access_to: Vec<AccessTo> = Vec::new();
for grant_account_id in [account_id].into_iter().chain(member_of.iter().copied()) {
for acl_item in self
.store()
.acl_query(AclQuery::HasAccess { grant_account_id })
.await
.caused_by(trc::location!())?
{
if acl_item.to_account_id != account_id
&& !member_of.contains(&acl_item.to_account_id)
{
let acl = Bitmap::<Acl>::from(acl_item.permissions);
let collection = acl_item.to_collection;
if !collection.is_valid() {
return Err(trc::StoreEvent::DataCorruption
.ctx(trc::Key::Reason, "Corrupted collection found in ACL key.")
.details(format!("{acl_item:?}"))
.account_id(grant_account_id)
.caused_by(trc::location!()));
}
let mut collections: Bitmap<Collection> = Bitmap::new();
if acl.contains(Acl::Read) {
collections.insert(collection);
}
if acl.contains(Acl::ReadItems)
&& let Some(child_col) = collection.child_collection()
{
collections.insert(child_col);
}
if !collections.is_empty() {
if let Some(idx) = access_to
.iter()
.position(|a| a.account_id == acl_item.to_account_id)
{
access_to[idx].collections.union(&collections);
} else {
access_to.push(AccessTo {
account_id: acl_item.to_account_id,
collections,
});
}
}
}
}
}
let now = now();
let mut credential_version = 0;
let mut credential_scopes = Vec::with_capacity(account.credentials.len());
credential_scopes.push(AccessScope::new(permissions.finalize(), u32::MAX));
for credential in account.credentials {
match credential {
structs::Credential::Password(credential) => {
credential_version = xxh3::xxh3_64(credential.secret.as_bytes()).max(1);
if credential.expires_at.is_some() || !credential.allowed_ips.is_empty()
{
let credential_scope = &mut credential_scopes[0];
credential_scope.expires_at = credential
.expires_at
.map(|v| v.timestamp() as u64)
.unwrap_or(u64::MAX);
credential_scope.allowed_ips =
credential.allowed_ips.into_inner().into_boxed_slice();
}
}
structs::Credential::ApiKey(credential)
| structs::Credential::AppPassword(credential) => {
let credential_id = credential.credential_id.document_id();
let expires_at = credential
.expires_at
.map(|v| v.timestamp() as u64)
.unwrap_or(u64::MAX);
if expires_at > now {
let permissions = &credential_scopes[0].permissions;
let permissions = match credential.permissions {
structs::CredentialPermissions::Inherit => permissions.clone(),
structs::CredentialPermissions::Disable(list) => {
let mut permissions = permissions.clone();
permissions.clear_many(&Permissions::from_permission(
list.permissions.as_slice(),
));
permissions
}
structs::CredentialPermissions::Replace(list) => {
let mut replace_permissions = Permissions::from_permission(
list.permissions.as_slice(),
);
replace_permissions.intersection(permissions);
replace_permissions
}
};
credential_scopes.push(AccessScope {
credential_id,
permissions,
expires_at,
allowed_ips: credential
.allowed_ips
.into_inner()
.into_boxed_slice(),
})
}
}
}
}
Ok(AccessTokenInner {
concurrent_imap_requests: self
.core
.imap
.rate_concurrent
.map(ConcurrencyLimiter::new),
concurrent_http_requests: self
.core
.jmap
.request_max_concurrent
.map(ConcurrencyLimiter::new),
concurrent_uploads: self
.core
.jmap
.upload_max_concurrent
.map(ConcurrencyLimiter::new),
obj_size: 0,
revision,
revision_account,
credential_version,
account_id,
tenant_id,
member_of,
access_to: access_to.into_boxed_slice(),
scopes: []
.into_iter()
.chain(credential_scopes)
.collect::<Box<[AccessScope]>>(),
}
.update_size())
}
Account::Group(account) => {
let tenant_id = account.member_tenant_id.map(|t| t.id() as u32);
let permissions = self
.effective_permissions(
&account.permissions,
account.roles.role_ids().unwrap_or(
self.core.network.security.default_role_ids_group.as_slice(),
),
tenant_id,
)
.await?;
Ok(AccessTokenInner {
concurrent_imap_requests: self
.core
.imap
.rate_concurrent
.map(ConcurrencyLimiter::new),
concurrent_http_requests: self
.core
.jmap
.request_max_concurrent
.map(ConcurrencyLimiter::new),
concurrent_uploads: self
.core
.jmap
.upload_max_concurrent
.map(ConcurrencyLimiter::new),
obj_size: 0,
revision,
revision_account,
credential_version: 0,
account_id,
tenant_id,
member_of: Default::default(),
access_to: Default::default(),
scopes: Box::new([AccessScope::new(permissions.finalize(), u32::MAX)]),
}
.update_size())
}
}
}
pub async fn access_token(&self, account_id: u32) -> trc::Result<Arc<AccessTokenInner>> {
match self
.inner
.cache
.access_tokens
.get_value_or_guard_async(&account_id)
.await
{
Ok(token) => {
trc::event!(
Store(StoreEvent::CacheHit),
Key = account_id,
Collection = "accessToken",
);
Ok(token)
}
Err(guard) => {
trc::event!(
Store(StoreEvent::CacheMiss),
Key = account_id,
Collection = "accessToken",
);
let token: Arc<AccessTokenInner> = if let Some(account) =
self.registry().object::<Account>(account_id.into()).await?
{
let revision = rand::random::<u64>();
let revision_account = hash_account(&account);
self.build_access_token(account, account_id, revision, revision_account)
.await?
.into()
} else if account_id == RECOVERY_ADMIN_ID {
AccessTokenInner::new_admin().into()
} else {
return Err(trc::SecurityEvent::Unauthorized
.into_err()
.details("Account not found")
.account_id(account_id)
.caused_by(trc::location!()));
};
let _ = guard.insert(token.clone());
Ok(token)
}
}
}
pub(crate) async fn access_token_from_account(
&self,
account_id: u32,
account: Account,
) -> trc::Result<Arc<AccessTokenInner>> {
let revision_account = hash_account(&account);
match self
.inner
.cache
.access_tokens
.get_value_or_guard_async(&account_id)
.await
{
Ok(token) => {
if token.revision_account == revision_account {
trc::event!(
Store(StoreEvent::CacheHit),
Key = account_id,
Collection = "accessToken",
);
Ok(token)
} else {
// Token is stale, rebuild it
trc::event!(
Store(StoreEvent::CacheStale),
Key = account_id,
Collection = "accessToken",
);
debug_assert!(
false,
"Token is stale, invalidation should have been triggered"
);
let revision = rand::random::<u64>();
let token: Arc<AccessTokenInner> = self
.build_access_token(account, account_id, revision, revision_account)
.await?
.into();
self.inner
.cache
.access_tokens
.update(account_id, token.clone());
Ok(token)
}
}
Err(guard) => {
trc::event!(
Store(StoreEvent::CacheMiss),
Key = account_id,
Collection = "accessToken",
);
let revision = rand::random::<u64>();
let token: Arc<AccessTokenInner> = self
.build_access_token(account, account_id, revision, revision_account)
.await?
.into();
let _ = guard.insert(token.clone());
Ok(token)
}
}
}
}
impl AccessToken {
pub fn new(inner: Arc<AccessTokenInner>, remote_ip: IpAddr) -> trc::Result<Self> {
AccessToken {
scope_idx: 0,
inner,
}
.assert_is_valid(remote_ip)
}
pub fn new_maybe_invalid(inner: Arc<AccessTokenInner>) -> Self {
AccessToken {
scope_idx: 0,
inner,
}
}
pub fn new_scoped(
inner: Arc<AccessTokenInner>,
credential_id: u32,
remote_ip: IpAddr,
) -> trc::Result<Self> {
inner
.scopes
.iter()
.position(|scope| scope.credential_id == credential_id)
.ok_or_else(|| {
trc::SecurityEvent::Unauthorized
.into_err()
.ctx(trc::Key::AccountId, inner.account_id)
.ctx(trc::Key::Id, credential_id)
.reason("Credential expired or removed.")
})
.map(|scope_idx| AccessToken { scope_idx, inner })
.and_then(|token| token.assert_is_valid(remote_ip))
}
pub fn renew(
inner: Arc<AccessTokenInner>,
credential_id: Option<u32>,
remote_ip: IpAddr,
) -> trc::Result<Self> {
if let Some(credential_id) = credential_id {
Self::new_scoped(inner, credential_id, remote_ip)
} else {
AccessToken {
scope_idx: 0,
inner,
}
.assert_is_valid(remote_ip)
}
}
pub fn state(&self) -> u32 {
// Hash state
let mut s = AHasher::default();
self.inner.member_of.hash(&mut s);
self.inner.access_to.hash(&mut s);
s.finish() as u32
}
#[inline(always)]
pub fn account_id(&self) -> u32 {
self.inner.account_id
}
#[inline(always)]
pub fn tenant_id(&self) -> Option<u32> {
self.inner.tenant_id
}
pub fn secondary_ids(&self) -> impl Iterator<Item = &u32> {
self.inner
.member_of
.iter()
.chain(self.inner.access_to.iter().map(|a| &a.account_id))
}
pub fn member_ids(&self) -> impl Iterator<Item = u32> {
[self.inner.account_id]
.into_iter()
.chain(self.inner.member_of.iter().copied())
}
pub fn all_ids(&self) -> impl Iterator<Item = u32> {
[self.inner.account_id]
.into_iter()
.chain(self.inner.member_of.iter().copied())
.chain(self.inner.access_to.iter().map(|a| a.account_id))
}
pub fn all_ids_by_collection(&self, collection: Collection) -> impl Iterator<Item = u32> {
[self.inner.account_id]
.into_iter()
.chain(self.inner.member_of.iter().copied())
.chain(self.inner.access_to.iter().filter_map(move |a| {
if a.collections.contains(collection) {
Some(a.account_id)
} else {
None
}
}))
}
pub fn is_member(&self, account_id: u32) -> bool {
self.inner.account_id == account_id
|| self.inner.member_of.contains(&account_id)
|| self.has_permission(Permission::Impersonate)
}
pub fn is_account_id(&self, account_id: u32) -> bool {
self.inner.account_id == account_id
}
pub fn personal_id(&self, account_id: u32, collection: Collection) -> u32 {
let child_collection = collection.child_collection();
if self.is_account_id(account_id)
|| self.inner.member_of.contains(&account_id)
|| self.inner.access_to.iter().any(|a| {
a.account_id == account_id
&& (a.collections.contains(collection)
|| child_collection.is_some_and(|child| a.collections.contains(child)))
})
{
self.inner.account_id
} else {
account_id
}
}
#[inline(always)]
pub fn has_permission(&self, permission: Permission) -> bool {
self.inner
.scopes
.get(self.scope_idx)
.is_some_and(|scope| scope.permissions.get(permission as usize))
}
pub fn assert_is_valid(self, remote_ip: IpAddr) -> trc::Result<Self> {
if let Some(scope) = self.inner.scopes.get(self.scope_idx) {
let has_expired = scope.expires_at <= now();
let is_valid_ip = scope.allowed_ips.is_empty()
|| scope
.allowed_ips
.iter()
.any(|ip_mask| ip_mask.matches(&remote_ip));
let mut access_token = self;
if has_expired {
if access_token.scope_idx > 0 {
return Err(trc::AuthEvent::CredentialExpired
.into_err()
.ctx(trc::Key::AccountId, access_token.inner.account_id)
.reason("Credential expired."));
} else {
trc::event!(
Auth(trc::AuthEvent::CredentialExpired),
AccountId = access_token.inner.account_id,
Reason = "Main credential expired, downgrading permissions.",
);
}
// Downgrade permissions to allow password change
let mut scopes = Vec::with_capacity(access_token.inner.scopes.len());
for (idx, scope) in access_token.inner.scopes.iter().enumerate() {
if idx == 0 {
let mut permissions = Permissions::new();
for permission in [
Permission::Authenticate,
Permission::AuthenticateWithAlias,
Permission::SysAccountPasswordGet,
Permission::SysAccountPasswordUpdate,
Permission::EmailReceive,
] {
if scope.permissions.get(permission as usize) {
permissions.set(permission as usize);
}
}
scopes.push(AccessScope {
permissions,
credential_id: scope.credential_id,
expires_at: u64::MAX,
allowed_ips: scope.allowed_ips.clone(),
});
} else {
scopes.push(scope.clone());
}
}
let old_inner = &access_token.inner;
let inner = AccessTokenInner {
scopes: scopes.into_boxed_slice(),
account_id: old_inner.account_id,
tenant_id: old_inner.tenant_id,
member_of: old_inner.member_of.clone(),
access_to: old_inner.access_to.clone(),
concurrent_http_requests: old_inner.concurrent_http_requests.clone(),
concurrent_imap_requests: old_inner.concurrent_imap_requests.clone(),
concurrent_uploads: old_inner.concurrent_uploads.clone(),
revision_account: old_inner.revision_account,
revision: old_inner.revision,
credential_version: old_inner.credential_version,
obj_size: old_inner.obj_size,
};
access_token = AccessToken {
scope_idx: access_token.scope_idx,
inner: Arc::new(inner),
};
}
if is_valid_ip {
Ok(access_token)
} else {
Err(trc::SecurityEvent::IpUnauthorized
.into_err()
.ctx(trc::Key::AccountId, access_token.inner.account_id)
.reason("IP address not allowed."))
}
} else {
Err(trc::SecurityEvent::Unauthorized
.into_err()
.ctx(trc::Key::AccountId, self.inner.account_id)
.reason("Credential not valid."))
}
}
#[inline(always)]
pub fn credential_id(&self) -> Option<u32> {
self.inner
.scopes
.get(self.scope_idx)
.map(|scope| scope.credential_id)
}
#[inline(always)]
pub fn revision(&self) -> u64 {
self.inner.revision
}
pub fn assert_has_permissions(self, permissions: &[Permission]) -> trc::Result<Self> {
for permission in permissions {
if !self.has_permission(*permission) {
return Err(trc::SecurityEvent::Unauthorized
.into_err()
.details(permission.as_str())
.account_id(self.account_id()));
}
}
Ok(self)
}
pub fn assert_has_permission(self, permission: Permission) -> trc::Result<Self> {
if self.has_permission(permission) {
Ok(self)
} else {
Err(trc::SecurityEvent::Unauthorized
.into_err()
.details(permission.as_str())
.account_id(self.account_id()))
}
}
pub fn enforce_permission(&self, permission: Permission) -> trc::Result<()> {
if self.has_permission(permission) {
Ok(())
} else {
Err(trc::SecurityEvent::Unauthorized
.into_err()
.details(permission.as_str())
.account_id(self.account_id()))
}
}
pub fn permissions(&self) -> Vec<Permission> {
if let Some(scope) = self.inner.scopes.get(self.scope_idx) {
scope.permissions.build_permissions_list()
} else {
vec![]
}
}
#[inline(always)]
pub fn access_scope(&self) -> Option<&AccessScope> {
self.inner.scopes.get(self.scope_idx)
}
pub(crate) fn permissions_bits(&self) -> &Permissions {
&self
.inner
.scopes
.get(self.scope_idx)
.unwrap_or(&self.inner.scopes[0])
.permissions
}
pub fn account_permissions(&self) -> &Permissions {
&self.inner.scopes[0].permissions
}
pub fn is_shared(&self, account_id: u32) -> bool {
!self.is_member(account_id)
&& self
.inner
.access_to
.iter()
.any(|a| a.account_id == account_id)
}
pub fn shared_accounts(&self, collection: Collection) -> impl Iterator<Item = &u32> {
self.inner
.member_of
.iter()
.chain(self.inner.access_to.iter().filter_map(move |a| {
if a.collections.contains(collection) {
Some(&a.account_id)
} else {
None
}
}))
}
pub fn has_access(&self, to_account_id: u32, to_collection: impl Into<Collection>) -> bool {
let to_collection = to_collection.into();
self.is_member(to_account_id)
|| self
.inner
.access_to
.iter()
.any(|a| a.account_id == to_account_id && a.collections.contains(to_collection))
}
pub fn has_account_access(&self, to_account_id: u32) -> bool {
self.is_member(to_account_id)
|| self
.inner
.access_to
.iter()
.any(|a| a.account_id == to_account_id)
}
pub fn is_http_request_allowed(&self) -> LimiterResult {
self.inner
.concurrent_http_requests
.as_ref()
.map_or(LimiterResult::Disabled, |limiter| limiter.is_allowed())
}
pub fn concurrent_http_requests(&self) -> u64 {
self.inner
.concurrent_http_requests
.as_ref()
.map(|limiter| limiter.max_concurrent())
.unwrap_or(0)
}
pub fn is_imap_request_allowed(&self) -> LimiterResult {
self.inner
.concurrent_imap_requests
.as_ref()
.map_or(LimiterResult::Disabled, |limiter| limiter.is_allowed())
}
pub fn is_upload_allowed(&self) -> LimiterResult {
self.inner
.concurrent_uploads
.as_ref()
.map_or(LimiterResult::Disabled, |limiter| limiter.is_allowed())
}
pub fn concurrent_uploads(&self) -> u64 {
self.inner
.concurrent_uploads
.as_ref()
.map(|limiter| limiter.max_concurrent())
.unwrap_or(0)
}
pub fn account_tenant_ids(&self) -> AccountTenantIds {
AccountTenantIds {
account_id: self.account_id(),
tenant_id: self.tenant_id(),
}
}
pub fn new_admin() -> AccessToken {
AccessToken {
scope_idx: 0,
inner: Arc::new(AccessTokenInner::new_admin()),
}
}
pub fn from_permissions(
account_id: u32,
set_permissions: impl IntoIterator<Item = Permission>,
) -> AccessToken {
let mut permissions = Permissions::new();
for permission in set_permissions {
permissions.set(permission as usize);
}
AccessToken {
scope_idx: 0,
inner: Arc::new(AccessTokenInner {
account_id,
tenant_id: Default::default(),
member_of: Default::default(),
access_to: Default::default(),
scopes: Box::new([AccessScope::new(permissions, u32::MAX)]),
concurrent_http_requests: Default::default(),
concurrent_imap_requests: Default::default(),
concurrent_uploads: Default::default(),
revision: Default::default(),
revision_account: Default::default(),
credential_version: Default::default(),
obj_size: Default::default(),
}),
}
}
pub fn from_id_maybe_invalid(account_id: u32) -> Self {
AccessToken::new_maybe_invalid(Arc::new(AccessTokenInner::from_id(account_id)))
}
}
impl AccessTokenInner {
pub fn from_id(account_id: u32) -> Self {
Self {
account_id,
..Default::default()
}
}
pub fn with_tenant_id(mut self, tenant_id: Option<u32>) -> Self {
self.tenant_id = tenant_id;
self
}
pub fn update_size(mut self) -> Self {
self.obj_size = (std::mem::size_of::<AccessToken>()
+ (self.member_of.len() * std::mem::size_of::<u32>())
+ (self.access_to.len() * (std::mem::size_of::<u32>() + std::mem::size_of::<u64>()))
+ (self.scopes.len() * std::mem::size_of::<AccessScope>()))
as u64;
self
}
pub fn new_admin() -> Self {
AccessTokenInner {
account_id: RECOVERY_ADMIN_ID,
tenant_id: Default::default(),
member_of: Default::default(),
access_to: Default::default(),
scopes: Box::new([AccessScope::new(Permissions::all(), u32::MAX)]),
concurrent_http_requests: Default::default(),
concurrent_imap_requests: Default::default(),
concurrent_uploads: Default::default(),
revision: Default::default(),
revision_account: Default::default(),
credential_version: Default::default(),
obj_size: Default::default(),
}
}
pub fn revision(&self) -> u64 {
self.revision
}
pub fn revision_account(&self) -> u64 {
self.revision_account
}
pub fn credential_version(&self) -> u64 {
self.credential_version
}
}
impl AccessScope {
pub fn new(permissions: Permissions, credential_id: u32) -> Self {
Self {
permissions,
credential_id,
expires_at: u64::MAX,
allowed_ips: Default::default(),
}
}
}
fn hash_account(account: &Account) -> u64 {
let mut s = AHasher::default();
match account {
Account::User(account) => {
account.member_tenant_id.hash(&mut s);
match &account.roles {
UserRoles::User => {
0u8.hash(&mut s);
}
UserRoles::Admin => {
1u8.hash(&mut s);
}
UserRoles::Custom(custom_roles) => {
2u8.hash(&mut s);
custom_roles.role_ids.as_slice().hash(&mut s);
}
}
hash_permissions(&mut s, &account.permissions);
for credential in account
.credentials
.iter()
.filter_map(|credential| credential.as_secondary_credential())
{
credential.credential_id.hash(&mut s);
credential.expires_at.hash(&mut s);
hash_credential_permissions(&mut s, &credential.permissions);
}
for group_id in account.member_group_ids.iter() {
group_id.hash(&mut s);
}
}
Account::Group(account) => {
account.member_tenant_id.hash(&mut s);
match &account.roles {
Roles::Default => {}
Roles::Custom(custom_roles) => {
custom_roles.role_ids.as_slice().hash(&mut s);
}
}
hash_permissions(&mut s, &account.permissions);
}
}
s.finish()
}
fn hash_permissions(hasher: &mut AHasher, permissions: &structs::Permissions) {
match permissions {
structs::Permissions::Inherit => {
0u8.hash(hasher);
}
structs::Permissions::Merge(permissions) => {
2u8.hash(hasher);
permissions.enabled_permissions.as_slice().hash(hasher);
permissions.disabled_permissions.as_slice().hash(hasher);
}
structs::Permissions::Replace(permissions) => {
3u8.hash(hasher);
permissions.enabled_permissions.as_slice().hash(hasher);
permissions.disabled_permissions.as_slice().hash(hasher);
}
}
}
fn hash_credential_permissions(hasher: &mut AHasher, permissions: &structs::CredentialPermissions) {
match permissions {
structs::CredentialPermissions::Inherit => {
0u8.hash(hasher);
}
structs::CredentialPermissions::Disable(permissions) => {
2u8.hash(hasher);
permissions.permissions.as_slice().hash(hasher);
}
structs::CredentialPermissions::Replace(permissions) => {
3u8.hash(hasher);
permissions.permissions.as_slice().hash(hasher);
}
}
}
crates/common/src/auth/authentication.rs
+644
View File
@@ -0,0 +1,644 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::{
Server,
auth::{
AccessToken, AuthRequest, DomainCache,
credential::{ApiKey, AppPassword},
oauth::GrantType,
},
};
use base64::{Engine, engine::general_purpose};
use directory::{
Credentials, Directory, Recipient,
core::secret::{SecretVerificationResult, verify_mfa_secret_hash, verify_secret_hash},
};
use registry::schema::{
enums::Permission,
structs::{self, Credential},
};
use std::{net::IpAddr, sync::Arc};
use store::write::now;
use trc::AddContext;
pub struct UsernameParts {
pub account: Username,
pub master_user: Option<Username>,
}
#[derive(PartialEq, Eq)]
pub struct Username {
pub name: String,
pub domain_start: usize,
}
impl Server {
pub async fn authenticate(&self, req: &AuthRequest) -> trc::Result<AccessToken> {
match Box::pin(self.route_auth_request(req))
.await
.and_then(|token| token.assert_has_permission(Permission::Authenticate))
{
Ok(token) => Ok(token),
Err(err) => {
// Random delay to mitigate user enumeration attacks
#[cfg(not(feature = "test_mode"))]
{
use store::rand::{self, RngExt};
let delay = rand::rng().random_range(50..500);
tokio::time::sleep(std::time::Duration::from_millis(delay)).await;
}
if matches!(
err.as_ref(),
trc::EventType::Auth(trc::AuthEvent::Failed)
| trc::EventType::Security(trc::SecurityEvent::IpUnauthorized)
) && self.has_auth_fail2ban()
&& self
.is_auth_fail2banned(req.remote_ip, req.username())
.await?
{
Err(trc::SecurityEvent::AuthenticationBan
.into_err()
.ctx(trc::Key::RemoteIp, req.remote_ip)
.ctx_opt(trc::Key::AccountName, req.username().map(|s| s.to_string())))
} else {
Err(err.ctx(trc::Key::RemoteIp, req.remote_ip))
}
}
}
}
async fn route_auth_request(&self, req: &AuthRequest) -> trc::Result<AccessToken> {
match &req.credentials {
Credentials::Basic {
username,
secret,
mfa_token,
} => {
let mut username = UsernameParts::new(username);
// Try to authenticate as fallback admin if configured
if let Some((fallback_user, fallback_hash)) = &self.registry().recovery_admin()
&& username.auth_as().address() == fallback_user
{
return if verify_secret_hash(fallback_hash, secret.as_bytes()).await? {
if username.is_master() {
let address = username.account().address();
if let Some(account_id) =
self.impersonated_account_id(username.account()).await?
{
trc::event!(
Auth(trc::AuthEvent::Success),
AccountName = address.to_string(),
AccountId = account_id,
SpanId = req.session_id,
Details = fallback_user.to_string(),
);
self.access_token(account_id)
.await
.and_then(|token| AccessToken::new(token, req.remote_ip))
} else {
Err(trc::AuthEvent::Failed
.into_err()
.ctx(trc::Key::AccountName, address.to_string())
.reason("Master user account not found for fallback admin authentication"))
}
} else {
trc::event!(
Auth(trc::AuthEvent::Success),
AccountName = fallback_user.to_string(),
SpanId = req.session_id,
);
Ok(AccessToken::new_admin())
}
} else {
Err(trc::AuthEvent::Failed
.into_err()
.ctx(trc::Key::AccountName, fallback_user.to_string())
.ctx(trc::Key::SpanId, req.session_id)
.reason("Fallback admin authentication failed"))
};
}
// Add domain if missing, use the default domain
self.add_missing_domain(&mut username.account);
if let Some(master_user) = &mut username.master_user {
self.add_missing_domain(master_user);
}
// Obtain domain
let auth_as = username.auth_as();
let auth_as_address = auth_as.address();
let auth_as_local = auth_as.local();
let auth_as_domain = auth_as.domain().unwrap();
let domain = self.resolve_domain(auth_as_domain).await?;
// Authenticate app passwords
if let Some(app_pass) = AppPassword::parse(secret) {
if username.is_master() {
return Err(trc::AuthEvent::Failed
.into_err()
.ctx(trc::Key::AccountName, auth_as_address.to_string())
.ctx(trc::Key::SpanId, req.session_id)
.reason("App passwords cannot be used for impersonation"));
}
return if let Some(account_id) =
self.account_id_from_parts(auth_as_local, domain.id).await?
{
self.validate_credential(
account_id,
app_pass.credential_id,
app_pass.secret.as_ref(),
req.remote_ip,
req.session_id,
)
.await
} else {
Err(trc::AuthEvent::Failed
.into_err()
.ctx(trc::Key::AccountName, auth_as_address.to_string())
.reason("App password authentication failed: account not found"))
};
}
// Obtain external directory, if any
let mut is_alias_login = false;
let token = if let Some(directory) = self.get_directory_for_cached_domain(&domain) {
let directory_account = if username.is_master() {
directory
.authenticate(&Credentials::Basic {
username: auth_as_address.to_string(),
secret: secret.clone(),
mfa_token: mfa_token.clone(),
})
.await?
} else {
directory.authenticate(&req.credentials).await?
};
is_alias_login = directory_account.email != auth_as_address;
self.build_directory_token(directory_account, req.remote_ip)
.await
} else if let Some(account_id) =
self.account_id_from_parts(auth_as_local, domain.id).await?
{
if let Some(account) = self
.registry()
.object::<structs::Account>(account_id.into())
.await?
.and_then(|account| account.into_user())
{
let Some(credential) = account.password_credential() else {
return Err(trc::AuthEvent::Failed
.into_err()
.ctx(trc::Key::AccountName, auth_as_address.to_string())
.ctx(trc::Key::AccountId, account_id)
.ctx(trc::Key::SpanId, req.session_id)
.reason("Password credential not found for account"));
};
match verify_mfa_secret_hash(
credential.otp_auth.as_deref(),
mfa_token.as_deref(),
credential.secret.as_str(),
secret,
)
.await?
{
SecretVerificationResult::Valid => {
is_alias_login = account.name != auth_as_local;
self.access_token(account_id)
.await
.and_then(|token| AccessToken::new(token, req.remote_ip))
}
SecretVerificationResult::Invalid => Err(trc::AuthEvent::Failed
.into_err()
.ctx(trc::Key::AccountName, auth_as_address.to_string())
.ctx(trc::Key::AccountId, account_id)
.ctx(trc::Key::SpanId, req.session_id)
.reason("Authentication failed")),
SecretVerificationResult::MissingMfaToken => {
Err(trc::AuthEvent::MfaRequired
.into_err()
.ctx(trc::Key::AccountName, auth_as_address.to_string())
.ctx(trc::Key::AccountId, account_id)
.ctx(trc::Key::SpanId, req.session_id)
.reason("MFA token required"))
}
}
} else {
Err(trc::AuthEvent::Error
.into_err()
.ctx(trc::Key::AccountName, auth_as_address.to_string())
.ctx(trc::Key::AccountId, account_id)
.reason("Account not found in registry"))
}
} else {
Err(trc::AuthEvent::Failed
.into_err()
.ctx(trc::Key::AccountName, auth_as_address.to_string())
.reason("Account not found"))
}?;
// Enforce alias login restrictions
if is_alias_login && !token.has_permission(Permission::AuthenticateWithAlias) {
return Err(trc::AuthEvent::Failed
.into_err()
.ctx(trc::Key::AccountName, auth_as_address.to_string())
.ctx(trc::Key::AccountId, token.account_id())
.ctx(trc::Key::SpanId, req.session_id)
.reason("Authenticated using an email alias but account does not have AuthenticateAlias permission"));
}
// Validate master user access
if username.is_master() {
token.assert_has_permissions(&[
Permission::Impersonate,
Permission::Authenticate,
])?;
let address = username.account().address();
let master_address = auth_as_address;
if let Some(account_id) =
self.impersonated_account_id(username.account()).await?
{
trc::event!(
Auth(trc::AuthEvent::Success),
AccountName = address.to_string(),
AccountId = account_id,
SpanId = req.session_id,
Details = master_address.to_string(),
);
self.access_token(account_id)
.await
.map(AccessToken::new_maybe_invalid)
} else {
Err(trc::AuthEvent::Failed
.into_err()
.ctx(trc::Key::AccountName, address.to_string())
.details(master_address.to_string())
.reason("Master user account not found"))
}
} else {
trc::event!(
Auth(trc::AuthEvent::Success),
AccountName = auth_as_address.to_string(),
AccountId = token.account_id(),
SpanId = req.session_id,
);
Ok(token)
}
}
Credentials::Bearer { username, token } => {
// Handle API key authentication
if let Some(key) = ApiKey::parse(token) {
return self
.validate_credential(
key.account_id,
key.credential_id,
key.secret.as_ref(),
req.remote_ip,
req.session_id,
)
.await;
}
#[cfg(feature = "dev_mode")]
if std::env::var("API_TOKEN_ADMIN").is_ok_and(|admin_token| &admin_token == token) {
return Ok(AccessToken::new_admin());
}
// Obtain external directory, if any. When no username is supplied
// (e.g. HTTP bearer auth), peek at the JWT claims to find the
// user's domain so per-domain OIDC directories are reachable.
let directory = if let Some(username) = username.as_deref().map(UsernameParts::new)
{
if let Some(domain_name) = username.auth_as().domain() {
self.get_directory_for_domain(domain_name).await?
} else if let Some(domain_name) = extract_jwt_domain(token) {
self.get_directory_for_domain(&domain_name).await?
} else {
self.get_default_directory()
}
} else if let Some(domain_name) = extract_jwt_domain(token) {
self.get_directory_for_domain(&domain_name).await?
} else {
self.get_default_directory()
};
// Try external directory authentication first if supported, then fallback to internal OAuth.
let mut external_error = None;
if let Some(directory) = directory
&& directory.has_bearer_token_support()
{
match directory.authenticate(&req.credentials).await {
Ok(result) => {
return self.build_directory_token(result, req.remote_ip).await;
}
Err(err) => {
external_error = Some(err);
}
}
}
// Internal OAuth
match self
.validate_access_token(GrantType::AccessToken.into(), token)
.await
{
Ok(token_info) => self
.access_token(token_info.account_id)
.await
.and_then(|token| AccessToken::new(token, req.remote_ip)),
Err(err) => {
if let Some(external_error) = external_error {
Err(external_error)
} else {
Err(err)
}
}
}
}
}
}
async fn impersonated_account_id(&self, username: &Username) -> trc::Result<Option<u32>> {
let address = username.address();
if let Some(account_id) = self.account_id_from_email(address, false).await? {
return Ok(Some(account_id));
}
if let Some(domain) = username.domain()
&& let Some(domain_cache) = self.domain(domain).await?
&& let Some(directory) = self.get_directory_for_cached_domain(&domain_cache)
&& let Recipient::Account(account) = directory.recipient(address).await?
{
return Ok(Some(Box::pin(self.synchronize_account(account)).await?.id));
}
Ok(None)
}
async fn validate_credential(
&self,
account_id: u32,
credential_id: u32,
secret: &[u8],
remote_ip: IpAddr,
span_id: u64,
) -> trc::Result<AccessToken> {
if let Some(account) = self
.registry()
.object::<structs::Account>(account_id.into())
.await?
.and_then(|account| account.into_user())
{
// Find credential by credential_id
let mut authenticated = false;
for (credential, credential_type) in
account.credentials.iter().filter_map(|credential| {
credential
.as_secondary_credential()
.map(|secondary_credential| (secondary_credential, credential))
})
{
if credential.credential_id.document_id() == credential_id {
if !verify_secret_hash(&credential.secret, secret).await? {
return Err(trc::AuthEvent::Failed
.into_err()
.ctx(trc::Key::AccountName, account.name)
.ctx(trc::Key::AccountId, account_id)
.ctx(trc::Key::Id, credential_id)
.ctx(trc::Key::SpanId, span_id)
.reason("Invalid credential secret"));
}
if credential
.expires_at
.as_ref()
.is_some_and(|exp| exp.timestamp() < now() as i64)
{
return Err(trc::AuthEvent::CredentialExpired
.into_err()
.ctx(trc::Key::AccountName, account.name)
.ctx(trc::Key::AccountId, account_id)
.ctx(trc::Key::Id, credential_id)
.ctx(trc::Key::SpanId, span_id)
.reason("Credential has expired"));
}
trc::event!(
Auth(trc::AuthEvent::Success),
AccountName = account.name.clone(),
AccountId = account_id,
Id = credential_id,
SpanId = span_id,
Details = match credential_type {
Credential::AppPassword(_) => "Authenticated with app password",
Credential::ApiKey(_) => "Authenticated with API key",
_ => "Authenticated with credential",
}
);
authenticated = true;
break;
}
}
if authenticated {
let token = self
.access_token_from_account(account_id, structs::Account::User(account))
.await?;
AccessToken::new_scoped(token, credential_id, remote_ip)
.add_context(|ctx| ctx.span_id(span_id))
} else {
Err(trc::AuthEvent::Failed
.into_err()
.ctx(trc::Key::AccountId, account_id)
.ctx(trc::Key::Id, credential_id)
.ctx(trc::Key::SpanId, span_id)
.reason("Credential not found for account"))
}
} else {
Err(trc::AuthEvent::Failed
.into_err()
.ctx(trc::Key::AccountId, account_id)
.ctx(trc::Key::SpanId, span_id)
.reason("Account not found for credential"))
}
}
async fn resolve_domain(&self, domain_name: &str) -> trc::Result<Arc<DomainCache>> {
if let Some(domain) = self.domain(domain_name).await? {
Ok(domain)
} else {
Err(trc::AuthEvent::Failed
.into_err()
.ctx(trc::Key::Details, domain_name.to_string())
.reason("Domain not found"))
}
}
fn add_missing_domain(&self, address: &mut Username) {
if address.domain().is_none() {
trc::event!(
Auth(trc::AuthEvent::Warning),
AccountName = address.address().to_string(),
Reason = "No domain in username",
);
address.domain_start = address.name.len() + 1;
address.name = format!("{}@{}", address.name, self.core.email.default_domain_name);
}
}
async fn build_directory_token(
&self,
account: directory::Account,
remote_ip: IpAddr,
) -> trc::Result<AccessToken> {
let account = Box::pin(self.synchronize_account(account)).await?;
self.access_token_from_account(account.id, account.account)
.await
.and_then(|token| AccessToken::new(token, remote_ip))
}
pub async fn get_directory_for_domain(
&self,
domain_name: &str,
) -> trc::Result<Option<&Arc<Directory>>> {
Ok(self.get_default_directory())
}
pub fn get_directory_for_cached_domain(&self, domain: &DomainCache) -> Option<&Arc<Directory>> {
self.get_default_directory()
}
}
fn extract_jwt_domain(token: &str) -> Option<String> {
let mut parts = token.split('.');
let _header = parts.next()?;
let payload = parts.next()?;
let _signature = parts.next()?;
if parts.next().is_some() {
return None;
}
let payload_bytes = general_purpose::URL_SAFE_NO_PAD.decode(payload).ok()?;
let claims: serde_json::Value = serde_json::from_slice(&payload_bytes).ok()?;
for claim in ["email", "preferred_username", "upn"] {
if let Some(val) = claims.get(claim).and_then(|v| v.as_str())
&& let Some((_, domain)) = val.rsplit_once('@')
&& !domain.is_empty()
{
return Some(domain.to_ascii_lowercase());
}
}
None
}
impl UsernameParts {
pub fn new(address: &str) -> Self {
let mut account = Username {
name: String::with_capacity(address.len()),
domain_start: usize::MAX,
};
let mut master_user = None;
for ch in address.chars() {
if ch == '%' {
master_user = Some(Username {
name: String::with_capacity(address.len()),
domain_start: usize::MAX,
});
} else {
let target = master_user.as_mut().unwrap_or(&mut account);
if ch != '@' {
for lower in ch.to_lowercase() {
target.name.push(lower);
}
} else {
target.name.push(ch);
target.domain_start = target.name.len();
}
}
}
UsernameParts {
master_user: master_user.filter(|u| u != &account),
account,
}
}
pub fn auth_as(&self) -> &Username {
self.master_user.as_ref().unwrap_or(&self.account)
}
pub fn account(&self) -> &Username {
&self.account
}
pub fn is_master(&self) -> bool {
self.master_user.is_some()
}
}
impl Username {
pub fn address(&self) -> &str {
self.name.as_str()
}
pub fn local(&self) -> &str {
self.name
.get(..self.domain_start.saturating_sub(1))
.unwrap_or_default()
}
pub fn domain(&self) -> Option<&str> {
self.name.get(self.domain_start..)
}
}
impl AuthRequest {
pub fn from_credentials(credentials: Credentials, session_id: u64, remote_ip: IpAddr) -> Self {
Self {
credentials,
session_id,
remote_ip,
}
}
pub fn from_plain(
user: impl Into<String>,
pass: impl Into<String>,
session_id: u64,
remote_ip: IpAddr,
) -> Self {
Self::from_credentials(
Credentials::Basic {
username: user.into(),
secret: pass.into(),
mfa_token: None,
},
session_id,
remote_ip,
)
}
pub fn username(&self) -> Option<&str> {
match &self.credentials {
Credentials::Basic { username, .. } => Some(username.as_str()),
Credentials::Bearer { username, .. } => username.as_deref(),
}
}
}
crates/common/src/auth/credential.rs
+93
View File
@@ -0,0 +1,93 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use std::io::Write;
use store::{
U32_LEN,
rand::{self},
};
use utils::codec::base32_custom::{Base32Reader, Base32Writer};
pub struct ApiKey {
pub account_id: u32,
pub credential_id: u32,
pub secret: [u8; 20],
}
pub struct AppPassword {
pub credential_id: u32,
pub secret: [u8; 18],
}
impl ApiKey {
pub fn new(account_id: u32, credential_id: u32) -> Self {
ApiKey {
account_id,
credential_id,
secret: rand::random::<[u8; 20]>(),
}
}
pub fn parse(token: &str) -> Option<Self> {
let decoded = URL_SAFE_NO_PAD.decode(token.strip_prefix("API_")?).ok()?;
Some(ApiKey {
account_id: u32::from_be_bytes(decoded.get(0..U32_LEN)?.try_into().ok()?),
credential_id: u32::from_be_bytes(decoded.get(U32_LEN..U32_LEN * 2)?.try_into().ok()?),
secret: decoded.get(U32_LEN * 2..)?.try_into().ok()?,
})
}
pub fn build(&self) -> String {
let mut bytes = Vec::with_capacity(U32_LEN * 2 + self.secret.len());
bytes.extend_from_slice(&self.account_id.to_be_bytes());
bytes.extend_from_slice(&self.credential_id.to_be_bytes());
bytes.extend_from_slice(&self.secret);
format!("API_{}", URL_SAFE_NO_PAD.encode(bytes))
}
}
impl AppPassword {
pub fn new(credential_id: u32) -> Self {
AppPassword {
credential_id,
secret: rand::random::<[u8; 18]>(),
}
}
pub fn parse(token: &str) -> Option<Self> {
let token = token.strip_prefix("app")?;
let mut reader = Base32Reader::new(token.as_bytes().get(1..)?);
let mut credential_id = [0u8; 4];
let mut secret = [0u8; 18];
for byte in credential_id.iter_mut() {
*byte = reader.next()?;
}
for byte in secret.iter_mut() {
*byte = reader.next()?;
}
if reader.next().is_none() {
Some(AppPassword {
credential_id: u32::from_be_bytes(credential_id),
secret,
})
} else {
None
}
}
pub fn build(&self) -> String {
let mut writer = Base32Writer::with_capacity(std::mem::size_of::<Self>().div_ceil(5) * 8);
writer.push_string("app_");
let _ = writer.write(&self.credential_id.to_be_bytes());
let _ = writer.write_all(&self.secret);
writer.finalize()
}
}
crates/common/src/auth/mod.rs
+332
View File
@@ -0,0 +1,332 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::{
expr::if_block::IfBlock,
network::limiter::ConcurrencyLimiter,
storage::{ObjectQuota, TenantQuota},
};
use directory::Credentials;
use quick_cache::Equivalent;
use registry::{
schema::enums::{Locale, Permission},
types::{EnumImpl, ipmask::IpAddrOrMask},
};
use std::{
hash::{Hash, Hasher},
net::IpAddr,
sync::Arc,
};
use tinyvec::TinyVec;
use trc::ipc::bitset::Bitset;
use types::collection::Collection;
use utils::{cache::CacheItemWeight, map::bitmap::Bitmap};
pub mod access_token;
pub mod authentication;
pub mod credential;
pub mod oauth;
pub mod permissions;
pub mod rate_limit;
pub const RECOVERY_ADMIN_ID: u32 = u32::MAX;
const PERMISSIONS_BITSET_SIZE: usize = Permission::COUNT.div_ceil(std::mem::size_of::<usize>());
pub type Permissions = Bitset<PERMISSIONS_BITSET_SIZE>;
#[derive(Debug, PartialEq, Eq, Clone)]
pub struct EmailAddress {
pub local_part: Box<str>,
pub domain_id: u32,
}
#[derive(Debug, PartialEq, Eq)]
pub struct EmailAddressRef<'x> {
local_part: &'x str,
domain_id: u32,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum EmailCache {
Account(u32),
MailingList(u32),
DisabledAccountAddress(u32),
DisabledListAddress(u32),
}
#[derive(Debug, Clone)]
pub struct DomainCache {
pub names: Box<[Box<str>]>,
pub id: u32,
pub id_directory: Option<u32>,
pub id_tenant: Option<u32>,
pub catch_all: Option<Box<str>>,
pub sub_addressing_custom: Option<Box<IfBlock>>,
pub flags: u8,
}
pub const DOMAIN_FLAG_RELAY: u8 = 1;
pub const DOMAIN_FLAG_SUB_ADDRESSING: u8 = 1 << 1;
#[derive(Debug, Clone, Default)]
pub struct AccountCache {
pub name: Box<str>,
pub id: u32,
pub addresses: Box<[EmailAddress]>,
pub id_tenant: Option<u32>,
pub id_member_of: TinyVec<[u32; 3]>,
pub quota_disk: u64,
pub quota_objects: Option<Box<ObjectQuota>>,
pub description: Option<Box<str>>,
pub encryption_key: Option<EncryptionKeys>,
pub locale: Locale,
pub flags: u64,
}
pub type EncryptionKeys = Box<[Box<[u8]>]>;
pub const ACCOUNT_IS_USER: u64 = 1;
pub const ACCOUNT_FLAG_ENCRYPT_TRAIN_SPAM_FILTER: u64 = 1 << 1;
pub const ACCOUNT_FLAG_ENCRYPT_METHOD_SMIME: u64 = 1 << 2;
pub const ACCOUNT_FLAG_ENCRYPT_METHOD_PGP: u64 = 1 << 3;
pub const ACCOUNT_FLAG_ENCRYPT_ALGO_AES256: u64 = 1 << 4;
pub const ACCOUNT_FLAG_ENCRYPT_ALGO_AES128: u64 = 1 << 5;
pub const ACCOUNT_FLAG_ENCRYPT_APPEND: u64 = 1 << 6;
pub const ACCOUNT_FLAG_ENCRYPT_ALGO_AES256_GCM: u64 = 1 << 7;
pub const ACCOUNT_FLAG_ENCRYPT_ALGO_CHACHA20_POLY1305: u64 = 1 << 8;
#[derive(Debug, Clone)]
pub struct RoleCache {
pub id_roles: TinyVec<[u32; 3]>,
pub permissions: PermissionsGroup,
}
#[derive(Debug, Clone)]
pub struct MailingListCache {
pub addresses: Box<[EmailAddress]>,
pub recipients: Arc<[Box<str>]>,
}
#[derive(Debug, Clone)]
pub struct TenantCache {
pub id_roles: TinyVec<[u32; 3]>,
pub quota_disk: u64,
pub quota_objects: Option<Box<TenantQuota>>,
pub permissions: Option<Box<PermissionsGroup>>,
}
#[derive(Debug, Clone, Default)]
pub struct PermissionsGroup {
pub enabled: Permissions,
pub disabled: Permissions,
pub merge: bool,
}
#[derive(Debug, Default, Clone)]
pub struct AccessToken {
scope_idx: usize,
inner: Arc<AccessTokenInner>,
}
#[derive(Debug, Default, Clone)]
pub struct AccessTokenInner {
pub(crate) account_id: u32,
pub(crate) tenant_id: Option<u32>,
pub(crate) member_of: TinyVec<[u32; 3]>,
pub(crate) access_to: Box<[AccessTo]>,
pub(crate) scopes: Box<[AccessScope]>,
pub(crate) concurrent_http_requests: Option<ConcurrencyLimiter>,
pub(crate) concurrent_imap_requests: Option<ConcurrencyLimiter>,
pub(crate) concurrent_uploads: Option<ConcurrencyLimiter>,
pub(crate) revision_account: u64,
pub(crate) revision: u64,
pub(crate) credential_version: u64,
pub(crate) obj_size: u64,
}
#[derive(Debug, Default, Hash, Clone)]
pub struct AccessScope {
pub permissions: Permissions,
pub credential_id: u32,
pub expires_at: u64,
pub allowed_ips: Box<[IpAddrOrMask]>,
}
#[derive(Debug, Default, Hash, PartialEq, Eq, Clone)]
pub(crate) struct AccessTo {
pub account_id: u32,
pub collections: Bitmap<Collection>,
}
#[derive(Clone)]
pub struct AccountInfo {
pub account_id: u32,
pub account: Arc<AccountCache>,
pub addresses: Vec<String>,
}
#[derive(Clone, Copy)]
pub struct AccountTenantIds {
pub account_id: u32,
pub tenant_id: Option<u32>,
}
pub struct AuthRequest {
pub credentials: Credentials,
pub session_id: u64,
pub remote_ip: IpAddr,
}
impl CacheItemWeight for AccessTokenInner {
fn weight(&self) -> u64 {
self.obj_size
}
}
impl CacheItemWeight for EmailAddress {
fn weight(&self) -> u64 {
std::mem::size_of::<EmailAddress>() as u64 + self.local_part.len() as u64
}
}
impl CacheItemWeight for EmailCache {
fn weight(&self) -> u64 {
std::mem::size_of::<EmailCache>() as u64
}
}
impl CacheItemWeight for DomainCache {
fn weight(&self) -> u64 {
std::mem::size_of::<DomainCache>() as u64
+ self
.names
.iter()
.map(|s| s.len() as u64 + std::mem::size_of::<Box<str>>() as u64)
.sum::<u64>()
+ self.catch_all.as_ref().map_or(0, |s| s.len() as u64)
+ self
.sub_addressing_custom
.as_ref()
.map_or(0, |s| s.weight())
}
}
impl Equivalent<EmailAddress> for EmailAddressRef<'_> {
fn equivalent(&self, key: &EmailAddress) -> bool {
self.local_part == &*key.local_part && self.domain_id == key.domain_id
}
}
impl Hash for EmailAddress {
fn hash<H: Hasher>(&self, state: &mut H) {
self.local_part.as_ref().hash(state);
self.domain_id.hash(state);
}
}
impl Hash for EmailAddressRef<'_> {
fn hash<H: Hasher>(&self, state: &mut H) {
self.local_part.hash(state);
self.domain_id.hash(state);
}
}
impl CacheItemWeight for AccountCache {
fn weight(&self) -> u64 {
std::mem::size_of::<AccountCache>() as u64
+ self.name.len() as u64
+ self
.addresses
.iter()
.map(|s| s.local_part.len() as u64 + std::mem::size_of::<EmailAddress>() as u64)
.sum::<u64>()
+ self.description.as_ref().map_or(0, |s| s.len() as u64)
+ self.encryption_key.as_ref().map_or(0, |keys| {
keys.iter()
.map(|k| k.len() as u64 + std::mem::size_of::<Box<[u8]>>() as u64)
.sum::<u64>()
})
}
}
impl CacheItemWeight for RoleCache {
fn weight(&self) -> u64 {
std::mem::size_of::<RoleCache>() as u64
}
}
impl CacheItemWeight for MailingListCache {
fn weight(&self) -> u64 {
std::mem::size_of::<MailingListCache>() as u64
+ self
.addresses
.iter()
.map(|s| s.local_part.len() as u64 + std::mem::size_of::<EmailAddress>() as u64)
.sum::<u64>()
+ self
.recipients
.iter()
.map(|s| s.len() as u64 + std::mem::size_of::<Box<str>>() as u64)
.sum::<u64>()
}
}
impl CacheItemWeight for TenantCache {
fn weight(&self) -> u64 {
std::mem::size_of::<TenantCache>() as u64
+ self.permissions.as_ref().map_or(0, |p| p.weight())
}
}
impl CacheItemWeight for PermissionsGroup {
fn weight(&self) -> u64 {
std::mem::size_of::<PermissionsGroup>() as u64
}
}
pub trait BuildAccessToken {
fn build(self) -> AccessToken;
}
impl BuildAccessToken for Arc<AccessTokenInner> {
fn build(self) -> AccessToken {
AccessToken {
scope_idx: 0,
inner: self,
}
}
}
impl EmailAddress {
pub fn new(local_part: impl Into<Box<str>>, domain_id: u32) -> Self {
Self {
local_part: local_part.into(),
domain_id,
}
}
}
impl<'x> EmailAddressRef<'x> {
pub fn new(local_part: &'x str, domain_id: u32) -> Self {
Self {
local_part,
domain_id,
}
}
}
impl AccountCache {
pub fn domain_id(&self) -> Option<u32> {
self.addresses.first().map(|address| address.domain_id)
}
}
impl DomainCache {
pub fn name(&self) -> &str {
self.names.first().map(|s| s.as_ref()).unwrap_or_default()
}
}
crates/common/src/auth/oauth/client_id.rs
+225
View File
@@ -0,0 +1,225 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use super::{
SCOPE_CALENDARS, SCOPE_CONTACTS, SCOPE_MAIL, SCOPE_OFFLINE_ACCESS, SCOPE_OPENID,
crypto::SymmetricEncrypt,
};
use base64::{Engine, engine::general_purpose};
use store::blake3;
use utils::codec::leb128::{Leb128Iterator, Leb128Vec};
const CLIENT_ID_HEADER: &str = "swc1.";
const CLIENT_ID_KEY_CONTEXT: &str = "stalwart-oauth-client-id-sw1";
const CLIENT_ID_VERSION: u8 = 1;
const SCOPE_BITS: &[&str] = &[
SCOPE_OPENID,
SCOPE_OFFLINE_ACCESS,
SCOPE_MAIL,
SCOPE_CONTACTS,
SCOPE_CALENDARS,
];
pub fn scopes_to_mask(scope: &str) -> u64 {
let mut mask = 0u64;
for scope in scope.split_ascii_whitespace() {
if let Some(bit) = SCOPE_BITS.iter().position(|known| *known == scope) {
mask |= 1 << bit;
}
}
mask
}
#[derive(Debug, Clone, PartialEq, Eq, Default)]
pub struct ClientMeta {
pub redirect_uris: Vec<String>,
pub scope_mask: u64,
pub client_name: Option<String>,
}
pub fn encode_client_id(key: &[u8], meta: &ClientMeta) -> Result<String, String> {
let client_name = meta.client_name.as_deref().unwrap_or_default();
let mut payload = Vec::with_capacity(
24 + meta
.redirect_uris
.iter()
.map(|u| u.len() + 2)
.sum::<usize>()
+ client_name.len(),
);
payload.push(CLIENT_ID_VERSION);
payload.push_leb128(meta.redirect_uris.len());
for uri in &meta.redirect_uris {
payload.push_leb128(uri.len());
payload.extend_from_slice(uri.as_bytes());
}
payload.push_leb128(meta.scope_mask);
payload.push_leb128(client_name.len());
payload.extend_from_slice(client_name.as_bytes());
let digest = blake3::hash(&payload);
let nonce = &digest.as_bytes()[..SymmetricEncrypt::NONCE_LEN];
let ciphertext =
SymmetricEncrypt::new(key, CLIENT_ID_KEY_CONTEXT).encrypt_with_aad(&payload, nonce, &[])?;
let mut body = Vec::with_capacity(nonce.len() + ciphertext.len());
body.extend_from_slice(nonce);
body.extend_from_slice(&ciphertext);
let mut out = String::with_capacity(CLIENT_ID_HEADER.len() + body.len().div_ceil(3) * 4);
out.push_str(CLIENT_ID_HEADER);
general_purpose::URL_SAFE_NO_PAD.encode_string(&body, &mut out);
Ok(out)
}
pub fn decode_client_id(key: &[u8], client_id: &str) -> Option<ClientMeta> {
let body = general_purpose::URL_SAFE_NO_PAD
.decode(client_id.strip_prefix(CLIENT_ID_HEADER)?.as_bytes())
.ok()?;
if body.len() < SymmetricEncrypt::NONCE_LEN + SymmetricEncrypt::ENCRYPT_TAG_LEN {
return None;
}
let (nonce, ciphertext) = body.split_at(SymmetricEncrypt::NONCE_LEN);
let payload = SymmetricEncrypt::new(key, CLIENT_ID_KEY_CONTEXT)
.decrypt_with_aad(ciphertext, nonce, &[])
.ok()?;
let mut bytes = payload.iter();
if bytes.next().copied()? != CLIENT_ID_VERSION {
return None;
}
let uri_count: usize = bytes.next_leb128()?;
if uri_count > u8::MAX as usize {
return None;
}
let mut redirect_uris = Vec::with_capacity(uri_count);
for _ in 0..uri_count {
redirect_uris.push(take_string(&mut bytes)?);
}
let scope_mask: u64 = bytes.next_leb128()?;
let client_name = take_string(&mut bytes)?;
Some(ClientMeta {
redirect_uris,
scope_mask,
client_name: (!client_name.is_empty()).then_some(client_name),
})
}
fn take_string(bytes: &mut std::slice::Iter<'_, u8>) -> Option<String> {
let len: usize = bytes.next_leb128()?;
let slice = bytes.as_slice();
if slice.len() < len {
return None;
}
let value = String::from_utf8(slice[..len].to_vec()).ok()?;
if len > 0 {
bytes.nth(len - 1)?;
}
Some(value)
}
#[cfg(test)]
mod tests {
use super::*;
const KEY: &[u8] = b"a-test-encryption-key-of-some-length";
fn sample() -> ClientMeta {
ClientMeta {
redirect_uris: vec![
"http://127.0.0.1/cb".to_string(),
"com.example.app:/oauth".to_string(),
],
scope_mask: scopes_to_mask(&format!("{SCOPE_OFFLINE_ACCESS} {SCOPE_MAIL}")),
client_name: Some("Example Client".to_string()),
}
}
#[test]
fn round_trip_preserves_all_fields() {
for meta in [
sample(),
ClientMeta {
redirect_uris: vec!["http://[::1]/".to_string()],
scope_mask: 0,
client_name: None,
},
ClientMeta::default(),
] {
let client_id = encode_client_id(KEY, &meta).unwrap();
assert!(client_id.starts_with(CLIENT_ID_HEADER));
assert_eq!(decode_client_id(KEY, &client_id), Some(meta));
}
}
#[test]
fn scope_mask_is_order_independent_and_drops_unknown() {
assert_eq!(
scopes_to_mask(&format!("{SCOPE_MAIL} {SCOPE_OFFLINE_ACCESS}")),
scopes_to_mask(&format!("{SCOPE_OFFLINE_ACCESS} {SCOPE_MAIL}"))
);
assert_eq!(
scopes_to_mask(&format!("{SCOPE_MAIL} custom:unknown")),
scopes_to_mask(SCOPE_MAIL)
);
assert_eq!(scopes_to_mask("totally unknown"), 0);
}
#[test]
fn identical_input_is_deterministic() {
let meta = sample();
assert_eq!(
encode_client_id(KEY, &meta).unwrap(),
encode_client_id(KEY, &meta).unwrap()
);
}
#[test]
fn wrong_key_is_rejected() {
let client_id = encode_client_id(KEY, &sample()).unwrap();
assert_eq!(
decode_client_id(b"a-completely-different-key-value!", &client_id),
None
);
}
#[test]
fn tampering_is_rejected() {
let client_id = encode_client_id(KEY, &sample()).unwrap();
let (header, body_b64) = client_id.split_at(CLIENT_ID_HEADER.len());
let mut body = general_purpose::URL_SAFE_NO_PAD.decode(body_b64).unwrap();
for idx in 0..body.len() {
let mut tampered = body.clone();
tampered[idx] ^= 0x01;
let forged = format!(
"{header}{}",
general_purpose::URL_SAFE_NO_PAD.encode(&tampered)
);
assert_eq!(decode_client_id(KEY, &forged), None, "byte {idx}");
}
body[0] ^= 0x00;
assert!(decode_client_id(KEY, &client_id).is_some());
}
#[test]
fn malformed_input_never_panics() {
for case in [
"",
"swc1.",
"swc1.!!!",
"swc1.AAAA",
"wrong.AAAA",
"swc1.AAAAAAAAAAAAAAAAAAAAAAAAAAAA",
] {
assert_eq!(decode_client_id(KEY, case), None, "{case:?}");
}
}
}
crates/common/src/auth/oauth/config.rs
+203
View File
@@ -0,0 +1,203 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::{
config::{EcKeyCurve, build_ecdsa_pem, build_rsa_keypair},
manager::application::Resource,
};
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use jsonwebtoken::{
Algorithm, EncodingKey,
jwk::{
AlgorithmParameters, CommonParameters, EllipticCurve, EllipticCurveKeyParameters,
EllipticCurveKeyType, Jwk, JwkSet, KeyAlgorithm, OctetKeyParameters, OctetKeyType,
PublicKeyUse, RSAKeyParameters, RSAKeyType,
},
};
use registry::schema::{enums::JwtSignatureAlgorithm, prelude::ObjectType, structs::OidcProvider};
use std::borrow::Cow;
use store::{
rand::{RngExt, distr::Alphanumeric, rng},
registry::bootstrap::Bootstrap,
};
#[derive(Clone)]
pub struct OAuthConfig {
pub oauth_key: String,
pub oauth_expiry_user_code: u64,
pub oauth_expiry_auth_code: u64,
pub oauth_expiry_token: u64,
pub oauth_expiry_refresh_token: u64,
pub oauth_expiry_refresh_token_renew: u64,
pub oauth_max_auth_attempts: u32,
pub allow_anonymous_client_registration: bool,
pub require_client_authentication: bool,
pub oidc_expiry_id_token: u64,
pub oidc_signing_secret: EncodingKey,
pub oidc_signature_algorithm: Algorithm,
pub oidc_jwks: Resource<Vec<u8>>,
}
impl OAuthConfig {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let auth = bp.setting_infallible::<OidcProvider>().await;
let oidc_signature_algorithm = match auth.signature_algorithm {
JwtSignatureAlgorithm::Es256 => Algorithm::ES256,
JwtSignatureAlgorithm::Es384 => Algorithm::ES384,
JwtSignatureAlgorithm::Ps256 => Algorithm::PS256,
JwtSignatureAlgorithm::Ps384 => Algorithm::PS384,
JwtSignatureAlgorithm::Ps512 => Algorithm::PS512,
JwtSignatureAlgorithm::Rs256 => Algorithm::RS256,
JwtSignatureAlgorithm::Rs384 => Algorithm::RS384,
JwtSignatureAlgorithm::Rs512 => Algorithm::RS512,
JwtSignatureAlgorithm::Hs256 => Algorithm::HS256,
JwtSignatureAlgorithm::Hs384 => Algorithm::HS384,
JwtSignatureAlgorithm::Hs512 => Algorithm::HS512,
};
let rand_key = rng()
.sample_iter(Alphanumeric)
.take(64)
.map(char::from)
.collect::<String>();
let signature_key = auth
.signature_key
.secret()
.await
.map_err(|err| {
bp.build_error(ObjectType::OidcProvider.singleton(), err);
})
.unwrap_or(Cow::Borrowed(rand_key.as_str()));
let fallback_key = || {
(
EncodingKey::from_secret(rand_key.as_bytes()),
AlgorithmParameters::OctetKey(OctetKeyParameters {
key_type: OctetKeyType::Octet,
value: URL_SAFE_NO_PAD.encode(&rand_key),
})
.into(),
)
};
let (oidc_signing_secret, algorithm) = match oidc_signature_algorithm {
Algorithm::HS256 | Algorithm::HS384 | Algorithm::HS512 => {
(EncodingKey::from_secret(signature_key.as_bytes()), None)
}
Algorithm::RS256
| Algorithm::RS384
| Algorithm::RS512
| Algorithm::PS256
| Algorithm::PS384
| Algorithm::PS512 => parse_rsa_key(&auth)
.await
.map_err(|err| {
bp.build_error(ObjectType::OidcProvider.singleton(), err);
})
.map(|(secret, alg)| (secret, Some(alg)))
.unwrap_or_else(|_| fallback_key()),
Algorithm::ES256 | Algorithm::ES384 => parse_ecdsa_key(&auth, oidc_signature_algorithm)
.await
.map_err(|err| {
bp.build_error(ObjectType::OidcProvider.singleton(), err);
})
.map(|(secret, alg)| (secret, Some(alg)))
.unwrap_or_else(|_| fallback_key()),
_ => {
bp.build_error(
ObjectType::OidcProvider.singleton(),
format!("Unsupported OIDC signature algorithm {oidc_signature_algorithm:?}"),
);
fallback_key()
}
};
let oidc_jwks = Resource {
content_type: "application/json".into(),
contents: serde_json::to_string(&JwkSet {
keys: algorithm
.into_iter()
.map(|algorithm| Jwk {
common: CommonParameters {
public_key_use: PublicKeyUse::Signature.into(),
key_algorithm: KeyAlgorithm::from(oidc_signature_algorithm).into(),
key_id: "default".to_string().into(),
..Default::default()
},
algorithm,
})
.collect(),
})
.unwrap_or_default()
.into_bytes(),
};
OAuthConfig {
oauth_key: auth
.encryption_key
.secret()
.await
.map_err(|err| bp.build_error(ObjectType::OidcProvider.singleton(), err))
.map_or_else(|_| rand_key.clone(), Cow::into_owned),
oauth_expiry_user_code: auth.user_code_expiry.as_secs(),
oauth_expiry_auth_code: auth.auth_code_expiry.as_secs(),
oauth_expiry_token: auth.access_token_expiry.as_secs(),
oauth_expiry_refresh_token: auth.refresh_token_expiry.as_secs(),
oauth_expiry_refresh_token_renew: auth.refresh_token_renewal.as_secs(),
oauth_max_auth_attempts: auth.auth_code_max_attempts as u32,
oidc_expiry_id_token: auth.id_token_expiry.as_secs(),
allow_anonymous_client_registration: auth.anonymous_client_registration,
require_client_authentication: auth.require_client_registration,
oidc_signing_secret,
oidc_signature_algorithm,
oidc_jwks,
}
}
}
async fn parse_rsa_key(auth: &OidcProvider) -> Result<(EncodingKey, AlgorithmParameters), String> {
let rsa_key = build_rsa_keypair(auth.signature_key.secret().await?.as_ref())?;
let rsa_key_params = RSAKeyParameters {
key_type: RSAKeyType::RSA,
n: URL_SAFE_NO_PAD.encode(&rsa_key.modulus),
e: URL_SAFE_NO_PAD.encode(&rsa_key.exponent),
};
Ok((
EncodingKey::from_rsa_der(&rsa_key.pkcs1_der),
AlgorithmParameters::RSA(rsa_key_params),
))
}
async fn parse_ecdsa_key(
auth: &OidcProvider,
oidc_signature_algorithm: Algorithm,
) -> Result<(EncodingKey, AlgorithmParameters), String> {
let (curve, ec_curve) = match oidc_signature_algorithm {
Algorithm::ES256 => (EllipticCurve::P256, EcKeyCurve::P256),
Algorithm::ES384 => (EllipticCurve::P384, EcKeyCurve::P384),
_ => unreachable!(),
};
let ecdsa_key = build_ecdsa_pem(ec_curve, auth.signature_key.secret().await?.as_ref())?;
let ecdsa_key_params = EllipticCurveKeyParameters {
key_type: EllipticCurveKeyType::EC,
curve,
x: URL_SAFE_NO_PAD.encode(&ecdsa_key.x),
y: URL_SAFE_NO_PAD.encode(&ecdsa_key.y),
};
Ok((
EncodingKey::from_ec_der(&ecdsa_key.pkcs8_der),
AlgorithmParameters::EllipticCurve(ecdsa_key_params),
))
}
crates/common/src/auth/oauth/crypto.rs
+54
View File
@@ -0,0 +1,54 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use aes_gcm_siv::{
Aes256GcmSiv, Key, KeyInit, Nonce,
aead::{Aead, Payload},
};
use store::blake3;
pub struct SymmetricEncrypt {
aes: Aes256GcmSiv,
}
impl SymmetricEncrypt {
pub const ENCRYPT_TAG_LEN: usize = 16;
pub const NONCE_LEN: usize = 12;
pub fn new(key: &[u8], context: &str) -> Self {
SymmetricEncrypt {
aes: Aes256GcmSiv::new(&Key::<Aes256GcmSiv>::from(blake3::derive_key(context, key))),
}
}
pub fn encrypt_with_aad(
&self,
bytes: &[u8],
nonce: &[u8],
aad: &[u8],
) -> Result<Vec<u8>, String> {
self.aes
.encrypt(
<&Nonce>::try_from(nonce).map_err(|e| e.to_string())?,
Payload { msg: bytes, aad },
)
.map_err(|e| e.to_string())
}
pub fn decrypt_with_aad(
&self,
bytes: &[u8],
nonce: &[u8],
aad: &[u8],
) -> Result<Vec<u8>, String> {
self.aes
.decrypt(
<&Nonce>::try_from(nonce).map_err(|e| e.to_string())?,
Payload { msg: bytes, aad },
)
.map_err(|e| e.to_string())
}
}
crates/common/src/auth/oauth/introspect.rs
+81
View File
@@ -0,0 +1,81 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::{Server, auth::AccessToken};
use serde::{Deserialize, Serialize};
use trc::{AddContext, AuthEvent, EventType};
#[derive(Debug, Default, Clone, Eq, PartialEq, Deserialize, Serialize)]
pub struct OAuthIntrospect {
#[serde(default)]
pub active: bool,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub scope: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub client_id: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub username: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub token_type: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub exp: Option<i64>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub iat: Option<i64>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub nbf: Option<i64>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub sub: Option<String>,
}
impl Server {
pub async fn introspect_access_token(
&self,
token: &str,
access_token: &AccessToken,
) -> trc::Result<OAuthIntrospect> {
match self.validate_access_token(None, token).await {
Ok(token_info) => Ok(OAuthIntrospect {
active: true,
username: self
.account(access_token.account_id())
.await
.caused_by(trc::location!())?
.name()
.to_string()
.into(),
token_type: Some("bearer".into()),
exp: Some(token_info.expiry as i64),
iat: Some(token_info.issued_at as i64),
..Default::default()
}),
Err(err)
if matches!(
err.event_type(),
EventType::Auth(AuthEvent::Error) | EventType::Auth(AuthEvent::TokenExpired)
) =>
{
Ok(OAuthIntrospect::default())
}
Err(err) => Err(err),
}
}
}
crates/common/src/auth/oauth/mod.rs
+80
View File
@@ -0,0 +1,80 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
pub mod client_id;
pub mod config;
pub mod crypto;
pub mod introspect;
pub mod oidc;
pub mod registration;
pub mod token;
pub const DEVICE_CODE_LEN: usize = 40;
pub const USER_CODE_LEN: usize = 8;
pub const RANDOM_CODE_LEN: usize = 32;
pub const CLIENT_ID_MAX_LEN: usize = 2048;
pub const USER_CODE_ALPHABET: &[u8] = b"ABCDEFGHJKLMNPQRSTUVWXYZ23456789"; // No 0, O, I, 1
pub const SCOPE_OPENID: &str = "openid";
pub const SCOPE_OFFLINE_ACCESS: &str = "offline_access";
pub const SCOPE_MAIL: &str = "urn:ietf:params:oauth:scope:mail";
pub const SCOPE_CONTACTS: &str = "urn:ietf:params:oauth:scope:contacts";
pub const SCOPE_CALENDARS: &str = "urn:ietf:params:oauth:scope:calendars";
pub const SUPPORTED_SCOPES: &[&str] = &[
SCOPE_OPENID,
SCOPE_OFFLINE_ACCESS,
SCOPE_MAIL,
SCOPE_CONTACTS,
SCOPE_CALENDARS,
];
#[derive(Debug, Clone, Copy, Eq, PartialEq)]
pub enum GrantType {
AccessToken,
RefreshToken,
LiveTracing,
LiveMetrics,
LiveDelivery,
Rsvp,
}
impl GrantType {
pub fn as_str(&self) -> &'static str {
match self {
GrantType::AccessToken => "access_token",
GrantType::RefreshToken => "refresh_token",
GrantType::LiveTracing => "live_tracing",
GrantType::LiveMetrics => "live_metrics",
GrantType::LiveDelivery => "live_delivery",
GrantType::Rsvp => "rsvp",
}
}
pub fn id(&self) -> u8 {
match self {
GrantType::AccessToken => 0,
GrantType::RefreshToken => 1,
GrantType::LiveTracing => 2,
GrantType::LiveMetrics => 3,
GrantType::LiveDelivery => 4,
GrantType::Rsvp => 5,
}
}
pub fn from_id(id: u8) -> Option<Self> {
match id {
0 => Some(GrantType::AccessToken),
1 => Some(GrantType::RefreshToken),
2 => Some(GrantType::LiveTracing),
3 => Some(GrantType::LiveMetrics),
4 => Some(GrantType::LiveDelivery),
5 => Some(GrantType::Rsvp),
_ => None,
}
}
}
crates/common/src/auth/oauth/oidc.rs
+182
View File
@@ -0,0 +1,182 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use std::fmt;
use jsonwebtoken::Header;
use serde::{
Deserialize, Deserializer, Serialize,
de::{self, Visitor},
};
use store::write::now;
use crate::Server;
#[derive(Debug, Default, Clone, Eq, PartialEq, Deserialize, Serialize)]
pub struct Userinfo {
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub sub: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub name: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub given_name: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub family_name: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub middle_name: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub nickname: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub preferred_username: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub profile: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub picture: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub website: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub email: Option<String>,
#[serde(default, deserialize_with = "any_bool")]
#[serde(skip_serializing_if = "std::ops::Not::not")]
pub email_verified: bool,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub zoneinfo: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub locale: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub updated_at: Option<i64>,
}
#[derive(Debug, Default, Eq, PartialEq, Deserialize, Serialize)]
pub struct StandardClaims {
#[serde(skip_serializing_if = "Option::is_none")]
#[serde(default)]
pub nonce: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
#[serde(default)]
pub preferred_username: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
#[serde(default)]
pub email: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
#[serde(default)]
pub description: Option<String>,
}
#[derive(Serialize)]
struct IdTokenClaims {
iss: String,
sub: String,
aud: String,
nbf: i64,
iat: i64,
exp: i64,
#[serde(flatten)]
private: StandardClaims,
}
impl Server {
pub fn issue_id_token(
&self,
subject: impl Into<String>,
issuer: impl Into<String>,
audience: impl Into<String>,
claims: StandardClaims,
) -> trc::Result<String> {
let now = now() as i64;
jsonwebtoken::encode(
&Header {
kid: Some("default".into()),
..Header::new(self.core.oauth.oidc_signature_algorithm)
},
&IdTokenClaims {
iss: issuer.into(),
sub: subject.into(),
aud: audience.into(),
nbf: now,
iat: now,
exp: now + self.core.oauth.oidc_expiry_id_token as i64,
private: claims,
},
&self.core.oauth.oidc_signing_secret,
)
.map_err(|err| {
trc::AuthEvent::Error
.into_err()
.reason(err)
.details("Failed to encode ID token")
})
}
}
fn any_bool<'de, D>(deserializer: D) -> Result<bool, D::Error>
where
D: Deserializer<'de>,
{
struct AnyBoolVisitor;
impl Visitor<'_> for AnyBoolVisitor {
type Value = bool;
fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
formatter.write_str("a boolean value")
}
fn visit_str<E>(self, value: &str) -> Result<bool, E>
where
E: de::Error,
{
match value {
"true" => Ok(true),
"false" => Ok(false),
_ => Err(E::custom(format!("Unknown boolean: {value}"))),
}
}
fn visit_bool<E>(self, value: bool) -> Result<bool, E>
where
E: de::Error,
{
Ok(value)
}
}
deserializer.deserialize_any(AnyBoolVisitor)
}
crates/common/src/auth/oauth/registration.rs
+295
View File
@@ -0,0 +1,295 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
#[derive(Serialize, Deserialize, Debug, Default)]
#[serde(rename_all = "snake_case")]
pub struct ClientRegistrationRequest {
pub redirect_uris: Vec<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub scope: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Vec::is_empty")]
pub response_types: Vec<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Vec::is_empty")]
pub grant_types: Vec<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub application_type: Option<ApplicationType>,
#[serde(default)]
#[serde(skip_serializing_if = "Vec::is_empty")]
pub contacts: Vec<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub client_name: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub logo_uri: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub client_uri: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub policy_uri: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub tos_uri: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub jwks_uri: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub jwks: Option<serde_json::Value>, // Using serde_json::Value for flexibility
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub sector_identifier_uri: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub subject_type: Option<SubjectType>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub id_token_signed_response_alg: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub id_token_encrypted_response_alg: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub id_token_encrypted_response_enc: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub userinfo_signed_response_alg: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub userinfo_encrypted_response_alg: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub userinfo_encrypted_response_enc: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub request_object_signing_alg: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub request_object_encryption_alg: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub request_object_encryption_enc: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub token_endpoint_auth_method: Option<TokenEndpointAuthMethod>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub token_endpoint_auth_signing_alg: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub default_max_age: Option<u64>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub require_auth_time: Option<bool>,
#[serde(default)]
#[serde(skip_serializing_if = "Vec::is_empty")]
pub default_acr_values: Vec<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")]
pub initiate_login_uri: Option<String>,
#[serde(default)]
#[serde(skip_serializing_if = "Vec::is_empty")]
pub request_uris: Vec<String>,
#[serde(flatten)]
#[serde(skip_serializing_if = "HashMap::is_empty")]
pub additional_fields: HashMap<String, serde_json::Value>,
}
#[derive(Serialize, Deserialize, Debug, Default)]
#[serde(rename_all = "snake_case")]
pub struct ClientRegistrationResponse {
// Required fields
pub client_id: String,
// Optional fields specific to the response
#[serde(skip_serializing_if = "Option::is_none")]
pub client_secret: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub registration_access_token: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub registration_client_uri: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub client_id_issued_at: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub client_secret_expires_at: Option<u64>,
// Echo back the request
#[serde(flatten)]
pub request: ClientRegistrationRequest,
}
#[derive(Serialize, Deserialize, Debug)]
#[serde(rename_all = "lowercase")]
pub enum ApplicationType {
Web,
Native,
}
#[derive(Serialize, Deserialize, Debug)]
#[serde(rename_all = "lowercase")]
pub enum SubjectType {
Pairwise,
Public,
}
#[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum TokenEndpointAuthMethod {
ClientSecretPost,
ClientSecretBasic,
ClientSecretJwt,
PrivateKeyJwt,
None,
}
#[derive(Serialize, Debug)]
pub struct ClientRegistrationError {
pub error: &'static str,
#[serde(skip_serializing_if = "Option::is_none")]
pub error_description: Option<&'static str>,
}
impl ClientRegistrationError {
pub fn invalid_redirect_uri(description: &'static str) -> Self {
ClientRegistrationError {
error: "invalid_redirect_uri",
error_description: Some(description),
}
}
pub fn invalid_client_metadata(description: &'static str) -> Self {
ClientRegistrationError {
error: "invalid_client_metadata",
error_description: Some(description),
}
}
}
pub fn loopback_redirect_parts(uri: &str) -> Option<(&str, &str)> {
let uri = uri.strip_prefix("http://")?;
for host in ["127.0.0.1", "[::1]"] {
if let Some(rest) = uri.strip_prefix(host) {
if let Some(path) = rest.strip_prefix('/') {
return Some((host, path));
} else if let Some(after_colon) = rest.strip_prefix(':')
&& let Some((port, path)) = after_colon.split_once('/')
&& !port.is_empty()
&& port.bytes().all(|b| b.is_ascii_digit())
{
return Some((host, path));
}
}
}
None
}
pub fn redirect_uri_matches(registered: &str, presented: &str) -> bool {
registered == presented
|| matches!(
(
loopback_redirect_parts(registered),
loopback_redirect_parts(presented),
),
(Some(reg), Some(pres)) if reg == pres
)
}
pub fn validate_redirect_uri(uri: &str) -> Result<(), ClientRegistrationError> {
if uri.contains('#') {
return Err(ClientRegistrationError::invalid_redirect_uri(
"Redirect URI must not contain a fragment.",
));
} else if uri.contains("..") {
return Err(ClientRegistrationError::invalid_redirect_uri(
"Redirect URI must not contain consecutive dots.",
));
} else if uri.starts_with("https://") || loopback_redirect_parts(uri).is_some() {
return Ok(());
} else if let Some((scheme, _)) = uri.split_once(':')
&& scheme.contains('.')
&& scheme
.as_bytes()
.first()
.is_some_and(u8::is_ascii_alphabetic)
&& scheme
.bytes()
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-' | b'+'))
{
return Ok(());
}
Err(ClientRegistrationError::invalid_redirect_uri(
"Redirect URI must be an https URL, a loopback (http://127.0.0.1/, http://[::1]/) or a private-use scheme URI.",
))
}
pub fn validate_grant_metadata(
request: &ClientRegistrationRequest,
) -> Result<(), ClientRegistrationError> {
if !request.response_types.is_empty() && !request.response_types.iter().any(|t| t == "code") {
return Err(ClientRegistrationError::invalid_client_metadata(
"response_types must include \"code\".",
));
}
if !request.grant_types.is_empty() {
if !request
.grant_types
.iter()
.any(|t| t == "authorization_code")
{
return Err(ClientRegistrationError::invalid_client_metadata(
"grant_types must include \"authorization_code\".",
));
}
if !request.grant_types.iter().any(|t| t == "refresh_token") {
return Err(ClientRegistrationError::invalid_client_metadata(
"grant_types must include \"refresh_token\".",
));
}
}
Ok(())
}
crates/common/src/auth/oauth/token.rs
+418
View File
@@ -0,0 +1,418 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use super::{GrantType, crypto::SymmetricEncrypt};
use crate::Server;
use base64::{Engine, engine::general_purpose};
use std::time::SystemTime;
use store::rand::{RngExt, rng};
use utils::codec::leb128::{Leb128Iterator, Leb128Vec};
pub const FAILED_TO_DECODE_TOKEN: &str = concat!(
"Failed to decode token. If you are using an ",
"external OIDC provider, make sure it is configured as the default directory under ",
"the Authentication object."
);
const TOKEN_HEADER: &str = "sw1.";
const TOKEN_KEY_CONTEXT: &str = "stalwart-oauth-token-sw1";
const OAUTH_EPOCH: u64 = 946684800; // Jan 1, 2000
pub struct TokenInfo {
pub grant_type: GrantType,
pub account_id: u32,
pub claims: Option<String>,
pub expiry: u64,
pub issued_at: u64,
pub expires_in: u64,
}
struct RawToken {
grant_type: GrantType,
account_id: u32,
claims: Option<String>,
issued_at: u64,
expiry: u64,
credential_version: u64,
}
impl Server {
pub async fn encode_access_token(
&self,
grant_type: GrantType,
account_id: u32,
account_name: &str,
expiry_in: u64,
claims: Option<&str>,
credential_version: Option<u64>,
) -> trc::Result<String> {
let issued_at = seconds_since_oauth_epoch();
let raw = RawToken {
grant_type,
account_id,
claims: claims.map(|claims| claims.to_string()),
issued_at,
expiry: issued_at + expiry_in,
credential_version: credential_version
.filter(|_| !matches!(grant_type, GrantType::Rsvp))
.unwrap_or_default(),
};
seal_token(
self.core.oauth.oauth_key.as_bytes(),
&raw,
account_name.as_bytes(),
)
.map_err(|err| {
trc::AuthEvent::Error
.into_err()
.ctx(trc::Key::Reason, "Failed to encrypt token")
.reason(err)
.caused_by(trc::location!())
})
}
pub async fn validate_access_token(
&self,
expected_grant_type: Option<GrantType>,
token_: &str,
) -> trc::Result<TokenInfo> {
let token = open_token(self.core.oauth.oauth_key.as_bytes(), token_).map_err(|_| {
trc::AuthEvent::Error
.into_err()
.ctx(trc::Key::Reason, FAILED_TO_DECODE_TOKEN)
.caused_by(trc::location!())
.details(token_.to_string())
})?;
// Validate expiration
let now = seconds_since_oauth_epoch();
if token.expiry <= now || token.issued_at > now {
return Err(trc::AuthEvent::TokenExpired.into_err());
}
// Validate grant type
if expected_grant_type.is_some_and(|g| g != token.grant_type) {
return Err(trc::AuthEvent::Error
.into_err()
.details("Invalid grant type"));
}
// Enforce credential revocation for long lived tokens
if token.credential_version != 0 {
let current = self
.access_token(token.account_id)
.await
.map_err(|err| trc::AuthEvent::Error.into_err().ctx(trc::Key::Details, err))?
.credential_version();
if current != token.credential_version {
return Err(trc::AuthEvent::TokenExpired
.into_err()
.details("Token revoked"));
}
}
Ok(TokenInfo {
grant_type: token.grant_type,
account_id: token.account_id,
claims: token.claims,
expiry: token.expiry + OAUTH_EPOCH,
issued_at: token.issued_at + OAUTH_EPOCH,
expires_in: token.expiry - now,
})
}
}
fn seal_token(key: &[u8], token: &RawToken, footer: &[u8]) -> Result<String, String> {
let mut payload = Vec::with_capacity(32);
payload.push_leb128(token.account_id);
payload.push(token.grant_type.id());
payload.push_leb128(token.issued_at);
payload.push_leb128(token.expiry);
payload.push_leb128(token.credential_version);
if let Some(claims) = token.claims.as_deref().filter(|claims| !claims.is_empty()) {
payload.extend_from_slice(claims.as_bytes());
}
let nonce = rng().random::<[u8; SymmetricEncrypt::NONCE_LEN]>();
let ciphertext =
SymmetricEncrypt::new(key, TOKEN_KEY_CONTEXT).encrypt_with_aad(&payload, &nonce, footer)?;
let mut body = Vec::with_capacity(nonce.len() + ciphertext.len());
body.extend_from_slice(&nonce);
body.extend_from_slice(&ciphertext);
let mut out = String::with_capacity(TOKEN_HEADER.len() + (body.len() + footer.len()) * 2);
out.push_str(TOKEN_HEADER);
general_purpose::URL_SAFE_NO_PAD.encode_string(&body, &mut out);
if !footer.is_empty() {
out.push('.');
general_purpose::URL_SAFE_NO_PAD.encode_string(footer, &mut out);
}
Ok(out)
}
fn open_token(key: &[u8], token: &str) -> Result<RawToken, ()> {
let rest = token.strip_prefix(TOKEN_HEADER).ok_or(())?;
let (body, footer) = match rest.split_once('.') {
Some((body, footer)) => (
body,
general_purpose::URL_SAFE_NO_PAD
.decode(footer.as_bytes())
.map_err(|_| ())?,
),
None => (rest, Vec::new()),
};
let body = general_purpose::URL_SAFE_NO_PAD
.decode(body.as_bytes())
.map_err(|_| ())?;
if body.len() < SymmetricEncrypt::NONCE_LEN + SymmetricEncrypt::ENCRYPT_TAG_LEN {
return Err(());
}
let (nonce, ciphertext) = body.split_at(SymmetricEncrypt::NONCE_LEN);
let payload = SymmetricEncrypt::new(key, TOKEN_KEY_CONTEXT)
.decrypt_with_aad(ciphertext, nonce, &footer)
.map_err(|_| ())?;
let mut bytes = payload.iter();
let account_id: u32 = bytes.next_leb128().ok_or(())?;
let grant_type = GrantType::from_id(bytes.next().copied().ok_or(())?).ok_or(())?;
let issued_at: u64 = bytes.next_leb128().ok_or(())?;
let expiry: u64 = bytes.next_leb128().ok_or(())?;
let credential_version: u64 = bytes.next_leb128().ok_or(())?;
let bytes = bytes.as_slice();
let claims = if bytes.is_empty() {
None
} else {
Some(String::from_utf8(bytes.to_vec()).map_err(|_| ())?)
};
Ok(RawToken {
grant_type,
account_id,
claims,
issued_at,
expiry,
credential_version,
})
}
#[inline(always)]
fn seconds_since_oauth_epoch() -> u64 {
SystemTime::now()
.duration_since(SystemTime::UNIX_EPOCH)
.map_or(0, |d| d.as_secs())
.saturating_sub(OAUTH_EPOCH)
}
#[cfg(test)]
mod tests {
use super::*;
const KEY: &[u8] = b"a-test-encryption-key-of-some-length";
const NAME: &[u8] = b"[email protected]";
fn sample(grant_type: GrantType, claims: Option<&str>, cv: u64) -> RawToken {
RawToken {
grant_type,
account_id: 42,
claims: claims.map(|c| c.to_string()),
issued_at: 1_000,
expiry: 2_000,
credential_version: cv,
}
}
fn assert_eq_fields(a: &RawToken, b: &RawToken) {
assert_eq!(a.account_id, b.account_id);
assert_eq!(a.grant_type, b.grant_type);
assert_eq!(a.claims, b.claims);
assert_eq!(a.issued_at, b.issued_at);
assert_eq!(a.expiry, b.expiry);
assert_eq!(a.credential_version, b.credential_version);
}
#[test]
fn round_trip_preserves_all_fields() {
for (raw, footer) in [
(sample(GrantType::AccessToken, None, 0), NAME),
(
sample(GrantType::RefreshToken, None, 0xdead_beef_cafe),
NAME,
),
(
sample(GrantType::Rsvp, Some("[email protected];7"), 0),
b"[email protected]",
),
(sample(GrantType::AccessToken, None, 0), b""),
(
RawToken {
account_id: u32::MAX,
credential_version: u64::MAX,
..sample(GrantType::AccessToken, Some("名前;1"), 1)
},
"名字@example.org".as_bytes(),
),
] {
let token = seal_token(KEY, &raw, footer).unwrap();
assert!(token.starts_with(TOKEN_HEADER));
let opened = open_token(KEY, &token).unwrap();
assert_eq_fields(&raw, &opened);
// The footer (account name) round-trips in clear text for proxies
if footer.is_empty() {
assert!(!token[TOKEN_HEADER.len()..].contains('.'));
} else {
let segment = token.rsplit_once('.').unwrap().1;
assert_eq!(
general_purpose::URL_SAFE_NO_PAD.decode(segment).unwrap(),
footer
);
}
}
}
#[test]
fn account_name_is_readable_in_clear_text_footer() {
let token = seal_token(
KEY,
&sample(GrantType::AccessToken, None, 0),
b"[email protected]",
)
.unwrap();
let footer = token.rsplit_once('.').unwrap().1;
let decoded = general_purpose::URL_SAFE_NO_PAD.decode(footer).unwrap();
assert_eq!(decoded, b"[email protected]");
}
#[test]
fn wrong_key_is_rejected() {
let token = seal_token(KEY, &sample(GrantType::AccessToken, None, 0), NAME).unwrap();
assert!(open_token(b"a-different-encryption-key-entirely!", &token).is_err());
}
#[test]
fn tampering_with_ciphertext_is_rejected() {
let raw = sample(GrantType::AccessToken, None, 0);
let token = seal_token(KEY, &raw, NAME).unwrap();
let (header, rest) = token.split_at(TOKEN_HEADER.len());
let (body_b64, footer) = match rest.split_once('.') {
Some((b, f)) => (b.to_string(), Some(f.to_string())),
None => (rest.to_string(), None),
};
let mut body = general_purpose::URL_SAFE_NO_PAD.decode(&body_b64).unwrap();
for idx in 0..body.len() {
let mut tampered = body.clone();
tampered[idx] ^= 0x01;
let mut rebuilt = String::from(header);
rebuilt.push_str(&general_purpose::URL_SAFE_NO_PAD.encode(&tampered));
if let Some(footer) = &footer {
rebuilt.push('.');
rebuilt.push_str(footer);
}
assert!(
open_token(KEY, &rebuilt).is_err(),
"flipping byte {idx} of the body must invalidate the token"
);
}
// Sanity: the untampered token still opens
body[0] ^= 0x00;
assert!(open_token(KEY, &token).is_ok());
}
#[test]
fn tampering_with_clear_text_footer_is_rejected() {
let raw = sample(GrantType::AccessToken, None, 0);
let token = seal_token(KEY, &raw, b"[email protected]").unwrap();
let (body, _) = token.rsplit_once('.').unwrap();
// An attacker rewrites the clear-text account name to impersonate another account
let forged_footer = general_purpose::URL_SAFE_NO_PAD.encode(b"[email protected]");
let forged = format!("{body}.{forged_footer}");
assert!(
open_token(KEY, &forged).is_err(),
"the footer is bound through the associated data and must be authenticated"
);
}
#[test]
fn swapping_footers_between_tokens_is_rejected() {
let a = seal_token(
KEY,
&sample(GrantType::AccessToken, None, 0),
b"[email protected]",
)
.unwrap();
let b = seal_token(
KEY,
&sample(GrantType::AccessToken, None, 0),
b"[email protected]",
)
.unwrap();
let a_body = a.rsplit_once('.').unwrap().0;
let b_footer = b.rsplit_once('.').unwrap().1;
let frankentoken = format!("{a_body}.{b_footer}");
assert!(open_token(KEY, &frankentoken).is_err());
}
#[test]
fn malformed_input_never_panics_and_is_rejected() {
let valid = seal_token(KEY, &sample(GrantType::AccessToken, None, 0), NAME).unwrap();
let cases = [
String::new(),
"sw1.".to_string(),
"sw1.!!!not-base64!!!".to_string(),
"sw1...".to_string(),
"wrong-prefix.".to_string(),
"sw1.AAAA".to_string(),
"sw1.AAAA.BBBB".to_string(),
valid.replace("sw1.", "sw2."),
valid[..valid.len() / 2].to_string(),
format!("sw1.{}", "A".repeat(10_000)),
"\u{0}\u{0}\u{0}".to_string(),
];
for case in cases {
assert!(open_token(KEY, &case).is_err(), "must reject {case:?}");
}
}
#[test]
fn truncating_the_body_is_rejected() {
let token = seal_token(KEY, &sample(GrantType::AccessToken, None, 0), NAME).unwrap();
let (header, rest) = token.split_at(TOKEN_HEADER.len());
let body_b64 = rest.split_once('.').map(|(b, _)| b).unwrap_or(rest);
let body = general_purpose::URL_SAFE_NO_PAD.decode(body_b64).unwrap();
for len in 0..body.len() {
let mut rebuilt = String::from(header);
rebuilt.push_str(&general_purpose::URL_SAFE_NO_PAD.encode(&body[..len]));
assert!(
open_token(KEY, &rebuilt).is_err(),
"truncation to {len} must be rejected"
);
}
}
#[test]
fn identical_input_produces_distinct_tokens() {
let raw = sample(GrantType::AccessToken, None, 7);
let a = seal_token(KEY, &raw, NAME).unwrap();
let b = seal_token(KEY, &raw, NAME).unwrap();
assert_ne!(a, b, "a random nonce must make each token unique");
assert_eq_fields(&open_token(KEY, &a).unwrap(), &open_token(KEY, &b).unwrap());
}
#[test]
fn claims_with_separators_round_trip_exactly() {
let raw = sample(GrantType::Rsvp, Some("a;b;c;[email protected];999"), 0);
let token = seal_token(KEY, &raw, b"[email protected]").unwrap();
let opened = open_token(KEY, &token).unwrap();
assert_eq!(opened.claims.as_deref(), Some("a;b;c;[email protected];999"));
}
}
crates/common/src/auth/permissions.rs
+320
View File
@@ -0,0 +1,320 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::{
Server,
auth::{AccessToken, Permissions, PermissionsGroup},
};
use ahash::AHashSet;
use registry::{
schema::{
enums::Permission,
structs::{self, Account, PermissionsList, UserRoles},
},
types::EnumImpl,
};
use trc::AddContext;
use types::id::Id;
use utils::map::vec_map::VecMap;
impl Server {
pub async fn add_role_permissions(
&self,
mut base_permissions: PermissionsGroup,
roles: impl IntoIterator<Item = u32>,
) -> trc::Result<PermissionsGroup> {
let mut role_ids = roles.into_iter().collect::<Vec<u32>>();
let mut fetched_role_ids = AHashSet::new();
while let Some(role_id) = role_ids.pop() {
if fetched_role_ids.insert(role_id) {
let role = self.role(role_id).await.caused_by(trc::location!())?;
base_permissions.union(&role.permissions);
role_ids.extend(role.id_roles.iter().copied());
}
}
Ok(base_permissions)
}
pub async fn effective_permissions(
&self,
permissions: &structs::Permissions,
role_ids: &[Id],
tenant_id: Option<u32>,
) -> trc::Result<PermissionsGroup> {
// Calculate effective permissions
let (mut permissions, roles) = match permissions {
structs::Permissions::Inherit => (PermissionsGroup::default(), role_ids),
structs::Permissions::Merge(permissions) => {
(PermissionsGroup::from(permissions), role_ids)
}
structs::Permissions::Replace(permissions) => {
(PermissionsGroup::from(permissions), &[][..])
}
};
if !roles.is_empty() {
permissions = self
.add_role_permissions(permissions, roles.iter().map(|v| v.id() as u32))
.await
.caused_by(trc::location!())?
}
Ok(permissions)
}
pub async fn can_set_permissions(
&self,
access_token: &AccessToken,
account: &Account,
) -> trc::Result<Result<(), Vec<Permission>>> {
let (permissions, role_ids, tenant_id) = match account {
Account::User(account) => (
&account.permissions,
match &account.roles {
UserRoles::User => self.core.network.security.default_role_ids_user.as_slice(),
UserRoles::Admin => {
if access_token.tenant_id().is_none() {
self.core.network.security.default_role_ids_admin.as_slice()
} else {
self.core
.network
.security
.default_role_ids_tenant
.as_slice()
}
}
UserRoles::Custom(custom_roles) => custom_roles.role_ids.as_slice(),
},
account.member_tenant_id.map(|t| t.document_id()),
),
Account::Group(account) => (
&account.permissions,
account
.roles
.role_ids()
.unwrap_or(self.core.network.security.default_role_ids_group.as_slice()),
account.member_tenant_id.map(|t| t.document_id()),
),
};
self.effective_permissions(permissions, role_ids, tenant_id)
.await
.map(|permissions| access_token.can_grant_permissions(permissions.finalize()))
}
}
impl AccessToken {
pub fn can_grant_permissions(
&self,
mut requested_permissions: Permissions,
) -> Result<(), Vec<Permission>> {
requested_permissions.difference(self.permissions_bits());
if requested_permissions.is_empty() {
Ok(())
} else {
Err(requested_permissions.build_permissions_list())
}
}
}
pub trait PermissionsListBuilder {
fn build_permissions_list(&self) -> Vec<Permission>;
}
impl PermissionsListBuilder for Permissions {
fn build_permissions_list(&self) -> Vec<Permission> {
const USIZE_BITS: usize = std::mem::size_of::<usize>() * 8;
const USIZE_MASK: u32 = USIZE_BITS as u32 - 1;
let mut permissions = Vec::new();
for (block_num, bytes) in self.inner().iter().enumerate() {
let mut bytes = *bytes;
while bytes != 0 {
let item = USIZE_MASK - bytes.leading_zeros();
bytes ^= 1 << item;
if let Some(permission) =
Permission::from_id(((block_num * USIZE_BITS) + item as usize) as u16)
{
permissions.push(permission);
}
}
}
permissions
}
}
pub struct DefaultPermissions {
pub user: Vec<Permission>,
pub group: Vec<Permission>,
pub tenant: Vec<Permission>,
pub superuser: Vec<Permission>,
}
impl PermissionsGroup {
pub fn with_merge(mut self, merge: bool) -> Self {
self.merge = merge;
self
}
pub fn union(&mut self, other: &PermissionsGroup) {
self.enabled.union(&other.enabled);
self.disabled.union(&other.disabled);
}
pub fn restrict(&mut self, other: &PermissionsGroup) {
self.enabled.intersection(&other.enabled);
self.disabled.union(&other.disabled);
}
pub fn finalize(mut self) -> Permissions {
self.enabled.difference(&self.disabled);
self.enabled
}
pub fn finalize_as_ref(&self) -> Permissions {
let mut enabled = self.enabled.clone();
enabled.difference(&self.disabled);
enabled
}
pub fn user() -> Self {
let mut permissions = PermissionsGroup::default();
for permission in DefaultPermissions::default().user {
permissions.enabled.set(permission as usize);
}
permissions
}
}
impl Default for DefaultPermissions {
fn default() -> Self {
let mut default = Self {
user: Default::default(),
group: Default::default(),
tenant: Default::default(),
superuser: Default::default(),
};
for permission_id in 0..Permission::COUNT {
let permission = Permission::from_id(permission_id as u16).unwrap();
match permission {
Permission::Authenticate
| Permission::AuthenticateWithAlias
| Permission::InteractAi => {
default.user.push(permission);
default.superuser.push(permission);
default.tenant.push(permission);
}
Permission::Impersonate
| Permission::UnlimitedRequests
| Permission::UnlimitedUploads
| Permission::LiveMetrics
| Permission::LiveTracing => {
default.superuser.push(permission);
}
Permission::FetchAnyBlob | Permission::LiveDeliveryTest => {
default.superuser.push(permission);
default.tenant.push(permission);
}
permission => {
let name = permission.as_str();
if name.starts_with("jmap")
|| name.starts_with("imap")
|| name.starts_with("pop3")
|| name.starts_with("calendar")
|| name.starts_with("email")
|| name.starts_with("dav")
|| name.starts_with("sieve")
{
default.user.push(permission);
default.group.push(permission);
} else if name.starts_with("sysMaskedEmail")
|| name.starts_with("sysArchivedItem")
|| name.starts_with("sysAccountSettings")
|| name.starts_with("sysPublicKey")
|| (name.starts_with("sysSpamTrainingSample") && !name.contains("Create"))
{
default.user.push(permission);
default.group.push(permission);
default.superuser.push(permission);
} else if name.starts_with("sysAccountPassword")
|| name.starts_with("sysApiKey")
|| name.starts_with("sysAppPassword")
{
default.user.push(permission);
default.superuser.push(permission);
} else if name.starts_with("sysDomain")
|| name.starts_with("sysDkimSignature")
|| name.starts_with("sysAcmeProvider")
|| name.starts_with("sysAccount")
|| name.starts_with("sysRole")
|| name.starts_with("sysOAuthClient")
|| name.starts_with("sysMailingList")
|| name.starts_with("sysExternalReport")
|| name.starts_with("sysDnsServer")
|| name.starts_with("sysQueuedMessage")
{
default.tenant.push(permission);
default.superuser.push(permission);
} else {
default.superuser.push(permission);
}
}
}
}
default
}
}
impl From<PermissionsList> for PermissionsGroup {
fn from(value: PermissionsList) -> Self {
Self::from(&value)
}
}
impl From<&PermissionsList> for PermissionsGroup {
fn from(value: &PermissionsList) -> Self {
PermissionsGroup {
enabled: Permissions::from_permission(value.enabled_permissions.as_slice()),
disabled: Permissions::from_permission(value.disabled_permissions.as_slice()),
merge: false,
}
}
}
impl From<&VecMap<Permission, bool>> for PermissionsGroup {
fn from(value: &VecMap<Permission, bool>) -> Self {
let mut permissions = PermissionsGroup::default();
for (permission, is_set) in value {
if *is_set {
permissions.enabled.set(*permission as usize);
} else {
permissions.disabled.set(*permission as usize);
}
}
permissions
}
}
pub trait BuildPermissions {
fn from_permission(list: &[Permission]) -> Permissions;
}
impl BuildPermissions for Permissions {
fn from_permission(list: &[Permission]) -> Permissions {
let mut permission = Permissions::default();
for p in list {
permission.set(*p as usize);
}
permission
}
}
crates/common/src/auth/rate_limit.rs
+107
View File
@@ -0,0 +1,107 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::auth::AccessToken;
use crate::network::ip_to_bytes;
use crate::network::limiter::{InFlight, LimiterResult};
use crate::{KV_RATE_LIMIT_HTTP_ANONYMOUS, KV_RATE_LIMIT_HTTP_AUTHENTICATED, Server};
use registry::schema::enums::Permission;
use std::net::IpAddr;
use trc::AddContext;
impl Server {
pub async fn is_http_authenticated_request_allowed(
&self,
access_token: &AccessToken,
addr: IpAddr,
) -> trc::Result<Option<InFlight>> {
let rate_reset = if let Some(rate) = &self.core.network.http.rate_authenticated {
if self.is_ip_allowed(addr) {
None
} else {
self.core
.storage
.memory
.is_rate_allowed(
KV_RATE_LIMIT_HTTP_AUTHENTICATED,
&access_token.account_id().to_be_bytes(),
rate,
false,
)
.await
.caused_by(trc::location!())?
.map(|reset| (reset, rate.count))
}
} else {
None
};
if let Some((reset, count)) = rate_reset {
if access_token.has_permission(Permission::UnlimitedRequests) {
Ok(None)
} else {
Err(trc::LimitEvent::TooManyRequests
.into_err()
.ctx(trc::Key::Expires, reset)
.ctx(trc::Key::Limit, count))
}
} else {
match access_token.is_http_request_allowed() {
LimiterResult::Allowed(in_flight) => Ok(Some(in_flight)),
LimiterResult::Forbidden => {
if access_token.has_permission(Permission::UnlimitedRequests) {
Ok(None)
} else {
Err(trc::LimitEvent::ConcurrentRequest
.into_err()
.ctx(trc::Key::Limit, access_token.concurrent_http_requests()))
}
}
LimiterResult::Disabled => Ok(None),
}
}
}
pub async fn is_http_anonymous_request_allowed(&self, addr: IpAddr) -> trc::Result<()> {
if let Some(rate) = &self.core.network.http.rate_anonymous
&& !self.is_ip_allowed(addr)
&& let Some(reset) = self
.core
.storage
.memory
.is_rate_allowed(
KV_RATE_LIMIT_HTTP_ANONYMOUS,
&ip_to_bytes(&addr),
rate,
false,
)
.await
.caused_by(trc::location!())?
{
return Err(trc::LimitEvent::TooManyRequests
.into_err()
.ctx(trc::Key::Expires, reset)
.ctx(trc::Key::Limit, rate.count));
}
Ok(())
}
pub fn is_upload_allowed(&self, access_token: &AccessToken) -> trc::Result<Option<InFlight>> {
match access_token.is_upload_allowed() {
LimiterResult::Allowed(in_flight) => Ok(Some(in_flight)),
LimiterResult::Forbidden => {
if access_token.has_permission(Permission::UnlimitedRequests) {
Ok(None)
} else {
Err(trc::LimitEvent::ConcurrentUpload
.into_err()
.ctx(trc::Key::Limit, access_token.concurrent_uploads()))
}
}
LimiterResult::Disabled => Ok(None),
}
}
}
crates/common/src/cache/directory.rs
Vendored
+435
View File
@@ -0,0 +1,435 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::{Server, auth::DomainCache, cache::invalidate::CacheInvalidationBuilder};
use registry::{
schema::{
prelude::{Object, ObjectType},
structs::{
Account, Credential, EmailAlias, GroupAccount, PasswordCredential, Roles, UserAccount,
UserRoles,
},
},
types::{datetime::UTCDateTime, id::ObjectId, list::List},
};
use std::sync::Arc;
use store::registry::write::{RegistryWrite, RegistryWriteResult};
use trc::AddContext;
use types::id::Id;
pub struct AccountWithId {
pub id: u32,
pub account: Account,
}
impl Server {
pub async fn synchronize_account(
&self,
account: directory::Account,
) -> trc::Result<AccountWithId> {
let (local, domain) = self.validate_address(&account.email).await?;
match self
.account_id_from_parts(local, domain.id)
.await
.caused_by(trc::location!())?
{
Some(account_id) => {
let current_account = self
.registry()
.get(ObjectId::new(ObjectType::Account, account_id.into()))
.await
.caused_by(trc::location!())?
.ok_or_else(|| {
trc::AuthEvent::Error
.into_err()
.details("Account ID from directory does not exist in registry")
.ctx(trc::Key::AccountName, account.email.clone())
.ctx(trc::Key::AccountId, account_id)
})?;
let mut updated_account = Account::from(current_account.clone())
.into_user()
.ok_or_else(|| {
trc::AuthEvent::Error
.into_err()
.details(
"Account ID from directory does not correspond to a user account",
)
.ctx(trc::Key::AccountName, account.email.clone())
.ctx(trc::Key::AccountId, account_id)
})?;
let mut has_changes = false;
if let Some(secret) = account.secret
&& secret != updated_account.password().unwrap_or_default()
{
has_changes = true;
updated_account.set_password(secret);
}
if account.description.is_some()
&& account.description != updated_account.description
{
updated_account.description = account.description;
has_changes = true;
}
for alias in account.email_aliases {
if let Some((local, alias_domain)) = self.validate_alias(&alias).await?
&& alias_domain.id_tenant == domain.id_tenant
&& self
.rcpt_id_from_parts(local, alias_domain.id)
.await?
.is_none()
{
updated_account.aliases.push(EmailAlias {
name: local.to_string(),
domain_id: Id::from(alias_domain.id),
enabled: true,
description: None,
});
has_changes = true;
}
}
if let Some(groups) = account.groups {
let mut member_group_ids = Vec::with_capacity(groups.len());
for email in groups {
member_group_ids.push(
self.synchronize_group(directory::Group {
email,
..Default::default()
})
.await
.caused_by(trc::location!())?
.into(),
);
}
if updated_account.member_group_ids.len() != member_group_ids.len()
|| !updated_account
.member_group_ids
.iter()
.all(|id| member_group_ids.contains(id))
{
updated_account.member_group_ids = member_group_ids.into();
has_changes = true;
}
}
if has_changes {
let updated_account = Object::from(Account::User(updated_account));
match self
.registry()
.write(RegistryWrite::update(
Id::from(account_id),
&updated_account,
&current_account,
))
.await
.caused_by(trc::location!())?
{
RegistryWriteResult::Success(id) => {
let mut invalidator = CacheInvalidationBuilder::default();
invalidator.process_update(id, &current_account, &updated_account);
self.invalidate_caches(invalidator)
.await
.caused_by(trc::location!())?;
Ok(AccountWithId {
id: id.document_id(),
account: updated_account.into(),
})
}
failure => Err(trc::AuthEvent::Error
.into_err()
.caused_by(trc::location!())
.details("Failed to synchronize account with directory")
.reason(failure)),
}
} else {
Ok(AccountWithId {
id: account_id,
account: Account::User(updated_account),
})
}
}
None => {
let mut aliases = Vec::with_capacity(account.email_aliases.len());
for alias in account.email_aliases {
if let Some((local, alias_domain)) = self.validate_alias(&alias).await?
&& alias_domain.id_tenant == domain.id_tenant
&& self
.rcpt_id_from_parts(local, alias_domain.id)
.await?
.is_none()
{
aliases.push(EmailAlias {
name: local.to_string(),
domain_id: Id::from(alias_domain.id),
enabled: true,
description: None,
});
}
}
let mut member_group_ids = Vec::new();
for email in account.groups.unwrap_or_default() {
member_group_ids.push(
self.synchronize_group(directory::Group {
email,
..Default::default()
})
.await
.caused_by(trc::location!())?
.into(),
);
}
let account = Object::from(Account::User(UserAccount {
name: local.to_string(),
domain_id: Id::from(domain.id),
aliases: aliases.into(),
created_at: UTCDateTime::now(),
description: account.description,
member_group_ids: member_group_ids.into(),
member_tenant_id: domain.id_tenant.map(Id::from),
roles: UserRoles::User,
credentials: List::from_iter(account.secret.map(|secret| {
Credential::Password(PasswordCredential {
credential_id: 0u64.into(),
secret,
..Default::default()
})
})),
..Default::default()
}));
match self
.registry()
.write(RegistryWrite::insert(&account))
.await
.caused_by(trc::location!())?
{
RegistryWriteResult::Success(id) => {
let mut invalidator = CacheInvalidationBuilder::default();
invalidator.process_create(&account);
self.invalidate_caches(invalidator)
.await
.caused_by(trc::location!())?;
Ok(AccountWithId {
id: id.document_id(),
account: account.into(),
})
}
failure => Err(trc::AuthEvent::Error
.into_err()
.caused_by(trc::location!())
.details("Failed to create account from directory")
.reason(failure)),
}
}
}
}
pub async fn synchronize_group(&self, group: directory::Group) -> trc::Result<u32> {
let (local, domain) = self.validate_address(&group.email).await?;
match self
.account_id_from_parts(local, domain.id)
.await
.caused_by(trc::location!())?
{
Some(account_id) => {
let current_account = self
.registry()
.get(ObjectId::new(ObjectType::Account, account_id.into()))
.await
.caused_by(trc::location!())?
.ok_or_else(|| {
trc::AuthEvent::Error
.into_err()
.details("Account ID from directory does not exist in registry")
.ctx(trc::Key::AccountName, group.email.clone())
.ctx(trc::Key::AccountId, account_id)
})?;
let mut updated_account = Account::from(current_account.clone())
.into_group()
.ok_or_else(|| {
trc::AuthEvent::Error
.into_err()
.details(
"Account ID from directory does not correspond to a group account",
)
.ctx(trc::Key::AccountName, group.email.clone())
.ctx(trc::Key::AccountId, account_id)
})?;
let mut has_changes = false;
if group.description.is_some() && group.description != updated_account.description {
updated_account.description = group.description;
has_changes = true;
}
for alias in group.email_aliases {
if let Some((local, alias_domain)) = self.validate_alias(&alias).await?
&& alias_domain.id_tenant == domain.id_tenant
&& self
.rcpt_id_from_parts(local, alias_domain.id)
.await?
.is_none()
{
updated_account.aliases.push(EmailAlias {
name: local.to_string(),
domain_id: Id::from(alias_domain.id),
enabled: true,
description: None,
});
has_changes = true;
}
}
if has_changes {
let updated_account = Object::from(Account::Group(updated_account));
match self
.registry()
.write(RegistryWrite::update(
Id::from(account_id),
&updated_account,
&current_account,
))
.await
.caused_by(trc::location!())?
{
RegistryWriteResult::Success(id) => {
let mut invalidator = CacheInvalidationBuilder::default();
invalidator.process_update(id, &current_account, &updated_account);
self.invalidate_caches(invalidator)
.await
.caused_by(trc::location!())?;
Ok(id.document_id())
}
failure => Err(trc::AuthEvent::Error
.into_err()
.caused_by(trc::location!())
.details("Failed to synchronize account with directory")
.reason(failure)),
}
} else {
Ok(account_id)
}
}
None => {
let mut aliases = Vec::with_capacity(group.email_aliases.len());
for alias in group.email_aliases {
if let Some((local, alias_domain)) = self.validate_alias(&alias).await?
&& alias_domain.id_tenant == domain.id_tenant
&& self
.rcpt_id_from_parts(local, alias_domain.id)
.await?
.is_none()
{
aliases.push(EmailAlias {
name: local.to_string(),
domain_id: Id::from(alias_domain.id),
enabled: true,
description: None,
});
}
}
let account = Object::from(Account::Group(GroupAccount {
name: local.to_string(),
domain_id: Id::from(domain.id),
aliases: aliases.into(),
created_at: UTCDateTime::now(),
description: group.description,
member_tenant_id: domain.id_tenant.map(Id::from),
roles: Roles::Default,
..Default::default()
}));
match self
.registry()
.write(RegistryWrite::insert(&account))
.await
.caused_by(trc::location!())?
{
RegistryWriteResult::Success(id) => {
let mut invalidator = CacheInvalidationBuilder::default();
invalidator.process_create(&account);
self.invalidate_caches(invalidator)
.await
.caused_by(trc::location!())?;
Ok(id.document_id())
}
failure => Err(trc::AuthEvent::Error
.into_err()
.caused_by(trc::location!())
.details("Failed to create account from directory")
.reason(failure)),
}
}
}
}
async fn validate_address<'x>(
&self,
email: &'x str,
) -> trc::Result<(&'x str, Arc<DomainCache>)> {
if email.is_empty() {
return Err(trc::AuthEvent::Error
.into_err()
.details("Account email cannot be empty"));
}
match email.rsplit_once('@') {
Some((local, domain)) => self
.domain(domain)
.await
.caused_by(trc::location!())?
.map(|domain| (local, domain))
.ok_or_else(|| {
trc::AuthEvent::Error
.into_err()
.details("Account domain does not exist or has been disabled")
.ctx(trc::Key::Domain, domain.to_string())
}),
None => {
trc::event!(
Auth(trc::AuthEvent::Warning),
AccountName = email.to_string().clone(),
Details = "Directory account is not an email, appended default domain",
);
self.domain_by_id(self.core.email.default_domain_id)
.await
.caused_by(trc::location!())?
.ok_or_else(|| {
trc::AuthEvent::Error
.into_err()
.details("Default domain does not exist or has been disabled")
.ctx(trc::Key::Id, self.core.email.default_domain_id)
})
.map(|domain| (email, domain))
}
}
}
async fn validate_alias<'x>(
&self,
email: &'x str,
) -> trc::Result<Option<(&'x str, Arc<DomainCache>)>> {
match email.rsplit_once('@') {
Some((local, domain)) => self
.domain(domain)
.await
.caused_by(trc::location!())
.map(|domain| domain.map(|domain| (local, domain))),
None => Ok(None),
}
}
}
crates/common/src/cache/invalidate.rs
Vendored
+440
View File
@@ -0,0 +1,440 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::{
Server,
auth::{EmailAddressRef, EmailCache},
ipc::{BroadcastEvent, CacheInvalidation},
};
use ahash::AHashSet;
use registry::{
schema::{
prelude::{Object, ObjectInner, ObjectType},
structs::{Account, EmailAlias},
},
types::id::ObjectId,
};
use store::{registry::RegistryQuery, roaring::RoaringBitmap};
use types::id::Id;
#[derive(Debug, Default)]
pub struct CacheInvalidationBuilder {
changes: AHashSet<CacheInvalidation>,
}
impl CacheInvalidationBuilder {
pub fn process_update(&mut self, id: Id, current_object: &Object, new_object: &Object) {
let id = id.document_id();
match (&current_object.inner, &new_object.inner) {
(
ObjectInner::Account(Account::User(current)),
ObjectInner::Account(Account::User(new)),
) => {
let was_renamed =
(current.name != new.name) || (current.domain_id != new.domain_id);
let quota_changed = current.quotas != new.quotas;
let permissions_changed = current.permissions != new.permissions;
let roles_changed = current.roles != new.roles;
let tenant_changed = current.member_tenant_id != new.member_tenant_id;
let details_changed =
current.locale != new.locale || current.description != new.description;
let groups_changed = current.member_group_ids != new.member_group_ids;
let aliases_changed = current.aliases != new.aliases;
let credentials_changed = current.credentials != new.credentials;
let encryption_changed = current.encryption_at_rest != new.encryption_at_rest;
if was_renamed
|| aliases_changed
|| tenant_changed
|| groups_changed
|| quota_changed
|| details_changed
|| encryption_changed
{
self.invalidate(CacheInvalidation::Account(id));
}
if was_renamed || aliases_changed {
self.invalidate_negative_email(&new_object.inner);
}
if tenant_changed
|| groups_changed
|| credentials_changed
|| roles_changed
|| permissions_changed
{
self.invalidate(CacheInvalidation::AccessToken(id));
}
if was_renamed {
self.invalidate(CacheInvalidation::DavResources(id));
}
}
(
ObjectInner::Account(Account::Group(current)),
ObjectInner::Account(Account::Group(new)),
) => {
let was_renamed =
(current.name != new.name) || (current.domain_id != new.domain_id);
let quota_changed = current.quotas != new.quotas;
let permissions_changed = current.permissions != new.permissions;
let roles_changed = current.roles != new.roles;
let tenant_changed = current.member_tenant_id != new.member_tenant_id;
let details_changed =
current.locale != new.locale || current.description != new.description;
let aliases_changed = current.aliases != new.aliases;
if was_renamed
|| aliases_changed
|| tenant_changed
|| quota_changed
|| details_changed
{
self.invalidate(CacheInvalidation::Account(id));
}
if was_renamed || aliases_changed {
self.invalidate_negative_email(&new_object.inner);
}
if tenant_changed || roles_changed || permissions_changed {
self.invalidate(CacheInvalidation::AccessToken(id));
}
if was_renamed {
self.invalidate(CacheInvalidation::DavResources(id));
}
}
(ObjectInner::Domain(current), ObjectInner::Domain(new)) => {
if (current.name != new.name)
|| (current.aliases != new.aliases)
|| (current.directory_id != new.directory_id)
|| (current.member_tenant_id != new.member_tenant_id)
|| (current.catch_all_address != new.catch_all_address)
|| (current.sub_addressing != new.sub_addressing)
|| (current.allow_relaying != new.allow_relaying)
|| (current.is_enabled != new.is_enabled)
{
self.invalidate(CacheInvalidation::Domain(id));
}
if (current.name != new.name) || (current.aliases != new.aliases) {
self.invalidate(CacheInvalidation::DomainNegative);
}
if current.logo != new.logo {
self.invalidate(CacheInvalidation::DomainLogo(id));
}
}
(ObjectInner::DkimSignature(current), ObjectInner::DkimSignature(new)) => {
let current_domain_id = current.domain_id().document_id();
let new_domain_id = new.domain_id().document_id();
self.invalidate(CacheInvalidation::DkimSignature(current_domain_id));
if current_domain_id != new_domain_id {
self.invalidate(CacheInvalidation::DkimSignature(new_domain_id));
}
}
(ObjectInner::Tenant(current), ObjectInner::Tenant(new)) => {
if (current.permissions != new.permissions)
|| (current.roles != new.roles)
|| (current.quotas != new.quotas)
{
self.invalidate(CacheInvalidation::Tenant(id));
}
if current.logo != new.logo {
self.invalidate(CacheInvalidation::TenantLogo(id));
}
}
(ObjectInner::Role(current), ObjectInner::Role(new))
if (current.enabled_permissions != new.enabled_permissions)
|| (current.disabled_permissions != new.disabled_permissions)
|| (current.member_tenant_id != new.member_tenant_id)
|| (current.role_ids != new.role_ids) =>
{
self.invalidate(CacheInvalidation::Role(id));
}
(ObjectInner::MailingList(current), ObjectInner::MailingList(new))
if (current.aliases != new.aliases)
|| (current.name != new.name)
|| (current.recipients != new.recipients)
|| (current.domain_id != new.domain_id) =>
{
self.invalidate(CacheInvalidation::List(id));
if (current.aliases != new.aliases)
|| (current.name != new.name)
|| (current.domain_id != new.domain_id)
{
self.invalidate_negative_email(&new_object.inner);
}
}
_ => {}
}
}
pub fn process_delete(&mut self, id: Id, object: &Object) {
let id = id.document_id();
match &object.inner {
ObjectInner::Account(_) => {
self.invalidate(CacheInvalidation::AccessToken(id));
self.invalidate(CacheInvalidation::Account(id));
self.invalidate(CacheInvalidation::DavResources(id));
}
ObjectInner::Domain(_) => {
self.invalidate(CacheInvalidation::Domain(id));
self.invalidate(CacheInvalidation::DomainLogo(id));
}
ObjectInner::DkimSignature(object) => {
self.invalidate(CacheInvalidation::DkimSignature(
object.domain_id().document_id(),
));
}
ObjectInner::Tenant(_) => {
self.invalidate(CacheInvalidation::Tenant(id));
self.invalidate(CacheInvalidation::TenantLogo(id));
}
ObjectInner::Role(_) => {
self.invalidate(CacheInvalidation::Role(id));
}
ObjectInner::MailingList(_) => {
self.invalidate(CacheInvalidation::List(id));
}
_ => {}
}
}
pub fn process_create(&mut self, object: &Object) {
if matches!(&object.inner, ObjectInner::Domain(_)) {
self.invalidate(CacheInvalidation::DomainNegative);
}
self.invalidate_negative_email(&object.inner);
}
fn invalidate_negative_email(&mut self, object: &ObjectInner) {
let (name, domain_id, aliases) = match object {
ObjectInner::Account(Account::User(account)) => {
(&account.name, account.domain_id, &account.aliases)
}
ObjectInner::Account(Account::Group(account)) => {
(&account.name, account.domain_id, &account.aliases)
}
ObjectInner::MailingList(list) => (&list.name, list.domain_id, &list.aliases),
_ => return,
};
self.invalidate(CacheInvalidation::EmailNegative {
domain_id: domain_id.document_id(),
local_part_hash: hash_local_part(name),
});
for alias in aliases.iter().filter(|alias: &&EmailAlias| alias.enabled) {
self.invalidate(CacheInvalidation::EmailNegative {
domain_id: alias.domain_id.document_id(),
local_part_hash: hash_local_part(&alias.name),
});
}
}
pub fn invalidate(&mut self, change: CacheInvalidation) {
self.changes.insert(change);
}
pub fn with_invalidation(mut self, change: CacheInvalidation) -> Self {
self.invalidate(change);
self
}
}
impl Server {
pub async fn invalidate_caches(&self, changes: CacheInvalidationBuilder) -> trc::Result<()> {
let mut changes = changes.changes;
if changes.is_empty() {
return Ok(());
}
// Invalidate objects linking roles
let mut role_ids = changes
.iter()
.filter_map(|change| {
if let CacheInvalidation::Role(role_id) = change {
Some(*role_id)
} else {
None
}
})
.collect::<Vec<_>>();
if !role_ids.is_empty() {
let mut fetched_role_ids = AHashSet::new();
while let Some(role_id) = role_ids.pop() {
if fetched_role_ids.insert(role_id) {
let linked_objects = self
.registry()
.linked_objects(ObjectId::new(ObjectType::Role, role_id.into()))
.await?;
for linked_object in linked_objects {
match linked_object.object() {
ObjectType::Account => {
changes.insert(CacheInvalidation::AccessToken(
linked_object.id().document_id(),
));
}
ObjectType::Role => {
role_ids.push(linked_object.id().document_id());
}
_ => {}
}
}
}
}
}
let changes = changes.into_iter().collect::<Vec<_>>();
self.invalidate_local_caches(&changes).await;
self.cluster_broadcast(BroadcastEvent::CacheInvalidate(changes))
.await;
Ok(())
}
pub fn invalidate_all_local_caches(&self) {
self.invalidate_all_local_negative_caches();
self.inner.cache.access_tokens.clear();
self.inner.cache.domains.clear();
self.inner.cache.domain_names.clear();
self.inner.cache.emails.clear();
self.inner.cache.tenants.clear();
self.inner.cache.files.clear();
self.inner.cache.contacts.clear();
self.inner.cache.events.clear();
self.inner.cache.scheduling.clear();
self.inner.cache.dkim_signers.clear();
self.inner.cache.accounts.clear();
self.inner.cache.roles.clear();
self.inner.cache.lists.clear();
self.inner.data.logos.lock().clear();
}
pub fn invalidate_all_local_negative_caches(&self) {
self.inner.cache.domain_names_negative.clear();
self.inner.cache.emails_negative.clear();
}
pub fn invalidate_local_negative_account_cache(
&self,
local_part: &str,
domain_id: u32,
) -> bool {
self.inner
.cache
.emails_negative
.remove(&EmailAddressRef::new(local_part, domain_id))
.is_some()
}
pub async fn invalidate_local_caches(&self, changes: &[CacheInvalidation]) {
let cache = &self.inner.cache;
let mut negative_emails: AHashSet<(u32, u32)> = AHashSet::new();
for change in changes {
match change {
CacheInvalidation::AccessToken(id) => {
cache.access_tokens.remove(id);
cache.http_auth.inner().retain(|_, v| v.account_id != *id);
}
CacheInvalidation::DavResources(id) => {
cache.files.remove(id);
cache.contacts.remove(id);
cache.events.remove(id);
cache.scheduling.remove(id);
}
CacheInvalidation::Domain(id) => {
cache.domains.remove(id);
cache.dkim_signers.remove(id);
cache.domain_names.inner().retain(|_, v| v != id);
}
CacheInvalidation::Account(id) => {
cache.accounts.remove(id);
cache.emails.inner().retain(|_, v| {
!matches!(
v,
EmailCache::Account(account_id)
| EmailCache::DisabledAccountAddress(account_id)
if account_id == id
)
});
}
CacheInvalidation::DkimSignature(id) => {
cache.dkim_signers.remove(id);
}
CacheInvalidation::Tenant(id) => {
cache.tenants.remove(id);
}
CacheInvalidation::Role(id) => {
cache.roles.remove(id);
}
CacheInvalidation::List(id) => {
cache.lists.remove(id);
cache.emails.inner().retain(|_, v| {
!matches!(
v,
EmailCache::MailingList(list_id)
| EmailCache::DisabledListAddress(list_id)
if list_id == id
)
});
}
CacheInvalidation::DomainLogo(id) => {
self.inner
.data
.logos
.lock()
.retain(|_, v| v.domain_id != *id);
}
CacheInvalidation::TenantLogo(id) => {
self.inner
.data
.logos
.lock()
.retain(|_, v| v.tenant_id != Some(*id));
}
CacheInvalidation::EmailNegative {
domain_id,
local_part_hash,
} => {
negative_emails.insert((*domain_id, *local_part_hash));
}
CacheInvalidation::DomainNegative => {
cache.domain_names_negative.clear();
}
}
}
if !negative_emails.is_empty() {
cache.emails_negative.retain(|key| {
!negative_emails.contains(&(key.domain_id, hash_local_part(&key.local_part)))
});
}
}
}
#[inline(always)]
fn hash_local_part(local_part: &str) -> u32 {
xxhash_rust::xxh3::xxh3_64(local_part.as_bytes()) as u32
}
impl From<CacheInvalidation> for CacheInvalidationBuilder {
fn from(invalidation: CacheInvalidation) -> Self {
let mut builder = CacheInvalidationBuilder::default();
builder.invalidate(invalidation);
builder
}
}
crates/common/src/cache/mod.rs
Vendored
+94
View File
@@ -0,0 +1,94 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::{DavResources, HttpAuthCache, MailboxCache, MessageStoreCache, UpdateLock};
use std::sync::atomic::{AtomicU64, Ordering};
use tokio::sync::{Semaphore, SemaphorePermit};
use utils::cache::CacheItemWeight;
pub mod directory;
pub mod invalidate;
pub mod principals;
pub mod reload;
impl MailboxCache {
pub fn parent_id(&self) -> Option<u32> {
if self.parent_id != u32::MAX {
Some(self.parent_id)
} else {
None
}
}
pub fn sort_order(&self) -> Option<u32> {
if self.sort_order != u32::MAX {
Some(self.sort_order)
} else {
None
}
}
pub fn is_root(&self) -> bool {
self.parent_id == u32::MAX
}
}
pub enum LockResult<'x> {
Acquired(SemaphorePermit<'x>),
Stale(SemaphorePermit<'x>),
}
impl UpdateLock {
pub fn new() -> Self {
Self {
semaphore: Semaphore::new(1),
revision: AtomicU64::new(0),
}
}
pub async fn acquire(&self, current_revision: u64) -> trc::Result<LockResult<'_>> {
let permit = self.semaphore.acquire().await.map_err(|err| {
trc::EventType::Server(trc::ServerEvent::ThreadError)
.reason(err)
.caused_by(trc::location!())
.details("Failed to acquire semaphore permit")
})?;
if self.revision.load(Ordering::Acquire) == current_revision {
Ok(LockResult::Acquired(permit))
} else {
Ok(LockResult::Stale(permit))
}
}
pub fn set_revision(&self, revision: u64) {
self.revision.store(revision, Ordering::Release);
}
}
impl Default for UpdateLock {
fn default() -> Self {
Self::new()
}
}
impl CacheItemWeight for MessageStoreCache {
fn weight(&self) -> u64 {
self.size
}
}
impl CacheItemWeight for HttpAuthCache {
fn weight(&self) -> u64 {
std::mem::size_of::<HttpAuthCache>() as u64
}
}
impl CacheItemWeight for DavResources {
fn weight(&self) -> u64 {
self.size
}
}
crates/common/src/cache/principals.rs
Vendored
+1009
View File
File diff suppressed because it is too large Load Diff
crates/common/src/cache/reload.rs
Vendored
+182
View File
@@ -0,0 +1,182 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::{
Core, Server,
config::{
server::{Listeners, tls::parse_certificates},
storage::Storage,
telemetry::Telemetry,
},
ipc::{QueueEvent, RegistryChange},
network::security::{BlockedIps, IpWithTtl},
};
use ahash::AHashMap;
use directory::Directories;
use registry::{
schema::{prelude::ObjectType, structs::BlockedIp},
types::error::{Error, Warning},
};
use std::sync::Arc;
use store::{LookupStores, registry::bootstrap::Bootstrap, write::now};
pub struct ReloadResult {
pub errors: Vec<Error>,
pub warnings: Vec<Warning>,
pub replaced_core: bool,
}
impl Server {
pub async fn reload_registry(&self, change: RegistryChange) -> trc::Result<ReloadResult> {
let mut bootstrap = Bootstrap::new(self.registry().clone()).await;
let object = match change {
RegistryChange::Insert(id) => {
if matches!(id.object(), ObjectType::BlockedIp) {
if let Some(ip) = bootstrap.get_infallible::<BlockedIp>(id.id()).await {
let expires_at = ip
.expires_at
.as_ref()
.map(|dt| dt.timestamp() as u64)
.unwrap_or(u64::MAX);
if expires_at > now() {
let mut ips = self.inner.data.blocked_ips.write();
if let Some(ip) = ip.address.try_to_ip() {
ips.blocked_ip_addresses
.insert(IpWithTtl::new(ip, expires_at));
} else {
ips.blocked_ip_networks
.push(IpWithTtl::new(ip.address, expires_at));
}
}
}
return Ok(bootstrap.into());
} else {
id.object()
}
}
RegistryChange::Delete(id) => id.object(),
RegistryChange::Reload(object) => object,
};
match object {
ObjectType::Certificate => {
let mut certificates = AHashMap::new();
parse_certificates(&mut bootstrap, &mut certificates, &mut Default::default())
.await;
self.inner
.data
.tls_certificates
.store(Arc::new(certificates));
}
ObjectType::MemoryLookupKey
| ObjectType::MemoryLookupKeyValue
| ObjectType::HttpLookup
| ObjectType::StoreLookup => {
let lookup = LookupStores::build(&mut bootstrap).await;
if bootstrap.errors.is_empty() {
self.inner.data.lookup_stores.store(Arc::new(lookup.stores));
}
}
ObjectType::BlockedIp => {
let blocked_ips = BlockedIps::parse(&mut bootstrap).await;
if bootstrap.errors.is_empty() {
*self.inner.data.blocked_ips.write() = blocked_ips;
}
}
ObjectType::Application => {
self.inner.data.applications.reload(&mut bootstrap).await;
if bootstrap.errors.is_empty() {
self.inner.data.applications.unpack_all(self, false).await;
}
}
_ => {
// Load stores
let directory = Directories::build(&mut bootstrap).await;
let storage = &self.core.storage;
let storage = Storage {
registry: storage.registry.clone(),
data: storage.data.clone(),
blob: storage.blob.clone(),
search: storage.search.clone(),
metrics: storage.metrics.clone(),
tracing: storage.tracing.clone(),
memory: storage.memory.clone(),
coordinator: storage.coordinator.clone(),
directory: directory.default_directory,
directories: directory.directories,
};
// Parse tracers
let tracers = Telemetry::parse(&mut bootstrap, &storage).await;
if bootstrap.errors.is_empty() {
let core = Box::pin(Core::parse(&mut bootstrap, storage)).await;
if bootstrap.errors.is_empty() {
let mut servers = Listeners::parse(&mut bootstrap).await;
servers
.parse_tcp_acceptors(&mut bootstrap, self.inner.clone())
.await;
if bootstrap.errors.is_empty() {
// Update core
self.inner.shared_core.store(core.into());
// Update tracers
#[cfg(not(feature = "enterprise"))]
tracers.update(false);
// Reload queue settings
self.inner
.ipc
.queue_tx
.send(QueueEvent::ReloadSettings)
.await
.ok();
return Ok(ReloadResult {
errors: bootstrap.errors,
warnings: bootstrap.warnings,
replaced_core: true,
});
}
}
}
}
}
Ok(bootstrap.into())
}
}
impl ReloadResult {
pub fn has_errors(&self) -> bool {
!self.errors.is_empty()
}
pub fn log(&self) {
for error in &self.errors {
error.log();
}
for warning in &self.warnings {
warning.log();
}
}
}
impl From<Bootstrap> for ReloadResult {
fn from(bootstrap: Bootstrap) -> Self {
Self {
errors: bootstrap.errors,
warnings: bootstrap.warnings,
replaced_core: false,
}
}
}
crates/common/src/config/groupware.rs
+283
View File
@@ -0,0 +1,283 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use calcard::vcard::VCardVersion;
use registry::schema::{
enums::VCardVersion as RegistryVCardVersion,
structs::{
AddressBook, Calendar, CalendarAlarm, CalendarScheduling, DataRetention, FileStorage,
Sharing, SystemSettings, WebDav,
},
};
use std::str::FromStr;
use store::registry::bootstrap::Bootstrap;
use utils::template::Template;
#[derive(Debug, Clone, Default)]
pub struct GroupwareConfig {
// DAV settings
pub max_request_size: usize,
pub dead_property_size: Option<usize>,
pub live_property_size: usize,
pub max_lock_timeout: u64,
pub max_locks_per_user: usize,
pub max_results: usize,
pub assisted_discovery: bool,
// Calendar settings
pub max_ical_size: usize,
pub max_ical_instances: usize,
pub max_ical_attendees_per_instance: usize,
pub default_calendar_name: Option<String>,
pub default_calendar_display_name: Option<String>,
pub alarms_enabled: bool,
pub alarms_minimum_interval: i64,
pub alarms_allow_external_recipients: bool,
pub alarms_from_name: String,
pub alarms_from_email: Option<String>,
pub alarms_template: Template<CalendarTemplateVariable>,
pub itip_enabled: bool,
pub itip_auto_add: bool,
pub itip_inbound_max_ical_size: usize,
pub itip_outbound_max_recipients: usize,
pub itip_http_rsvp_url: Option<String>,
pub itip_http_rsvp_expiration: u64,
pub itip_inbox_auto_expunge: Option<u64>,
pub itip_template: Template<CalendarTemplateVariable>,
// Addressbook settings
pub max_vcard_size: usize,
pub vcard_version: VCardVersion,
pub default_addressbook_name: Option<String>,
pub default_addressbook_display_name: Option<String>,
// File storage settings
pub max_file_size: usize,
// Sharing settings
pub max_shares_per_item: usize,
pub allow_directory_query: bool,
}
#[derive(Debug, Clone, PartialEq, Eq, Default, Hash)]
pub enum CalendarTemplateVariable {
#[default]
PageTitle,
Lang,
Dir,
Header,
Footer,
EventTitle,
EventDescription,
EventDetails,
Actions,
ActionUrl,
ActionName,
AttendeesTitle,
Attendees,
Key,
Color,
Changed,
Value,
Link,
LogoCid,
OldValue,
Rsvp,
}
impl GroupwareConfig {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let calendar = bp.setting_infallible::<Calendar>().await;
let alarm = bp.setting_infallible::<CalendarAlarm>().await;
let sched = bp.setting_infallible::<CalendarScheduling>().await;
let book = bp.setting_infallible::<AddressBook>().await;
let dav = bp.setting_infallible::<WebDav>().await;
let file = bp.setting_infallible::<FileStorage>().await;
let share = bp.setting_infallible::<Sharing>().await;
let dr = bp.setting_infallible::<DataRetention>().await;
let system = bp.setting_infallible::<SystemSettings>().await;
GroupwareConfig {
max_request_size: dav.request_max_size as usize,
dead_property_size: dav.dead_property_max_size.map(|v| v as usize),
live_property_size: dav.live_property_max_size as usize,
assisted_discovery: dav.enable_assisted_discovery,
max_lock_timeout: dav.max_lock_timeout.into_inner().as_secs(),
max_locks_per_user: dav.max_locks as usize,
max_results: dav.max_results as usize,
default_calendar_name: calendar.default_href_name,
default_calendar_display_name: calendar.default_display_name,
default_addressbook_name: book.default_href_name,
default_addressbook_display_name: book.default_display_name,
max_ical_size: calendar.max_i_calendar_size as usize,
max_ical_instances: calendar.max_recurrence_expansions as usize,
max_ical_attendees_per_instance: calendar.max_attendees as usize,
max_vcard_size: book.max_v_card_size as usize,
vcard_version: match book.v_card_version {
RegistryVCardVersion::V3 => VCardVersion::V3_0,
RegistryVCardVersion::V4 => VCardVersion::V4_0,
},
max_file_size: file.max_size as usize,
alarms_enabled: alarm.enable,
alarms_minimum_interval: alarm.min_trigger_interval.into_inner().as_secs() as i64,
alarms_allow_external_recipients: alarm.allow_external_rcpts,
alarms_from_name: alarm.from_name,
alarms_from_email: alarm.from_email,
alarms_template: Template::parse(include_str!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../resources/html-templates/calendar-alarm.html.min"
)))
.expect("Failed to parse calendar template"),
itip_enabled: sched.enable,
itip_auto_add: sched.auto_add_invitations,
itip_inbound_max_ical_size: sched.itip_max_size as usize,
itip_outbound_max_recipients: sched.max_recipients as usize,
itip_inbox_auto_expunge: dr
.expunge_scheduling_inbox_after
.map(|d| d.into_inner().as_secs()),
itip_http_rsvp_url: if sched.http_rsvp_enable {
if let Some(url) = sched
.http_rsvp_url
.as_deref()
.map(|v| v.trim().trim_end_matches('/'))
.filter(|v| !v.is_empty())
{
Some(url.to_string())
} else {
Some(format!("https://{}/calendar/rsvp", system.default_hostname))
}
} else {
None
},
max_shares_per_item: share.max_shares as usize,
allow_directory_query: share.allow_directory_queries,
itip_http_rsvp_expiration: sched.http_rsvp_link_expiry.into_inner().as_secs(),
itip_template: Template::parse(include_str!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../resources/html-templates/calendar-invite.html.min"
)))
.expect("Failed to parse calendar template"),
}
}
}
impl FromStr for CalendarTemplateVariable {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
match s {
"page_title" => Ok(CalendarTemplateVariable::PageTitle),
"lang" => Ok(CalendarTemplateVariable::Lang),
"dir" => Ok(CalendarTemplateVariable::Dir),
"header" => Ok(CalendarTemplateVariable::Header),
"footer" => Ok(CalendarTemplateVariable::Footer),
"event_title" => Ok(CalendarTemplateVariable::EventTitle),
"event_description" => Ok(CalendarTemplateVariable::EventDescription),
"event_details" => Ok(CalendarTemplateVariable::EventDetails),
"action_url" => Ok(CalendarTemplateVariable::ActionUrl),
"action_name" => Ok(CalendarTemplateVariable::ActionName),
"attendees" => Ok(CalendarTemplateVariable::Attendees),
"attendees_title" => Ok(CalendarTemplateVariable::AttendeesTitle),
"key" => Ok(CalendarTemplateVariable::Key),
"value" => Ok(CalendarTemplateVariable::Value),
"link" => Ok(CalendarTemplateVariable::Link),
"logo_cid" => Ok(CalendarTemplateVariable::LogoCid),
"actions" => Ok(CalendarTemplateVariable::Actions),
"changed" => Ok(CalendarTemplateVariable::Changed),
"old_value" => Ok(CalendarTemplateVariable::OldValue),
"rsvp" => Ok(CalendarTemplateVariable::Rsvp),
"color" => Ok(CalendarTemplateVariable::Color),
_ => Err(format!("Unknown calendar template variable: {}", s)),
}
}
}
#[cfg(test)]
mod tests {
use super::CalendarTemplateVariable;
use utils::template::Template;
const TEMPLATES: [(&str, &str, &str); 2] = [
(
"calendar-invite.html",
include_str!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../resources/html-templates/calendar-invite.html"
)),
include_str!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../resources/html-templates/calendar-invite.html.min"
)),
),
(
"calendar-alarm.html",
include_str!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../resources/html-templates/calendar-alarm.html"
)),
include_str!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../resources/html-templates/calendar-alarm.html.min"
)),
),
];
// Every `{{...}}` token in a template, in order of appearance
fn tokens(contents: &str) -> Vec<&str> {
let mut tokens = Vec::new();
let mut rest = contents;
while let Some((_, after)) = rest.split_once("{{") {
match after.split_once("}}") {
Some((token, tail)) => {
tokens.push(token.trim());
rest = tail;
}
None => break,
}
}
tokens
}
#[test]
fn shipped_calendar_templates_parse() {
for (name, source, minified) in TEMPLATES {
Template::<CalendarTemplateVariable>::parse(source)
.unwrap_or_else(|err| panic!("{name} failed to parse: {err}"));
Template::<CalendarTemplateVariable>::parse(minified)
.unwrap_or_else(|err| panic!("{name}.min failed to parse: {err}"));
}
}
#[test]
fn minified_calendar_templates_are_in_sync() {
for (name, source, minified) in TEMPLATES {
let source = tokens(source);
assert!(source.len() > 10, "{name} yielded no tokens to compare");
assert_eq!(
source,
tokens(minified),
"{name}.min is stale, re-run resources/scripts/minify_html.sh"
);
}
}
#[test]
fn calendar_template_tokens_are_single_line() {
// A newline inside `{{...}}` makes the parser reject the block
for (name, source, minified) in TEMPLATES {
for (suffix, contents) in [("", source), (".min", minified)] {
for token in tokens(contents) {
assert!(
!token.contains('\n') && !token.contains('\r'),
"{name}{suffix} has a multi-line token: {token:?}"
);
}
}
}
}
}
crates/common/src/config/inner.rs
+249
View File
@@ -0,0 +1,249 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use super::server::tls::build_self_signed_cert;
use crate::{
Caches, Data, DavResource, DavResources, MailboxCache, MessageStoreCache, MessageUidCache,
TlsConnectors,
auth::{AccessTokenInner, AccountCache, DomainCache, MailingListCache, RoleCache, TenantCache},
config::{
mailstore::spamfilter::SpamClassifier,
server::tls::parse_certificates,
smtp::{
auth::DkimSigners,
resolver::{Policy, Tlsa},
},
},
manager::application::WebApplications,
network::security::BlockedIps,
};
use ahash::{AHashMap, AHashSet};
use arc_swap::ArcSwap;
use mail_auth::{MX, Parameters, RecordSet, Txt};
use parking_lot::RwLock;
use registry::schema::{prelude::ObjectType, structs};
use std::{
net::{IpAddr, Ipv4Addr, Ipv6Addr},
sync::Arc,
};
use store::{LookupStores, registry::bootstrap::Bootstrap};
use utils::{
UnwrapFailure,
cache::{Cache, CacheWithTtl},
snowflake::{MAX_NODE_ID, SnowflakeIdGenerator},
tls::build_tls_connector,
};
impl Data {
pub async fn parse(bp: &mut Bootstrap) -> Self {
// Parse certificates
let mut certificates = AHashMap::new();
let mut subject_names = AHashSet::new();
parse_certificates(bp, &mut certificates, &mut subject_names).await;
if subject_names.is_empty() {
subject_names.insert("localhost".into());
}
// Build and test snowflake id generator
let node_id = bp.node_id();
if node_id > MAX_NODE_ID {
panic!("Node id {node_id} exceeds {MAX_NODE_ID}, panicking to avoid data corruption");
}
SnowflakeIdGenerator::set_node_id(node_id as u64);
let id_generator = SnowflakeIdGenerator::new();
if !id_generator.is_valid() {
panic!("Invalid system time, panicking to avoid data corruption");
}
// Initialize apps
let applications = WebApplications::new();
applications.reload(bp).await;
let blocked_ips = BlockedIps::parse(bp).await;
let lookup_stores = LookupStores::build(bp).await;
Data {
spam_classifier: ArcSwap::from_pointee(SpamClassifier::default()),
tls_certificates: ArcSwap::from_pointee(certificates),
tls_self_signed_cert: build_self_signed_cert(
subject_names
.into_iter()
.map(Into::into)
.collect::<Vec<_>>(),
)
.or_else(|err| {
bp.build_error(
ObjectType::Certificate.singleton(),
format!("Failed to build self-signed TLS certificate: {err}"),
);
build_self_signed_cert(vec!["localhost".to_string()])
})
.ok()
.map(Arc::new),
lookup_stores: ArcSwap::from_pointee(lookup_stores.stores),
blocked_ips: RwLock::new(blocked_ips),
jmap_id_gen: id_generator.clone(),
queue_id_gen: id_generator.clone(),
registry_id_gen: id_generator.clone(),
span_id_gen: id_generator,
queue_status: true.into(),
applications,
logos: Default::default(),
smtp_connectors: TlsConnectors::try_new().failed("Failed to build TLS connectors"),
asn_geo_data: Default::default(),
}
}
}
impl Caches {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let cache = bp.setting_infallible::<structs::Cache>().await;
Caches {
access_tokens: Cache::new_single_shard(
cache.access_tokens,
(std::mem::size_of::<AccessTokenInner>() + 255) as u64,
),
http_auth: Cache::new(cache.http_auth, (50 + std::mem::size_of::<u32>()) as u64),
messages: Cache::new_single_shard(
cache.messages,
(std::mem::size_of::<u32>()
+ std::mem::size_of::<Arc<MessageStoreCache>>()
+ (1024 * std::mem::size_of::<MessageUidCache>())
+ (15 * (std::mem::size_of::<MailboxCache>() + 60))) as u64,
),
files: Cache::new_single_shard(
cache.files,
(std::mem::size_of::<DavResources>() + (500 * std::mem::size_of::<DavResource>()))
as u64,
),
events: Cache::new_single_shard(
cache.events,
(std::mem::size_of::<DavResources>() + (500 * std::mem::size_of::<DavResource>()))
as u64,
),
contacts: Cache::new_single_shard(
cache.contacts,
(std::mem::size_of::<DavResources>() + (500 * std::mem::size_of::<DavResource>()))
as u64,
),
scheduling: Cache::new_single_shard(
cache.scheduling,
(std::mem::size_of::<DavResources>() + (500 * std::mem::size_of::<DavResource>()))
as u64,
),
emails: Cache::new(cache.email_addresses, 255u64),
emails_negative: CacheWithTtl::new(
cache.email_addresses_negative,
(std::mem::size_of::<DomainCache>() + 255) as u64,
),
domain_names: Cache::new(
cache.domain_names,
(std::mem::size_of::<DomainCache>() + 255) as u64,
),
domain_names_negative: CacheWithTtl::new(
cache.domain_names_negative,
(std::mem::size_of::<DomainCache>() + 255) as u64,
),
domains: Cache::new(
cache.domains,
(std::mem::size_of::<DomainCache>() + 255) as u64,
),
accounts: Cache::new(
cache.accounts,
(std::mem::size_of::<AccountCache>() + 255) as u64,
),
roles: Cache::new(cache.roles, (std::mem::size_of::<RoleCache>() + 255) as u64),
tenants: Cache::new(
cache.tenants,
(std::mem::size_of::<TenantCache>() + 255) as u64,
),
lists: Cache::new(
cache.mailing_lists,
(std::mem::size_of::<MailingListCache>() + 255) as u64,
),
dkim_signers: Cache::new(
cache.dkim_signatures,
(std::mem::size_of::<DkimSigners>() + 255) as u64,
),
dns_txt: CacheWithTtl::new(cache.dns_txt, (std::mem::size_of::<Txt>() + 255) as u64),
dns_mx: CacheWithTtl::new(cache.dns_mx, ((std::mem::size_of::<MX>() + 255) * 2) as u64),
dns_ptr: CacheWithTtl::new(cache.dns_ptr, (std::mem::size_of::<IpAddr>() + 255) as u64),
dns_ipv4: CacheWithTtl::new(
cache.dns_ipv4,
((std::mem::size_of::<Ipv4Addr>() + 255) * 2) as u64,
),
dns_ipv6: CacheWithTtl::new(
cache.dns_ipv6,
((std::mem::size_of::<Ipv6Addr>() + 255) * 2) as u64,
),
dns_tlsa: CacheWithTtl::new(cache.dns_tlsa, (std::mem::size_of::<Tlsa>() + 255) as u64),
dns_mta_sts: CacheWithTtl::new(
cache.dns_mta_sts,
(std::mem::size_of::<Policy>() + 255) as u64,
),
dns_rbl: CacheWithTtl::new(
cache.dns_rbl,
((std::mem::size_of::<Ipv4Addr>() + 255) * 2) as u64,
),
negative_cache_ttl: cache.negative_ttl.into_inner(),
}
}
#[allow(clippy::type_complexity)]
#[inline(always)]
pub fn build_auth_parameters<T>(
&self,
params: T,
) -> Parameters<
'_,
T,
CacheWithTtl<Box<str>, Txt>,
CacheWithTtl<Box<str>, RecordSet<MX>>,
CacheWithTtl<Box<str>, RecordSet<Ipv4Addr>>,
CacheWithTtl<Box<str>, RecordSet<Ipv6Addr>>,
CacheWithTtl<IpAddr, RecordSet<Box<str>>>,
> {
Parameters {
params,
cache_txt: Some(&self.dns_txt),
cache_mx: Some(&self.dns_mx),
cache_ptr: Some(&self.dns_ptr),
cache_ipv4: Some(&self.dns_ipv4),
cache_ipv6: Some(&self.dns_ipv6),
}
}
}
impl Default for Data {
fn default() -> Self {
Self {
spam_classifier: Default::default(),
tls_certificates: Default::default(),
tls_self_signed_cert: Default::default(),
blocked_ips: Default::default(),
jmap_id_gen: Default::default(),
queue_id_gen: Default::default(),
span_id_gen: Default::default(),
registry_id_gen: Default::default(),
queue_status: true.into(),
applications: WebApplications::new(),
logos: Default::default(),
smtp_connectors: TlsConnectors::try_new().unwrap(),
asn_geo_data: Default::default(),
lookup_stores: Default::default(),
}
}
}
impl TlsConnectors {
fn try_new() -> Result<Self, String> {
Ok(TlsConnectors {
pki_verify: build_tls_connector(false)?,
dummy_verify: build_tls_connector(true)?,
})
}
}
crates/common/src/config/mailstore/capabilities.rs
+323
View File
@@ -0,0 +1,323 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::config::mailstore::jmap::JmapConfig;
use ahash::AHashSet;
use calcard::icalendar::ICalendarDuration;
use jmap_proto::{
object::{email::EmailComparator, file_node::FileNodeComparator},
request::capability::{
BlobCapabilities, CalendarCapabilities, Capabilities, Capability, ContactsCapabilities,
CoreCapabilities, EmptyCapabilities, FileNodeCapabilities, MailCapabilities,
PrincipalAvailabilityCapabilities, PrincipalCapabilities, SieveAccountCapabilities,
SieveSessionCapabilities, SubmissionCapabilities, WebPushCapabilities,
},
types::date::UTCDate,
};
use registry::{
schema::structs::{Calendar, Email, SieveUserInterpreter},
types::EnumImpl,
};
use store::registry::bootstrap::Bootstrap;
use types::type_state::DataType;
use utils::map::vec_map::VecMap;
impl JmapConfig {
pub async fn add_capabilities(&mut self, bp: &mut Bootstrap) {
// Add core capabilities
self.capabilities.session.append(
Capability::Core,
Capabilities::Core(CoreCapabilities {
max_size_upload: self.upload_max_size as u64,
max_concurrent_upload: self.upload_max_concurrent.unwrap_or(u32::MAX as u64),
max_size_request: self.request_max_size as u64,
max_concurrent_requests: self.request_max_concurrent.unwrap_or(u32::MAX as u64),
max_calls_in_request: self.request_max_calls as u64,
max_objects_in_get: self.get_max_objects as u64,
max_objects_in_set: self.set_max_objects as u64,
collation_algorithms: vec![
"i;ascii-numeric".to_string(),
"i;ascii-casemap".to_string(),
"i;unicode-casemap".to_string(),
],
}),
);
// Add email capabilities
let email = bp.setting_infallible::<Email>().await;
self.capabilities.session.append(
Capability::Mail,
Capabilities::Empty(EmptyCapabilities::default()),
);
self.capabilities.account.insert(
Capability::Mail,
Capabilities::Mail(MailCapabilities {
max_mailboxes_per_email: None,
max_mailbox_depth: email.max_mailbox_depth,
max_size_mailbox_name: email.max_mailbox_name_length,
max_size_attachments_per_email: email.max_attachment_size,
email_query_sort_options: vec![
EmailComparator::ReceivedAt,
EmailComparator::Size,
EmailComparator::From,
EmailComparator::To,
EmailComparator::Subject,
EmailComparator::SentAt,
EmailComparator::HasKeyword(Default::default()),
EmailComparator::AllInThreadHaveKeyword(Default::default()),
EmailComparator::SomeInThreadHaveKeyword(Default::default()),
],
may_create_top_level_mailbox: true,
}),
);
// Add calendar capabilities
self.capabilities.session.append(
Capability::Calendars,
Capabilities::Empty(EmptyCapabilities::default()),
);
self.capabilities.account.insert(
Capability::Calendars,
Capabilities::Calendar(CalendarCapabilities {
max_calendars_per_event: None,
min_date_time: UTCDate {
year: 1,
month: 1,
day: 1,
hour: 0,
minute: 0,
second: 0,
tz_before_gmt: false,
tz_hour: 0,
tz_minute: 0,
},
max_date_time: UTCDate {
year: 9999,
month: 12,
day: 31,
hour: 23,
minute: 59,
second: 59,
tz_before_gmt: false,
tz_hour: 0,
tz_minute: 0,
},
max_expanded_query_duration: ICalendarDuration::from_seconds(86400 * 365)
.to_string(),
max_participants_per_event: bp
.setting_infallible::<Calendar>()
.await
.max_attendees
.into(),
may_create_calendar: true,
}),
);
self.capabilities.session.append(
Capability::CalendarsParse,
Capabilities::Empty(EmptyCapabilities::default()),
);
self.capabilities.account.insert(
Capability::CalendarsParse,
Capabilities::Empty(EmptyCapabilities::default()),
);
// Add contacts capabilities
self.capabilities.session.append(
Capability::Contacts,
Capabilities::Empty(EmptyCapabilities::default()),
);
self.capabilities.account.insert(
Capability::Contacts,
Capabilities::Contacts(ContactsCapabilities {
max_address_books_per_card: None,
may_create_address_book: true,
}),
);
self.capabilities.session.append(
Capability::ContactsParse,
Capabilities::Empty(EmptyCapabilities::default()),
);
self.capabilities.account.insert(
Capability::ContactsParse,
Capabilities::Empty(EmptyCapabilities::default()),
);
// Add file node capabilities
self.capabilities.session.append(
Capability::FileNode,
Capabilities::Empty(EmptyCapabilities::default()),
);
self.capabilities.account.insert(
Capability::FileNode,
Capabilities::FileNode(FileNodeCapabilities {
max_file_node_depth: None,
max_size_file_node_name: 255,
forbidden_name_chars: Some("/<>:\"\\|?*".to_string()),
forbidden_node_names: Some(
[
".", "..", "CON", "PRN", "AUX", "NUL", "COM0", "COM1", "COM2", "COM3",
"COM4", "COM5", "COM6", "COM7", "COM8", "COM9", "LPT0", "LPT1", "LPT2",
"LPT3", "LPT4", "LPT5", "LPT6", "LPT7", "LPT8", "LPT9",
]
.into_iter()
.map(str::to_string)
.collect(),
),
file_node_query_sort_options: vec![
FileNodeComparator::Name,
FileNodeComparator::Size,
FileNodeComparator::NodeType,
],
may_create_top_level_file_node: true,
case_insensitive_names: false,
web_trash_url: None,
web_url_template: None,
web_write_url_template: None,
}),
);
// Add principal capabilities
self.capabilities.session.append(
Capability::Principals,
Capabilities::Empty(EmptyCapabilities::default()),
);
self.capabilities.account.insert(
Capability::Principals,
Capabilities::Principals(PrincipalCapabilities {
current_user_principal_id: None,
}),
);
self.capabilities.session.append(
Capability::PrincipalsAvailability,
Capabilities::Empty(EmptyCapabilities::default()),
);
self.capabilities.account.insert(
Capability::PrincipalsAvailability,
Capabilities::PrincipalsAvailability(PrincipalAvailabilityCapabilities {
max_availability_duration: ICalendarDuration::from_seconds(86400 * 365).to_string(),
}),
);
// Add submission capabilities
self.capabilities.session.append(
Capability::Submission,
Capabilities::Empty(EmptyCapabilities::default()),
);
self.capabilities.account.insert(
Capability::Submission,
Capabilities::Submission(SubmissionCapabilities {
max_delayed_send: 86400 * 30,
submission_extensions: VecMap::from_iter([
("FUTURERELEASE".to_string(), Vec::new()),
("SIZE".to_string(), Vec::new()),
("DSN".to_string(), Vec::new()),
("DELIVERYBY".to_string(), Vec::new()),
("MT-PRIORITY".to_string(), vec!["MIXER".to_string()]),
("REQUIRETLS".to_string(), vec![]),
]),
}),
);
// Add vacation response capabilities
self.capabilities.session.append(
Capability::VacationResponse,
Capabilities::Empty(EmptyCapabilities::default()),
);
self.capabilities.account.insert(
Capability::VacationResponse,
Capabilities::Empty(EmptyCapabilities::default()),
);
// Add Sieve capabilities
let sieve = bp.setting_infallible::<SieveUserInterpreter>().await;
let disabled_capabilities = sieve
.disable_capabilities
.into_iter()
.map(|v| v.as_str())
.collect::<AHashSet<&str>>();
let mut extensions = sieve::compiler::grammar::Capability::all()
.iter()
.map(|c| c.to_string())
.filter(|c| !disabled_capabilities.contains(c.as_str()))
.collect::<Vec<String>>();
extensions.sort_unstable();
self.capabilities.session.append(
Capability::Sieve,
Capabilities::SieveSession(SieveSessionCapabilities::default()),
);
self.capabilities.account.insert(
Capability::Sieve,
Capabilities::SieveAccount(SieveAccountCapabilities {
max_script_name: sieve.max_script_name_length as u64,
max_script_size: sieve.max_script_size,
max_scripts: sieve.max_scripts.unwrap_or(u32::MAX as u64),
max_redirects: sieve.max_redirects,
extensions,
notification_methods: if !sieve.allowed_notify_uris.is_empty() {
sieve.allowed_notify_uris.into_inner().into()
} else {
None
},
ext_lists: None,
}),
);
// Add Blob capabilities
self.capabilities.session.append(
Capability::Blob,
Capabilities::Empty(EmptyCapabilities::default()),
);
self.capabilities.account.insert(
Capability::Blob,
Capabilities::Blob(BlobCapabilities {
max_size_blob_set: (self.request_max_size as u64 * 3 / 4) - 512,
max_data_sources: self.request_max_calls as u64,
supported_type_names: vec![
DataType::Email,
DataType::Thread,
DataType::SieveScript,
],
supported_digest_algorithms: vec!["sha", "sha-256", "sha-512"],
}),
);
// Add Quota capabilities
self.capabilities.session.append(
Capability::Quota,
Capabilities::Empty(EmptyCapabilities::default()),
);
self.capabilities.account.insert(
Capability::Quota,
Capabilities::Empty(EmptyCapabilities::default()),
);
// Add Email Delivery Push capabilities
self.capabilities.session.append(
Capability::EmailPush,
Capabilities::Empty(EmptyCapabilities::default()),
);
self.capabilities.account.insert(
Capability::EmailPush,
Capabilities::Empty(EmptyCapabilities::default()),
);
// Add Web Push VAPID capabilities
if let Some(application_server_key) = self
.vapid
.as_ref()
.map(|vapid| vapid.public_key().to_string())
{
self.capabilities.session.append(
Capability::WebPushVapid,
Capabilities::WebPush(WebPushCapabilities {
application_server_key,
}),
);
}
}
}
crates/common/src/config/mailstore/email.rs
+298
View File
@@ -0,0 +1,298 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use ahash::{AHashMap, AHashSet};
use nlp::language::Language;
use registry::{
schema::{
enums::{
CompressionAlgo, SearchCalendarField, SearchContactField, SearchEmailField,
StorageQuota,
},
prelude::ObjectType,
structs::{
AddressBook, Authentication, Calendar, DataRetention, Domain, Email, FileStorage, Jmap,
Search, SieveUserInterpreter, SystemSettings,
},
},
types::EnumImpl,
};
use std::time::Duration;
use store::{
registry::bootstrap::Bootstrap,
search::{CalendarSearchField, ContactSearchField, EmailSearchField, SearchField},
write::SearchIndex,
};
use types::special_use::SpecialUse;
use utils::cron::SimpleCron;
use crate::storage::ObjectQuota;
#[derive(Clone)]
pub struct EmailConfig {
pub default_language: Language,
pub default_domain_id: u32,
pub default_domain_name: String,
pub mailbox_max_depth: usize,
pub mailbox_name_max_len: usize,
pub mail_attachments_max_size: usize,
pub mail_max_size: usize,
pub mail_autoexpunge_after: Option<u64>,
pub email_submission_autoexpunge_after: Option<u64>,
pub changes_max_history: Option<usize>,
pub share_notification_max_history: Option<Duration>,
pub sieve_max_script_name: usize,
pub default_folders: Vec<DefaultFolder>,
pub shared_folder: String,
pub encrypt: bool,
pub encrypt_append: bool,
pub index_batch_size: usize,
pub index_fields: AHashMap<SearchIndex, AHashSet<SearchField>>,
pub max_objects: ObjectQuota,
pub compression: CompressionAlgo,
pub account_purge_frequency: SimpleCron,
pub data_purge_frequency: SimpleCron,
pub blob_purge_frequency: SimpleCron,
}
#[derive(Clone, Debug)]
pub struct DefaultFolder {
pub name: String,
pub aliases: Vec<String>,
pub special_use: SpecialUse,
pub subscribe: bool,
pub create: bool,
}
impl EmailConfig {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let email = bp.setting_infallible::<Email>().await;
let dr = bp.setting_infallible::<DataRetention>().await;
let sieve = bp.setting_infallible::<SieveUserInterpreter>().await;
let search = bp.setting_infallible::<Search>().await;
let jmap = bp.setting_infallible::<Jmap>().await;
let file = bp.setting_infallible::<FileStorage>().await;
let calendar = bp.setting_infallible::<Calendar>().await;
let address_book = bp.setting_infallible::<AddressBook>().await;
let system = bp.setting_infallible::<SystemSettings>().await;
let auth = bp.setting_infallible::<Authentication>().await;
// Obtain default domain name
let default_domain_name = if system.default_domain_id.is_valid()
&& let Some(default_domain) =
bp.get_infallible::<Domain>(system.default_domain_id).await
{
default_domain.name
} else {
if system.default_domain_id.is_valid() {
bp.build_error(
ObjectType::SystemSettings.singleton(),
format!(
"Default domain with ID {} not found",
system.default_domain_id
),
);
}
"localhost.local".to_string()
};
// Parse default object quotas
let mut max_objects = ObjectQuota::default();
for (item, max) in [
(StorageQuota::MaxEmails, email.max_messages),
(StorageQuota::MaxMailboxes, email.max_mailboxes),
(StorageQuota::MaxSieveScripts, sieve.max_scripts),
(StorageQuota::MaxEmailIdentities, email.max_identities),
(StorageQuota::MaxEmailSubmissions, email.max_submissions),
(StorageQuota::MaxMaskedAddresses, email.max_masked_addresses),
(StorageQuota::MaxAppPasswords, auth.max_app_passwords),
(StorageQuota::MaxApiKeys, auth.max_api_keys),
(StorageQuota::MaxPublicKeys, email.max_public_keys),
(StorageQuota::MaxPushSubscriptions, jmap.max_subscriptions),
(StorageQuota::MaxCalendars, calendar.max_calendars),
(StorageQuota::MaxCalendarEvents, calendar.max_events),
(
StorageQuota::MaxParticipantIdentities,
calendar.max_participant_identities,
),
(
StorageQuota::MaxCalendarEventNotifications,
calendar.max_event_notifications,
),
(
StorageQuota::MaxAddressBooks,
address_book.max_address_books,
),
(StorageQuota::MaxContactCards, address_book.max_contacts),
(StorageQuota::MaxFiles, file.max_files),
(StorageQuota::MaxFolders, file.max_folders),
] {
if let Some(max) = max {
max_objects.set(item, max as u32);
}
}
// Parse default folders
let mut default_folders = Vec::new();
let mut shared_folder = "Shared Folders".to_string();
for (special_use, folder) in email.default_folders {
let special_use = match special_use {
registry::schema::enums::SpecialUse::Inbox => SpecialUse::Inbox,
registry::schema::enums::SpecialUse::Trash => SpecialUse::Trash,
registry::schema::enums::SpecialUse::Junk => SpecialUse::Junk,
registry::schema::enums::SpecialUse::Drafts => SpecialUse::Drafts,
registry::schema::enums::SpecialUse::Archive => SpecialUse::Archive,
registry::schema::enums::SpecialUse::Sent => SpecialUse::Sent,
registry::schema::enums::SpecialUse::Important => SpecialUse::Important,
registry::schema::enums::SpecialUse::Memos => SpecialUse::Memos,
registry::schema::enums::SpecialUse::Scheduled => SpecialUse::Scheduled,
registry::schema::enums::SpecialUse::Snoozed => SpecialUse::Snoozed,
registry::schema::enums::SpecialUse::Shared => {
shared_folder = folder.name;
continue;
}
};
default_folders.push(DefaultFolder {
name: folder.name,
aliases: folder.aliases.into_inner(),
special_use,
subscribe: folder.subscribe,
create: folder.create
|| matches!(
special_use,
SpecialUse::Inbox | SpecialUse::Trash | SpecialUse::Junk
),
});
}
for (special_use, name) in [
(SpecialUse::Inbox, "Inbox"),
(SpecialUse::Trash, "Deleted Items"),
(SpecialUse::Junk, "Junk Mail"),
(SpecialUse::Drafts, "Drafts"),
(SpecialUse::Sent, "Sent Items"),
] {
if !default_folders.iter().any(|f| f.special_use == special_use) {
default_folders.push(DefaultFolder {
name: name.to_string(),
aliases: Vec::new(),
special_use,
subscribe: true,
create: true,
});
}
}
// Search Index settings
let mut index_fields = AHashMap::new();
if search.index_email {
index_fields.insert(
SearchIndex::Email,
search
.index_email_fields
.into_iter()
.map(|field| {
SearchField::Email(match field {
SearchEmailField::From => EmailSearchField::From,
SearchEmailField::To => EmailSearchField::To,
SearchEmailField::Cc => EmailSearchField::Cc,
SearchEmailField::Bcc => EmailSearchField::Bcc,
SearchEmailField::Subject => EmailSearchField::Subject,
SearchEmailField::Body => EmailSearchField::Body,
SearchEmailField::Attachment => EmailSearchField::Attachment,
SearchEmailField::ReceivedAt => EmailSearchField::ReceivedAt,
SearchEmailField::SentAt => EmailSearchField::SentAt,
SearchEmailField::Size => EmailSearchField::Size,
SearchEmailField::HasAttachment => EmailSearchField::HasAttachment,
SearchEmailField::Headers => EmailSearchField::Headers,
})
})
.collect(),
);
}
if search.index_contacts {
index_fields.insert(
SearchIndex::Contacts,
search
.index_contact_fields
.into_iter()
.map(|field| {
SearchField::Contact(match field {
SearchContactField::Member => ContactSearchField::Member,
SearchContactField::Kind => ContactSearchField::Kind,
SearchContactField::Name => ContactSearchField::Name,
SearchContactField::Nickname => ContactSearchField::Nickname,
SearchContactField::Organization => ContactSearchField::Organization,
SearchContactField::Email => ContactSearchField::Email,
SearchContactField::Phone => ContactSearchField::Phone,
SearchContactField::OnlineService => ContactSearchField::OnlineService,
SearchContactField::Address => ContactSearchField::Address,
SearchContactField::Note => ContactSearchField::Note,
SearchContactField::Uid => ContactSearchField::Uid,
})
})
.collect(),
);
}
if search.index_calendar {
index_fields.insert(
SearchIndex::Calendar,
search
.index_calendar_fields
.into_iter()
.map(|field| {
SearchField::Calendar(match field {
SearchCalendarField::Title => CalendarSearchField::Title,
SearchCalendarField::Description => CalendarSearchField::Description,
SearchCalendarField::Location => CalendarSearchField::Location,
SearchCalendarField::Owner => CalendarSearchField::Owner,
SearchCalendarField::Attendee => CalendarSearchField::Attendee,
SearchCalendarField::Start => CalendarSearchField::Start,
SearchCalendarField::Uid => CalendarSearchField::Uid,
})
})
.collect(),
);
}
EmailConfig {
default_language: Language::from_iso_639(search.default_language.as_str())
.unwrap_or(Language::English),
mailbox_max_depth: email.max_mailbox_depth as usize,
mailbox_name_max_len: email.max_mailbox_name_length as usize,
mail_attachments_max_size: email.max_attachment_size as usize,
mail_max_size: email.max_message_size as usize,
mail_autoexpunge_after: dr.expunge_trash_after.map(|d| d.into_inner().as_secs()),
email_submission_autoexpunge_after: dr
.expunge_submissions_after
.map(|d| d.into_inner().as_secs()),
changes_max_history: dr.max_changes_history.map(|v| v as usize),
share_notification_max_history: dr.expunge_share_notify_after.map(|v| v.into_inner()),
sieve_max_script_name: sieve.max_script_name_length as usize,
encrypt: email.encrypt_at_rest,
encrypt_append: email.encrypt_on_append,
index_batch_size: search.index_batch_size as usize,
index_fields,
max_objects,
default_folders,
shared_folder,
account_purge_frequency: dr.expunge_schedule.into(),
data_purge_frequency: dr.data_cleanup_schedule.into(),
blob_purge_frequency: dr.blob_cleanup_schedule.into(),
compression: email.compression_algorithm,
default_domain_id: system.default_domain_id.id() as u32,
default_domain_name,
}
}
}
crates/common/src/config/mailstore/imap.rs
+49
View File
@@ -0,0 +1,49 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use registry::schema::structs::{Imap, Rate};
use std::time::Duration;
use store::registry::bootstrap::Bootstrap;
#[derive(Default, Clone)]
pub struct ImapConfig {
pub max_request_size: usize,
pub max_auth_failures: u32,
pub allow_plain_auth: bool,
pub timeout_auth: Duration,
pub timeout_unauth: Duration,
pub timeout_idle: Duration,
pub rate_requests: Option<Rate>,
pub rate_concurrent: Option<u64>,
pub max_messages_per_command: u32,
pub max_messages_per_save: u32,
pub min_uid_batch_size: u32,
pub max_uid_batches: u32,
}
impl ImapConfig {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let imap = bp.setting_infallible::<Imap>().await;
ImapConfig {
max_request_size: imap.max_request_size as usize,
max_auth_failures: imap.max_auth_failures as u32,
timeout_auth: imap.timeout_authenticated.into_inner(),
timeout_unauth: imap.timeout_anonymous.into_inner(),
timeout_idle: imap.timeout_idle.into_inner(),
rate_requests: imap.max_request_rate,
rate_concurrent: imap.max_concurrent,
allow_plain_auth: imap.allow_plain_text_auth,
max_messages_per_command: imap.max_messages_per_command.min(u32::MAX as u64) as u32,
max_messages_per_save: imap.max_messages_per_save.min(u32::MAX as u64) as u32,
min_uid_batch_size: imap.min_uid_batch_size.min(u32::MAX as u64) as u32,
max_uid_batches: imap.max_uid_batches.min(u32::MAX as u64) as u32,
}
}
}
crates/common/src/config/mailstore/jmap.rs
+134
View File
@@ -0,0 +1,134 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::network::webpush::{Vapid, VapidKey};
use jmap_proto::request::capability::BaseCapabilities;
use registry::schema::{prelude::ObjectType, structs::Jmap};
use std::time::Duration;
use store::registry::bootstrap::Bootstrap;
#[derive(Default, Clone)]
pub struct JmapConfig {
pub query_max_results: usize,
pub snippet_max_results: usize,
pub changes_max_results: usize,
pub request_max_size: usize,
pub request_max_calls: usize,
pub request_max_concurrent: Option<u64>,
pub get_max_objects: usize,
pub set_max_objects: usize,
pub upload_max_size: usize,
pub upload_max_concurrent: Option<u64>,
pub upload_tmp_quota_size: usize,
pub upload_tmp_quota_amount: usize,
pub upload_tmp_ttl: u64,
pub mail_parse_max_items: usize,
pub contact_parse_max_items: usize,
pub calendar_parse_max_items: usize,
pub event_source_throttle: Duration,
pub push_attempt_interval: Duration,
pub push_attempts_max: u32,
pub push_retry_interval: Duration,
pub push_timeout: Duration,
pub push_verify_timeout: Duration,
pub push_throttle: Duration,
pub push_total_shards: u32,
pub push_max_size: usize,
pub web_socket_throttle: Duration,
pub web_socket_timeout: Duration,
pub web_socket_heartbeat: Duration,
pub vapid: Option<Vapid>,
pub capabilities: BaseCapabilities,
}
impl JmapConfig {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let jmap = bp.setting_infallible::<Jmap>().await;
let web_push_key = jmap
.web_push_key
.secret()
.await
.map_err(|err| {
bp.build_error(
ObjectType::Jmap.singleton(),
format!("Unable to retrieve Web Push key: {err}"),
);
})
.unwrap_or_default()
.map(|k| k.into_owned());
let web_push_contact = jmap
.web_push_contact
.as_deref()
.and_then(crate::network::webpush::normalize_contact)
.or_else(|| {
let hostname = bp.registry.local_hostname();
(!hostname.is_empty()).then(|| format!("mailto:postmaster@{hostname}"))
});
let mut jmap = JmapConfig {
query_max_results: jmap.query_max_results as usize,
changes_max_results: jmap.changes_max_results as usize,
snippet_max_results: jmap.snippet_max_results as usize,
request_max_size: jmap.max_request_size as usize,
request_max_calls: jmap.max_method_calls as usize,
request_max_concurrent: jmap.max_concurrent_requests,
get_max_objects: jmap.get_max_results as usize,
set_max_objects: jmap.set_max_objects as usize,
upload_max_size: jmap.max_upload_size as usize,
upload_max_concurrent: jmap.max_concurrent_uploads,
upload_tmp_quota_size: jmap.upload_quota as usize,
upload_tmp_quota_amount: jmap.max_upload_count as usize,
upload_tmp_ttl: jmap.upload_ttl.into_inner().as_secs().max(1),
mail_parse_max_items: jmap.parse_limit_email as usize,
contact_parse_max_items: jmap.parse_limit_contact as usize,
calendar_parse_max_items: jmap.parse_limit_event as usize,
event_source_throttle: jmap.event_source_throttle.into_inner(),
web_socket_throttle: jmap.websocket_throttle.into_inner(),
web_socket_timeout: jmap.websocket_timeout.into_inner(),
web_socket_heartbeat: jmap.websocket_heartbeat.into_inner(),
push_attempt_interval: jmap.push_attempt_wait.into_inner(),
push_attempts_max: jmap.push_max_attempts as u32,
push_retry_interval: jmap.push_retry_wait.into_inner(),
push_timeout: jmap.push_request_timeout.into_inner(),
push_verify_timeout: jmap.push_verify_timeout.into_inner(),
push_throttle: jmap.push_throttle.into_inner(),
push_total_shards: jmap.push_shards_total as u32,
push_max_size: jmap.max_push_size as usize,
vapid: None,
capabilities: BaseCapabilities::default(),
};
// Enable Web Push VAPID only when a signing key is configured
jmap.vapid = web_push_key
.as_deref()
.map(str::trim)
.filter(|pem| !pem.is_empty())
.and_then(|pem| match VapidKey::from_pkcs8_pem(pem) {
Ok(key) => Some(key),
Err(err) => {
bp.build_error(
ObjectType::Jmap.singleton(),
format!("Invalid Web Push VAPID key: {err}"),
);
None
}
})
.map(|key| Vapid::new(key, web_push_contact));
// Add capabilities
jmap.add_capabilities(bp).await;
jmap
}
}
crates/common/src/config/mailstore/mod.rs
+12
View File
@@ -0,0 +1,12 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
pub mod capabilities;
pub mod email;
pub mod imap;
pub mod jmap;
pub mod scripts;
pub mod spamfilter;
crates/common/src/config/mailstore/scripts.rs
+279
View File
@@ -0,0 +1,279 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::{
VERSION_PUBLIC,
expr::if_block::{BootstrapExprExt, IfBlock},
scripts::{
functions::{register_functions_trusted, register_functions_untrusted},
plugins::RegisterSievePlugins,
},
};
use ahash::AHashMap;
use registry::{
schema::{
prelude::ObjectType,
structs::{
SieveSystemInterpreter, SieveSystemScript, SieveUserInterpreter, SieveUserScript,
SystemSettings,
},
},
types::EnumImpl,
};
use sieve::{Compiler, Runtime, Sieve, compiler::grammar::Capability};
use std::{collections::hash_map::Entry, sync::Arc};
use store::registry::bootstrap::Bootstrap;
pub struct Scripting {
pub untrusted_compiler: Compiler,
pub untrusted_runtime: Runtime,
pub trusted_runtime: Runtime,
pub trusted_compiler: Compiler,
pub max_received_headers: usize,
pub from_addr: IfBlock,
pub from_name: IfBlock,
pub return_path: IfBlock,
pub sign: IfBlock,
pub untrusted_sign: IfBlock,
pub trusted_scripts: AHashMap<String, Arc<Sieve>>,
pub untrusted_scripts: AHashMap<String, Arc<Sieve>>,
pub http_client: reqwest::Client,
}
impl Scripting {
pub async fn parse(bp: &mut Bootstrap) -> Self {
// Parse untrusted compiler
let untrusted = bp.setting_infallible::<SieveUserInterpreter>().await;
let untrusted_sign = bp.compile_expr(
ObjectType::SieveUserInterpreter.singleton(),
&untrusted.ctx_dkim_sign_domain(),
);
let mut fnc_map_untrusted = register_functions_untrusted().register_plugins_untrusted();
let untrusted_compiler = Compiler::new()
.with_max_script_size(untrusted.max_script_size as usize)
.with_max_string_size(untrusted.max_string_length as usize)
.with_max_variable_name_size(untrusted.max_var_name_length as usize)
.with_max_nested_blocks(untrusted.max_nested_blocks as usize)
.with_max_nested_tests(untrusted.max_nested_tests as usize)
.with_max_nested_foreverypart(untrusted.max_nested_for_every as usize)
.with_max_match_variables(untrusted.max_match_vars as usize)
.with_max_local_variables(untrusted.max_local_vars as usize)
.with_max_header_size(untrusted.max_header_size as usize)
.with_max_includes(untrusted.max_includes as usize)
.register_functions(&mut fnc_map_untrusted);
// Parse untrusted runtime
let mut untrusted_runtime = Runtime::new()
.with_functions(&mut fnc_map_untrusted)
.with_max_nested_includes(untrusted.max_nested_includes as usize)
.with_cpu_limit(untrusted.max_cpu_cycles as usize)
.with_max_variable_size(untrusted.max_var_size as usize)
.with_max_redirects(untrusted.max_redirects as usize)
.with_max_received_headers(usize::MAX) // This is set to usize::MAX here, but the actual limit is enforced during ingestion.
.with_max_header_size(untrusted.max_header_size as usize)
.with_max_out_messages(untrusted.max_out_messages as usize)
.with_default_vacation_expiry(untrusted.default_expiry_vacation.into_inner().as_secs())
.with_default_duplicate_expiry(
untrusted.default_expiry_duplicate.into_inner().as_secs(),
)
.with_capability(Capability::Expressions)
.without_capabilities(
untrusted
.disable_capabilities
.iter()
.map(|cap| cap.as_str()),
)
.with_valid_notification_uris(untrusted.allowed_notify_uris)
.with_protected_headers(untrusted.protected_headers)
.with_vacation_default_subject(untrusted.default_subject)
.with_vacation_subject_prefix(untrusted.default_subject_prefix)
.with_env_variable("name", "Stalwart Server")
.with_env_variable("version", VERSION_PUBLIC)
.with_env_variable("location", "MS")
.with_env_variable("phase", "during");
// Parse trusted compiler and runtime
let mut fnc_map_trusted = register_functions_trusted().register_plugins_trusted();
// Allocate compiler and runtime
let trusted = bp.setting_infallible::<SieveSystemInterpreter>().await;
let system = bp.setting_infallible::<SystemSettings>().await;
let local_hostname = if !system.default_hostname.is_empty() {
system.default_hostname.clone()
} else {
bp.registry.local_hostname().to_string()
};
let trusted_compiler = Compiler::new()
.with_max_string_size(52428800)
.with_max_variable_name_size(100)
.with_max_nested_blocks(50)
.with_max_nested_tests(50)
.with_max_nested_foreverypart(10)
.with_max_local_variables(8192)
.with_max_header_size(10240)
.with_max_includes(10)
.with_no_capability_check(trusted.no_capability_check)
.register_functions(&mut fnc_map_trusted);
let mut trusted_runtime = Runtime::new()
.without_capabilities([
Capability::FileInto,
Capability::Vacation,
Capability::VacationSeconds,
Capability::Fcc,
Capability::Mailbox,
Capability::MailboxId,
Capability::MboxMetadata,
Capability::ServerMetadata,
Capability::ImapSieve,
Capability::Duplicate,
])
.with_capability(Capability::Expressions)
.with_capability(Capability::While)
.with_max_variable_size(trusted.max_var_size as usize)
.with_max_header_size(10240)
.with_valid_notification_uri("mailto")
.with_functions(&mut fnc_map_trusted)
.with_max_redirects(trusted.max_redirects as usize)
.with_max_out_messages(trusted.max_out_messages as usize)
.with_cpu_limit(trusted.max_cpu_cycles as usize)
.with_max_nested_includes(trusted.max_nested_includes as usize)
.with_max_received_headers(trusted.max_received_headers as usize)
.with_default_duplicate_expiry(trusted.duplicate_expiry.into_inner().as_secs());
trusted_runtime.set_local_hostname(local_hostname.clone());
untrusted_runtime.set_local_hostname(local_hostname);
// Parse trusted scripts
let mut trusted_scripts: AHashMap<String, Arc<Sieve>> = AHashMap::new();
for script in bp.list_infallible::<SieveSystemScript>().await {
if !script.object.is_active {
continue;
}
match trusted_compiler.compile(script.object.contents.as_bytes()) {
Ok(compiled) => match trusted_scripts.entry(script.object.name.to_lowercase()) {
Entry::Vacant(entry) => {
entry.insert(compiled.into());
}
Entry::Occupied(_) => {
bp.build_error(
script.id,
format!(
"Another active system Sieve script is already named {:?}, script names are case insensitive",
script.object.name
),
);
}
},
Err(err) => {
bp.build_error(
script.id,
format!("Failed to compile system Sieve script: {err}"),
);
}
}
}
// Parse untrusted scripts
let mut untrusted_scripts: AHashMap<String, Arc<Sieve>> = AHashMap::new();
for script in bp.list_infallible::<SieveUserScript>().await {
if !script.object.is_active {
continue;
}
match untrusted_compiler.compile(script.object.contents.as_bytes()) {
Ok(compiled) => match untrusted_scripts.entry(script.object.name.to_lowercase()) {
Entry::Vacant(entry) => {
entry.insert(compiled.into());
}
Entry::Occupied(_) => {
bp.build_error(
script.id,
format!(
"Another active user global Sieve script is already named {:?}, script names are case insensitive",
script.object.name
),
);
}
},
Err(err) => {
bp.build_error(
script.id,
format!("Failed to compile user global Sieve script: {err}"),
);
}
}
}
Scripting {
untrusted_compiler,
untrusted_runtime,
trusted_runtime,
trusted_compiler,
untrusted_scripts,
trusted_scripts,
http_client: utils::http::http_client_builder(cfg!(feature = "test_mode"))
.pool_max_idle_per_host(0)
.redirect(reqwest::redirect::Policy::none())
.build()
.unwrap_or_default(),
max_received_headers: untrusted.max_received_headers as usize,
from_addr: bp.compile_expr(
ObjectType::SieveSystemInterpreter.singleton(),
&trusted.ctx_default_from_address(),
),
from_name: bp.compile_expr(
ObjectType::SieveSystemInterpreter.singleton(),
&trusted.ctx_default_from_name(),
),
return_path: bp.compile_expr(
ObjectType::SieveSystemInterpreter.singleton(),
&trusted.ctx_default_return_path(),
),
sign: bp.compile_expr(
ObjectType::SieveSystemInterpreter.singleton(),
&trusted.ctx_dkim_sign_domain(),
),
untrusted_sign,
}
}
pub fn trusted_script(&self, name: &str) -> Option<&Arc<Sieve>> {
script_by_name(&self.trusted_scripts, name)
}
pub fn untrusted_script(&self, name: &str) -> Option<&Arc<Sieve>> {
script_by_name(&self.untrusted_scripts, name)
}
}
fn script_by_name<'x>(
scripts: &'x AHashMap<String, Arc<Sieve>>,
name: &str,
) -> Option<&'x Arc<Sieve>> {
scripts
.get(name)
.or_else(|| scripts.get(name.to_lowercase().as_str()))
}
impl Clone for Scripting {
fn clone(&self) -> Self {
Self {
untrusted_compiler: self.untrusted_compiler.clone(),
untrusted_runtime: self.untrusted_runtime.clone(),
trusted_runtime: self.trusted_runtime.clone(),
from_addr: self.from_addr.clone(),
from_name: self.from_name.clone(),
return_path: self.return_path.clone(),
max_received_headers: self.max_received_headers,
sign: self.sign.clone(),
untrusted_sign: self.untrusted_sign.clone(),
trusted_scripts: self.trusted_scripts.clone(),
untrusted_scripts: self.untrusted_scripts.clone(),
trusted_compiler: self.trusted_compiler.clone(),
http_client: self.http_client.clone(),
}
}
}
crates/common/src/config/mailstore/spamfilter.rs
+742
View File
@@ -0,0 +1,742 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::expr::{
Variable,
functions::ResolveVariable,
if_block::{BootstrapExprExt, IfBlock},
};
use ahash::AHashSet;
use mail_auth::common::resolver::ToReverseName;
use nlp::classifier::model::{CcfhClassifier, FhClassifier};
use registry::schema::{
enums::{ExpressionVariable, ModelSize},
prelude::ObjectType,
structs::{
self, SpamDnsblServer, SpamDnsblSettings, SpamFileExtension, SpamPyzor, SpamRule,
SpamSettings, SpamTag,
},
};
use sieve::SpamStatus;
use std::{
net::{IpAddr, SocketAddr},
time::Duration,
};
use store::registry::{RegistryObject, bootstrap::Bootstrap};
use tokio::net::lookup_host;
use utils::{cache::CacheItemWeight, glob::GlobMap};
#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default)]
pub enum SpamClassifier {
FhClassifier {
classifier: FhClassifier,
last_trained_at: u64,
},
CcfhClassifier {
classifier: CcfhClassifier,
last_trained_at: u64,
},
#[default]
Disabled,
}
#[derive(Debug, Clone, Default)]
pub struct SpamFilterConfig {
pub enabled: bool,
pub card_is_ham: bool,
pub trusted_reply: bool,
pub grey_list_expiry: Option<u64>,
pub dnsbl: DnsBlConfig,
pub rules: SpamFilterRules,
pub lists: SpamFilterLists,
pub pyzor: Option<PyzorConfig>,
pub classifier: Option<ClassifierConfig>,
pub scores: SpamFilterScoreConfig,
pub spam_rules_url: Option<String>,
pub url_client: reqwest::Client,
}
#[derive(Debug, Clone, Default)]
pub struct SpamFilterScoreConfig {
pub reject_threshold: f32,
pub discard_threshold: f32,
pub spam_threshold: f32,
}
impl SpamFilterScoreConfig {
pub fn spam_percentage(&self, score: f32) -> u8 {
let spam_threshold = self.spam_threshold;
if spam_threshold <= 0.0 {
return if score >= spam_threshold { 100 } else { 0 };
}
let max_threshold = [self.reject_threshold, self.discard_threshold]
.into_iter()
.filter(|threshold| *threshold > spam_threshold)
.min_by(f32::total_cmp)
.unwrap_or(spam_threshold * 2.0);
if score <= 0.0 {
0
} else if score < spam_threshold {
((50.0 * score / spam_threshold) as u8).min(49)
} else {
((50.0 + 50.0 * (score - spam_threshold) / (max_threshold - spam_threshold)) as u8)
.min(100)
}
}
pub fn is_spam(&self, score: f32) -> bool {
score >= self.spam_threshold
}
}
pub fn spam_status(percentage: Option<u8>) -> SpamStatus {
match percentage {
Some(0) => SpamStatus::Ham,
Some(100) => SpamStatus::Spam,
Some(percentage) => SpamStatus::MaybeSpam(percentage as f64 / 100.0),
None => SpamStatus::Unknown,
}
}
#[derive(Debug, Clone, Default)]
pub struct DnsBlConfig {
pub max_ip_checks: usize,
pub max_domain_checks: usize,
pub max_email_checks: usize,
pub max_url_checks: usize,
pub servers: Vec<DnsBlServer>,
}
#[derive(Debug, Clone, Default)]
pub struct SpamFilterLists {
pub file_extensions: GlobMap<FileExtension>,
pub scores: GlobMap<SpamFilterAction<f32>>,
}
#[derive(Debug, Clone)]
pub enum SpamFilterAction<T> {
Allow(T),
Discard,
Reject,
Disabled,
}
#[derive(Debug, Clone, Default)]
pub struct ClassifierConfig {
pub w_params: FtrlParameters,
pub i_params: Option<FtrlParameters>,
pub reservoir_capacity: usize,
pub min_ham_samples: u64,
pub min_spam_samples: u64,
pub auto_learn_reply_ham: bool,
pub auto_learn_card_is_ham: bool,
pub auto_learn_spam_trap: bool,
pub auto_learn_spam_rbl_count: u32,
pub hold_samples_for: u64,
pub train_frequency: Option<u64>,
pub log_scale: bool,
pub l2_normalize: bool,
}
#[derive(Debug, Clone, Default)]
pub struct FtrlParameters {
pub feature_hash_size: usize,
pub alpha: f64,
pub beta: f64,
pub l1_ratio: f64,
pub l2_ratio: f64,
}
#[derive(Debug, Clone)]
pub struct PyzorConfig {
pub address: SocketAddr,
pub timeout: Duration,
pub min_count: u64,
pub min_wl_count: u64,
pub ratio: f64,
}
#[derive(Debug, Default, Clone, PartialEq, Eq)]
pub struct SpamFilterRules {
pub url: Vec<IfBlock>,
pub domain: Vec<IfBlock>,
pub email: Vec<IfBlock>,
pub ip: Vec<IfBlock>,
pub header: Vec<IfBlock>,
pub body: Vec<IfBlock>,
pub any: Vec<IfBlock>,
}
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct FileExtension {
pub known_types: AHashSet<String>,
pub is_bad: bool,
pub is_archive: bool,
pub is_nz: bool,
}
#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Hash)]
pub enum Element {
Url,
Domain,
Email,
Ip,
Header,
Body,
#[default]
Any,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum Location {
EnvelopeFrom,
EnvelopeTo,
HeaderDkimPass,
HeaderReceived,
HeaderFrom,
HeaderReplyTo,
HeaderSubject,
HeaderTo,
HeaderCc,
HeaderBcc,
HeaderMid,
HeaderDnt,
Ehlo,
BodyText,
BodyHtml,
Attachment,
Tcp,
}
#[derive(Debug, Clone)]
pub struct DnsBlServer {
pub id: String,
pub zone: IfBlock,
pub scope: Element,
pub tags: IfBlock,
}
impl SpamFilterConfig {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let spam = bp.setting_infallible::<SpamSettings>().await;
SpamFilterConfig {
enabled: spam.enable,
card_is_ham: spam.trust_contacts,
trusted_reply: spam.trust_replies,
dnsbl: DnsBlConfig::parse(bp).await,
rules: SpamFilterRules::parse(bp).await,
lists: SpamFilterLists::parse(bp).await,
pyzor: PyzorConfig::parse(bp).await,
classifier: ClassifierConfig::parse(bp).await,
scores: SpamFilterScoreConfig {
reject_threshold: spam.score_reject.into_inner() as f32,
discard_threshold: spam.score_discard.into_inner() as f32,
spam_threshold: spam.score_spam.into_inner() as f32,
},
grey_list_expiry: spam.greylist_for.map(|d| d.into_inner().as_secs()),
spam_rules_url: spam.spam_filter_rules_url,
url_client: utils::http::http_client_builder(true)
.pool_max_idle_per_host(0)
.redirect(reqwest::redirect::Policy::none())
.user_agent("Mozilla/5.0 (X11; Linux i686; rv:109.0) Gecko/20100101 Firefox/118.0")
.build()
.unwrap_or_default(),
}
}
}
impl SpamFilterRules {
pub async fn parse(bp: &mut Bootstrap) -> SpamFilterRules {
let mut rules = vec![];
for rule in bp.list_infallible::<SpamRule>().await {
if let Some(rule) = SpamFilterRule::parse(bp, rule) {
rules.push(rule);
}
}
rules.sort_by_key(|a| a.priority);
let mut result = SpamFilterRules::default();
for rule in rules {
match rule.scope {
Element::Url => result.url.push(rule.rule),
Element::Domain => result.domain.push(rule.rule),
Element::Email => result.email.push(rule.rule),
Element::Ip => result.ip.push(rule.rule),
Element::Header => result.header.push(rule.rule),
Element::Body => result.body.push(rule.rule),
Element::Any => result.any.push(rule.rule),
}
}
result
}
}
struct SpamFilterRule {
rule: IfBlock,
priority: i32,
scope: Element,
}
impl SpamFilterRule {
pub fn parse(bp: &mut Bootstrap, obj: RegistryObject<SpamRule>) -> Option<Self> {
match obj.object {
SpamRule::Any(rule) if rule.enable => SpamFilterRule {
rule: bp.compile_expr(obj.id, &rule.ctx_condition()),
scope: Element::Any,
priority: rule.priority as i32,
}
.into(),
SpamRule::Url(rule) if rule.enable => SpamFilterRule {
rule: bp.compile_expr(obj.id, &rule.ctx_condition()),
scope: Element::Url,
priority: rule.priority as i32,
}
.into(),
SpamRule::Domain(rule) if rule.enable => SpamFilterRule {
rule: bp.compile_expr(obj.id, &rule.ctx_condition()),
scope: Element::Domain,
priority: rule.priority as i32,
}
.into(),
SpamRule::Email(rule) if rule.enable => SpamFilterRule {
rule: bp.compile_expr(obj.id, &rule.ctx_condition()),
scope: Element::Email,
priority: rule.priority as i32,
}
.into(),
SpamRule::Ip(rule) if rule.enable => SpamFilterRule {
rule: bp.compile_expr(obj.id, &rule.ctx_condition()),
scope: Element::Ip,
priority: rule.priority as i32,
}
.into(),
SpamRule::Header(rule) if rule.enable => SpamFilterRule {
rule: bp.compile_expr(obj.id, &rule.ctx_condition()),
scope: Element::Header,
priority: rule.priority as i32,
}
.into(),
SpamRule::Body(rule) if rule.enable => SpamFilterRule {
rule: bp.compile_expr(obj.id, &rule.ctx_condition()),
scope: Element::Body,
priority: rule.priority as i32,
}
.into(),
_ => None,
}
}
}
impl DnsBlConfig {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let mut servers = vec![];
for server in bp.list_infallible::<SpamDnsblServer>().await {
if let Some(server) = DnsBlServer::parse(bp, server) {
servers.push(server);
}
}
let dnsbl = bp.setting_infallible::<SpamDnsblSettings>().await;
DnsBlConfig {
max_ip_checks: dnsbl.ip_limit as usize,
max_domain_checks: dnsbl.domain_limit as usize,
max_email_checks: dnsbl.email_limit as usize,
max_url_checks: dnsbl.url_limit as usize,
servers,
}
}
}
impl DnsBlServer {
pub fn parse(bp: &mut Bootstrap, obj: RegistryObject<SpamDnsblServer>) -> Option<Self> {
match obj.object {
SpamDnsblServer::Any(server) if server.enable => DnsBlServer {
zone: bp.compile_expr(obj.id, &server.ctx_zone()),
tags: bp.compile_expr(obj.id, &server.ctx_tag()),
scope: Element::Any,
id: server.name,
}
.into(),
SpamDnsblServer::Url(server) if server.enable => DnsBlServer {
zone: bp.compile_expr(obj.id, &server.ctx_zone()),
tags: bp.compile_expr(obj.id, &server.ctx_tag()),
scope: Element::Url,
id: server.name,
}
.into(),
SpamDnsblServer::Domain(server) if server.enable => DnsBlServer {
zone: bp.compile_expr(obj.id, &server.ctx_zone()),
tags: bp.compile_expr(obj.id, &server.ctx_tag()),
scope: Element::Domain,
id: server.name,
}
.into(),
SpamDnsblServer::Email(server) if server.enable => DnsBlServer {
zone: bp.compile_expr(obj.id, &server.ctx_zone()),
tags: bp.compile_expr(obj.id, &server.ctx_tag()),
scope: Element::Email,
id: server.name,
}
.into(),
SpamDnsblServer::Ip(server) if server.enable => DnsBlServer {
zone: bp.compile_expr(obj.id, &server.ctx_zone()),
tags: bp.compile_expr(obj.id, &server.ctx_tag()),
scope: Element::Ip,
id: server.name,
}
.into(),
SpamDnsblServer::Header(server) if server.enable => DnsBlServer {
zone: bp.compile_expr(obj.id, &server.ctx_zone()),
tags: bp.compile_expr(obj.id, &server.ctx_tag()),
scope: Element::Header,
id: server.name,
}
.into(),
SpamDnsblServer::Body(server) if server.enable => DnsBlServer {
zone: bp.compile_expr(obj.id, &server.ctx_zone()),
tags: bp.compile_expr(obj.id, &server.ctx_tag()),
scope: Element::Body,
id: server.name,
}
.into(),
_ => None,
}
}
}
impl SpamFilterLists {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let mut lists = SpamFilterLists {
file_extensions: GlobMap::default(),
scores: GlobMap::default(),
};
for tag in bp.list_infallible::<SpamTag>().await {
match tag.object {
SpamTag::Score(tag) => lists.scores.insert_pattern(
&tag.tag,
SpamFilterAction::Allow(tag.score.into_inner() as f32),
),
SpamTag::Discard(tag) => lists
.scores
.insert_pattern(&tag.tag, SpamFilterAction::Discard),
SpamTag::Reject(tag) => lists
.scores
.insert_pattern(&tag.tag, SpamFilterAction::Reject),
}
}
for ext in bp.list_infallible::<SpamFileExtension>().await {
let ext = ext.object;
lists.file_extensions.insert_pattern(
&ext.extension,
FileExtension {
known_types: ext.content_types.into_iter().collect(),
is_bad: ext.is_bad,
is_archive: ext.is_archive,
is_nz: ext.is_nz,
},
);
}
lists
}
}
impl PyzorConfig {
pub async fn parse(bp: &mut Bootstrap) -> Option<Self> {
let pyzor = bp.setting_infallible::<SpamPyzor>().await;
if !pyzor.enable {
return None;
}
let port = pyzor.port;
let host = pyzor.host;
let address = match lookup_host(format!("{host}:{port}"))
.await
.map(|mut a| a.next())
{
Ok(Some(address)) => address,
Ok(None) => {
bp.build_error(
ObjectType::SpamPyzor.singleton(),
"Invalid address: No addresses found.",
);
return None;
}
Err(err) => {
bp.build_error(
ObjectType::SpamPyzor.singleton(),
format!("Invalid address: {}", err),
);
return None;
}
};
PyzorConfig {
address,
timeout: pyzor.timeout.into_inner(),
min_count: pyzor.block_count,
min_wl_count: pyzor.allow_count,
ratio: pyzor.ratio.into_inner(),
}
.into()
}
}
impl ClassifierConfig {
pub async fn parse(bp: &mut Bootstrap) -> Option<Self> {
let classifier = bp.setting_infallible::<structs::SpamClassifier>().await;
let (log_scale, l2_normalize, w_params, i_params) = match classifier.model {
structs::SpamClassifierModel::FtrlFh(model) => (
model.feature_log_scale,
model.feature_l2_normalize,
FtrlParameters::parse(&model.parameters),
None,
),
structs::SpamClassifierModel::FtrlCcfh(model) => (
model.feature_log_scale,
model.feature_l2_normalize,
FtrlParameters::parse(&model.parameters),
Some(FtrlParameters::parse(&model.indicator_parameters)),
),
structs::SpamClassifierModel::Disabled => return None,
};
ClassifierConfig {
w_params,
i_params,
reservoir_capacity: classifier.reservoir_capacity as usize,
auto_learn_card_is_ham: classifier.learn_ham_from_card,
auto_learn_reply_ham: classifier.learn_ham_from_reply,
auto_learn_spam_trap: classifier.learn_spam_from_traps,
auto_learn_spam_rbl_count: classifier.learn_spam_from_rbl_hits as u32,
hold_samples_for: classifier.hold_samples_for.into_inner().as_secs(),
min_ham_samples: classifier.min_ham_samples,
min_spam_samples: classifier.min_spam_samples,
train_frequency: classifier.train_frequency.map(|d| d.into_inner().as_secs()),
log_scale,
l2_normalize,
}
.into()
}
}
impl FtrlParameters {
pub fn parse(params: &structs::FtrlParameters) -> Self {
let hash_size = match params.num_features {
ModelSize::V16 => 16,
ModelSize::V17 => 17,
ModelSize::V18 => 18,
ModelSize::V19 => 19,
ModelSize::V20 => 20,
ModelSize::V21 => 21,
ModelSize::V22 => 22,
ModelSize::V23 => 23,
ModelSize::V24 => 24,
ModelSize::V25 => 25,
ModelSize::V26 => 26,
ModelSize::V27 => 27,
ModelSize::V28 => 28,
};
FtrlParameters {
feature_hash_size: 1 << hash_size,
alpha: params.alpha.into_inner(),
beta: params.beta.into_inner(),
l1_ratio: params.l1_ratio.into_inner(),
l2_ratio: params.l2_ratio.into_inner(),
}
}
}
impl SpamClassifier {
pub fn is_active(&self) -> bool {
!matches!(self, SpamClassifier::Disabled)
}
}
impl Location {
pub fn as_str(&self) -> &'static str {
match self {
Location::EnvelopeFrom => "env_from",
Location::EnvelopeTo => "env_to",
Location::HeaderDkimPass => "dkim_pass",
Location::HeaderReceived => "received",
Location::HeaderFrom => "from",
Location::HeaderReplyTo => "reply_to",
Location::HeaderSubject => "subject",
Location::HeaderTo => "to",
Location::HeaderCc => "cc",
Location::HeaderBcc => "bcc",
Location::HeaderMid => "message_id",
Location::HeaderDnt => "dnt",
Location::Ehlo => "ehlo",
Location::BodyText => "body_text",
Location::BodyHtml => "body_html",
Location::Attachment => "attachment",
Location::Tcp => "tcp",
}
}
}
impl Element {
pub fn as_str(&self) -> &'static str {
match self {
Element::Url => "url",
Element::Domain => "domain",
Element::Email => "email",
Element::Ip => "ip",
Element::Header => "header",
Element::Body => "body",
Element::Any => "any",
}
}
}
pub struct IpResolver {
ip: IpAddr,
ip_string: String,
reverse: String,
octets: Variable<'static>,
}
impl ResolveVariable for IpResolver {
fn resolve_variable(&self, variable: ExpressionVariable) -> Variable<'_> {
match variable {
ExpressionVariable::Ip | ExpressionVariable::Value => self.ip_string.as_str().into(),
ExpressionVariable::IpReverse => self.reverse.as_str().into(),
ExpressionVariable::Octets => self.octets.clone(),
ExpressionVariable::IsV4 => Variable::Integer(self.ip.is_ipv4() as _),
ExpressionVariable::IsV6 => Variable::Integer(self.ip.is_ipv6() as _),
_ => Variable::Integer(0),
}
}
fn resolve_global(&self, _: &str) -> Variable<'_> {
Variable::Integer(0)
}
}
impl IpResolver {
pub fn new(ip: IpAddr) -> Self {
Self {
ip_string: ip.to_string(),
reverse: ip.to_reverse_name(),
octets: Variable::Array(match ip {
IpAddr::V4(ipv4_addr) => ipv4_addr
.octets()
.iter()
.map(|o| Variable::Integer(*o as _))
.collect(),
IpAddr::V6(ipv6_addr) => ipv6_addr
.octets()
.iter()
.map(|o| Variable::Integer(*o as _))
.collect(),
}),
ip,
}
}
}
impl CacheItemWeight for IpResolver {
fn weight(&self) -> u64 {
(std::mem::size_of::<IpResolver>() + self.ip_string.len() + self.reverse.len()) as u64
}
}
impl<T> SpamFilterAction<T> {
pub fn as_score(&self) -> Option<&T> {
match self {
SpamFilterAction::Allow(value) => Some(value),
_ => None,
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn config(spam: f32, discard: f32, reject: f32) -> SpamFilterScoreConfig {
SpamFilterScoreConfig {
reject_threshold: reject,
discard_threshold: discard,
spam_threshold: spam,
}
}
#[test]
fn spam_percentage_defaults() {
let config = config(5.0, 0.0, 0.0);
for (score, expected) in [
(-10.0, 0),
(0.0, 0),
(0.5, 5),
(2.5, 25),
(4.9, 49),
(4.999, 49),
(5.0, 50),
(7.5, 75),
(9.9, 99),
(10.0, 100),
(50.0, 100),
] {
assert_eq!(config.spam_percentage(score), expected, "score {score}");
}
}
#[test]
fn spam_percentage_matches_is_spam() {
for config in [
config(5.0, 0.0, 0.0),
config(5.0, 20.0, 15.0),
config(1.0, 0.0, 3.0),
config(12.5, 25.0, 0.0),
config(0.0, 0.0, 0.0),
] {
for score in (-2000..=4000).map(|score| score as f32 / 100.0) {
assert_eq!(
config.spam_percentage(score) >= 50,
config.is_spam(score),
"score {score} with {config:?}"
);
}
}
}
#[test]
fn spam_percentage_ceiling_is_lowest_enabled_threshold() {
let reject_lowest = config(5.0, 20.0, 15.0);
assert_eq!(reject_lowest.spam_percentage(10.0), 75);
assert_eq!(reject_lowest.spam_percentage(15.0), 100);
let discard_only = config(5.0, 15.0, 0.0);
assert_eq!(discard_only.spam_percentage(10.0), 75);
let below_spam_threshold = config(5.0, 3.0, 0.0);
assert_eq!(below_spam_threshold.spam_percentage(7.5), 75);
}
#[test]
fn spam_status_from_percentage() {
assert!(matches!(spam_status(None), SpamStatus::Unknown));
assert!(matches!(spam_status(Some(0)), SpamStatus::Ham));
assert!(matches!(spam_status(Some(100)), SpamStatus::Spam));
assert!(matches!(
spam_status(Some(50)),
SpamStatus::MaybeSpam(fraction) if fraction == 0.5
));
}
}
crates/common/src/config/mod.rs
+342
View File
@@ -0,0 +1,342 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use std::io::Cursor;
use self::{mailstore::jmap::JmapConfig, smtp::SmtpConfig, storage::Storage};
use crate::{
Core, Network,
auth::oauth::config::OAuthConfig,
config::mailstore::{
email::EmailConfig, imap::ImapConfig, scripts::Scripting, spamfilter::SpamFilterConfig,
},
};
use arc_swap::ArcSwap;
use groupware::GroupwareConfig;
use hyper::HeaderMap;
use p256::elliptic_curve::sec1::ToEncodedPoint;
use pkcs8::EncodePrivateKey;
use rsa::{
RsaPrivateKey,
pkcs1::{DecodeRsaPrivateKey, EncodeRsaPrivateKey},
pkcs8::DecodePrivateKey as _,
traits::PublicKeyParts,
};
use store::registry::bootstrap::Bootstrap;
use telemetry::Metrics;
pub mod groupware;
pub mod inner;
pub mod mailstore;
pub mod network;
pub mod server;
pub mod smtp;
pub mod storage;
pub mod telemetry;
impl Core {
pub async fn parse(bp: &mut Bootstrap, mut storage: Storage) -> Self {
Self {
sieve: Scripting::parse(bp).await,
network: Network::parse(bp).await,
smtp: Box::pin(SmtpConfig::parse(bp)).await,
jmap: JmapConfig::parse(bp).await,
imap: ImapConfig::parse(bp).await,
oauth: OAuthConfig::parse(bp).await,
metrics: Metrics::parse(bp).await,
spam: SpamFilterConfig::parse(bp).await,
email: EmailConfig::parse(bp).await,
groupware: GroupwareConfig::parse(bp).await,
storage,
}
}
pub fn into_shared(self) -> ArcSwap<Self> {
ArcSwap::from_pointee(self)
}
}
const RSA_MIN_MODULUS_BITS: usize = 2048;
const RSA_MAX_MODULUS_BITS: usize = 8192;
fn no_key_found(pem: &str, expected: &str) -> String {
if pem.contains("ENCRYPTED PRIVATE KEY") || pem.contains("Proc-Type: 4,ENCRYPTED") {
format!(
"No usable {expected} private key found in PEM: the key is password-protected, \
which is not supported. Decrypt it first with 'openssl pkcs8 -topk8 -nocrypt'."
)
} else {
format!("No usable {expected} private key found in PEM")
}
}
pub struct RsaSigningKey {
pub pkcs1_der: Vec<u8>,
pub modulus: Vec<u8>,
pub exponent: Vec<u8>,
}
pub fn build_rsa_keypair(pem: &str) -> Result<RsaSigningKey, String> {
for item in rustls_pemfile::read_all(&mut Cursor::new(pem)) {
let key = match item.map_err(|err| format!("Failed to read private key PEM: {err}"))? {
rustls_pemfile::Item::Pkcs1Key(key) => {
RsaPrivateKey::from_pkcs1_der(key.secret_pkcs1_der())
.map_err(|err| format!("Failed to parse PKCS1 RSA key: {err}"))?
}
rustls_pemfile::Item::Pkcs8Key(key) => {
RsaPrivateKey::from_pkcs8_der(key.secret_pkcs8_der())
.map_err(|err| format!("Failed to parse PKCS8 RSA key: {err}"))?
}
_ => continue,
};
let bits = key.n().bits();
if !(RSA_MIN_MODULUS_BITS..=RSA_MAX_MODULUS_BITS).contains(&bits) {
return Err(format!(
"RSA key modulus is {bits} bits, expected between {RSA_MIN_MODULUS_BITS} and {RSA_MAX_MODULUS_BITS}"
));
}
let pkcs1_der = key
.to_pkcs1_der()
.map_err(|err| format!("Failed to encode RSA key as PKCS1: {err}"))?;
return Ok(RsaSigningKey {
pkcs1_der: pkcs1_der.as_bytes().to_vec(),
modulus: key.n().to_bytes_be(),
exponent: key.e().to_bytes_be(),
});
}
Err(no_key_found(pem, "RSA"))
}
#[derive(Clone, Copy)]
pub enum EcKeyCurve {
P256,
P384,
}
pub struct EcdsaSigningKey {
pub pkcs8_der: Vec<u8>,
pub x: Vec<u8>,
pub y: Vec<u8>,
}
pub fn build_ecdsa_pem(curve: EcKeyCurve, pem: &str) -> Result<EcdsaSigningKey, String> {
for item in rustls_pemfile::read_all(&mut Cursor::new(pem)) {
let pkcs8 = match item.map_err(|err| format!("Failed to read private key PEM: {err}"))? {
rustls_pemfile::Item::Pkcs8Key(key) => key.secret_pkcs8_der().to_vec(),
rustls_pemfile::Item::Sec1Key(key) => curve
.sec1_to_pkcs8(key.secret_sec1_der())?
.as_bytes()
.to_vec(),
_ => continue,
};
let (x, y) = curve.public_coordinates(&pkcs8)?;
return Ok(EcdsaSigningKey {
pkcs8_der: pkcs8,
x,
y,
});
}
Err(no_key_found(pem, "ECDSA"))
}
impl EcKeyCurve {
fn sec1_to_pkcs8(self, der: &[u8]) -> Result<pkcs8::SecretDocument, String> {
match self {
EcKeyCurve::P256 => p256::SecretKey::from_sec1_der(der)
.map_err(|err| format!("Failed to parse SEC1 ECDSA key: {err}"))?
.to_pkcs8_der()
.map_err(|err| format!("Failed to convert SEC1 ECDSA key to PKCS8: {err}")),
EcKeyCurve::P384 => p384::SecretKey::from_sec1_der(der)
.map_err(|err| format!("Failed to parse SEC1 ECDSA key: {err}"))?
.to_pkcs8_der()
.map_err(|err| format!("Failed to convert SEC1 ECDSA key to PKCS8: {err}")),
}
}
fn public_coordinates(self, pkcs8: &[u8]) -> Result<(Vec<u8>, Vec<u8>), String> {
use pkcs8::DecodePrivateKey;
match self {
EcKeyCurve::P256 => {
let point = p256::SecretKey::from_pkcs8_der(pkcs8)
.map_err(|err| format!("Failed to parse PKCS8 ECDSA key: {err}"))?
.public_key()
.to_encoded_point(false);
Ok((
point.x().map(|x| x.to_vec()).unwrap_or_default(),
point.y().map(|y| y.to_vec()).unwrap_or_default(),
))
}
EcKeyCurve::P384 => {
let point = p384::SecretKey::from_pkcs8_der(pkcs8)
.map_err(|err| format!("Failed to parse PKCS8 ECDSA key: {err}"))?
.public_key()
.to_encoded_point(false);
Ok((
point.x().map(|x| x.to_vec()).unwrap_or_default(),
point.y().map(|y| y.to_vec()).unwrap_or_default(),
))
}
}
}
}
#[cfg(test)]
mod tests {
use super::{EcKeyCurve, build_ecdsa_pem, build_rsa_keypair};
const P256_SEC1: &str = "-----BEGIN EC PRIVATE KEY-----
MHcCAQEEIJ9a6n/cu7XaQez5ZX8z8jDFkkfsMB1P9Vbqzbaes2zOoAoGCCqGSM49
AwEHoUQDQgAEPCbID7bo+8Nk1vIsTFhVKwRWvb9GWTzzwS75Dd8iZuFl23Twn6Sp
V2ZO1FC0WyXxcVOMZN2sJFlCjtaQS+p5Zg==
-----END EC PRIVATE KEY-----";
const P256_PKCS8: &str = "-----BEGIN PRIVATE KEY-----
MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgn1rqf9y7tdpB7Pll
fzPyMMWSR+wwHU/1VurNtp6zbM6hRANCAAQ8JsgPtuj7w2TW8ixMWFUrBFa9v0ZZ
PPPBLvkN3yJm4WXbdPCfpKlXZk7UULRbJfFxU4xk3awkWUKO1pBL6nlm
-----END PRIVATE KEY-----";
const P384_SEC1: &str = "-----BEGIN EC PRIVATE KEY-----
MIGkAgEBBDAeecJf8ju/70Nf5nbI4DeRo/+Z3VWXUvB+GwuUczew7fyMbyc6B3EE
BskOIqvqu6egBwYFK4EEACKhZANiAAQQjDW03Xn2h9ZmmCMRx+uRaLLfg4o2XITE
pwACH9EY4IjTe9LNNp5CTjERd+RlpWxkYopmDS5Trzycz9sDxxSzzXmq90vomJqt
fTnNHPFHuR2SAiwuzUf26rcPwa7DCWk=
-----END EC PRIVATE KEY-----";
const P384_PKCS8: &str = "-----BEGIN PRIVATE KEY-----
MIG2AgEAMBAGByqGSM49AgEGBSuBBAAiBIGeMIGbAgEBBDAeecJf8ju/70Nf5nbI
4DeRo/+Z3VWXUvB+GwuUczew7fyMbyc6B3EEBskOIqvqu6ehZANiAAQQjDW03Xn2
h9ZmmCMRx+uRaLLfg4o2XITEpwACH9EY4IjTe9LNNp5CTjERd+RlpWxkYopmDS5T
rzycz9sDxxSzzXmq90vomJqtfTnNHPFHuR2SAiwuzUf26rcPwa7DCWk=
-----END PRIVATE KEY-----";
#[test]
fn ecdsa_pem_accepts_sec1_and_pkcs8() {
let sec1 =
build_ecdsa_pem(EcKeyCurve::P256, P256_SEC1).expect("P-256 SEC1 key should parse");
let pkcs8 =
build_ecdsa_pem(EcKeyCurve::P256, P256_PKCS8).expect("P-256 PKCS8 key should parse");
assert_eq!((&sec1.x, &sec1.y), (&pkcs8.x, &pkcs8.y));
assert_eq!(sec1.x.len(), 32);
let sec1 =
build_ecdsa_pem(EcKeyCurve::P384, P384_SEC1).expect("P-384 SEC1 key should parse");
let pkcs8 =
build_ecdsa_pem(EcKeyCurve::P384, P384_PKCS8).expect("P-384 PKCS8 key should parse");
assert_eq!((&sec1.x, &sec1.y), (&pkcs8.x, &pkcs8.y));
assert_eq!(sec1.x.len(), 48);
}
const RSA_PKCS1: &str = "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEAt5Kp7o945bxbnvviI9Kzbjcwi6B5cStu4dBbNhe/ld0Js4tQ\n8Uq9qYaFBlocYzWkEd3e2IG0+uIVB37ewUe0xjq25u6G4ZWeK+SZgzXB4jHinXvh\nuhHW+KzbmO+aYO115451Cu6ymJ8DLVaR6zxT9CJsiS4lMsYZ5JHcLY3az1A5z0df\nF+chjR+sLxdc0ggKqnX6fT/sVXHIlVk6riyeFV929k/v1f0pmRQ2nNu0NMSOK7Mk\nqsvHiAb1e/41LIwlbmbzd5ASHitYYXKP+2YR29SRr2D+52S1M29h4/XbUcP6Zo2U\np5mKgQ0kFZ8pHFhbruamzRp87+yhu98IbZ9ksQIDAQABAoIBAAu2+BGxhbNReR5U\n8Co9krZEntw2NjHG5glSkNOLoe4IIEudJyHy1VYpb7lHTFr3bBw4xrUV1+0PuuxS\nyBfZAdwJmKz1iVWBhQnDiZliN5h9+vp2UqIba9bMPypMFhO766OGh4kWUP7k3ODK\njr7Oh4QDo14AvB54nmPj/ANLM2y50/Upy5s7FK0tm0ntzxSscwQFSZAJ9B0ne6Qe\nu1/PXgiXW4JKNOgrCTrRB2BcOi/Ke6OA/kg54sD+Z9PZivO/qHTx9xXzqivmbg9a\nGmoivaWH/pKwAywFogJnWH/iTe+r//fKdlEDeK+s/iCr0ht//c0w+GxPvPF//wz0\n+1u9+n0CgYEA5M7jzpT8rCYWdORvvRP1BC4+A5jb83zXW4FS+rZRSmq775zDPAif\npm653vAlNHIphEvqSdVw64+36nJFtjuBI17BHCQi0j3iNVjrLC7lbfqIobnNDdmR\n9VeqZ6qwPYt2oi4iBY2dAnPdYVTDMomHSC4vW/SER0l9A9bxt3Co1a0CgYEAzWOQ\n490s6K186CyUMFrNrUmIWEJNd7b6JGI+oCioZLtPZzxO4ebc+bHEPbpbSqx7lJRJ\nt5u6zw/RwUc+6YXXImekvMfZpZMH9v1wjp3djnxGQO4ucmvmu6H25qcYup8tRtlo\n2AVLd1jg3yka1yr7O26M3bhVfm5LOUQfoLuCA5UCgYEA3Iw7882SfFE+RjBHMIcD\nHqOALTFzmhDU+SQAGyAP3V5ihwWg/sYFNYT3btgl1JbSQ+51B/RQIw9mJPs/DPfw\nc2qLU5fVZLg3ylpKXU1a4xaiCtmwuM/mLAnzfHd/5+L9WDiFnLqzBEEwu/fbK2R7\nXOz/w3A+7QP+F+xhFAPpCgUCgYEAsnZOIkA/UlnUi6SYir+LsYOQLihGSbw687xN\n8DoDv6sl3mz/mbhQz8GP45b21hazNrH2r8xn8J0tRATU/HIoMaPe942rZvwv0oP6\n9mDjb3g6TxbmUtPA485iy53rldTTsZkdSX6oSSZ4FlAQG2AkdkqjqdAOsVHCmRrB\nZJco7FUCgYBwk7tQt3YS5b0wi8fH3BIfAH31vJ2VGlAin860H8FXjAj8EZ7Ff9Iq\n5dQIyPbp89TOSxIVxPGniI2ruLy4DZQM7xa42oyxyRir4UeHN2P5D2yEAHaVSwSd\nO6yiiOBj62OATapI8BqeFJZGRFltDsj6XbwC/Z9S2tRKCuE/zp+FLg==\n-----END RSA PRIVATE KEY-----";
const RSA_PKCS8: &str = "-----BEGIN PRIVATE KEY-----\nMIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQC3kqnuj3jlvFue\n++Ij0rNuNzCLoHlxK27h0Fs2F7+V3Qmzi1DxSr2phoUGWhxjNaQR3d7YgbT64hUH\nft7BR7TGOrbm7obhlZ4r5JmDNcHiMeKde+G6Edb4rNuY75pg7XXnjnUK7rKYnwMt\nVpHrPFP0ImyJLiUyxhnkkdwtjdrPUDnPR18X5yGNH6wvF1zSCAqqdfp9P+xVcciV\nWTquLJ4VX3b2T+/V/SmZFDac27Q0xI4rsySqy8eIBvV7/jUsjCVuZvN3kBIeK1hh\nco/7ZhHb1JGvYP7nZLUzb2Hj9dtRw/pmjZSnmYqBDSQVnykcWFuu5qbNGnzv7KG7\n3whtn2SxAgMBAAECggEAC7b4EbGFs1F5HlTwKj2StkSe3DY2McbmCVKQ04uh7ggg\nS50nIfLVVilvuUdMWvdsHDjGtRXX7Q+67FLIF9kB3AmYrPWJVYGFCcOJmWI3mH36\n+nZSohtr1sw/KkwWE7vro4aHiRZQ/uTc4MqOvs6HhAOjXgC8HnieY+P8A0szbLnT\n9SnLmzsUrS2bSe3PFKxzBAVJkAn0HSd7pB67X89eCJdbgko06CsJOtEHYFw6L8p7\no4D+SDniwP5n09mK87+odPH3FfOqK+ZuD1oaaiK9pYf+krADLAWiAmdYf+JN76v/\n98p2UQN4r6z+IKvSG3/9zTD4bE+88X//DPT7W736fQKBgQDkzuPOlPysJhZ05G+9\nE/UELj4DmNvzfNdbgVL6tlFKarvvnMM8CJ+mbrne8CU0cimES+pJ1XDrj7fqckW2\nO4EjXsEcJCLSPeI1WOssLuVt+oihuc0N2ZH1V6pnqrA9i3aiLiIFjZ0Cc91hVMMy\niYdILi9b9IRHSX0D1vG3cKjVrQKBgQDNY5Dj3SzorXzoLJQwWs2tSYhYQk13tvok\nYj6gKKhku09nPE7h5tz5scQ9ultKrHuUlEm3m7rPD9HBRz7phdciZ6S8x9mlkwf2\n/XCOnd2OfEZA7i5ya+a7ofbmpxi6ny1G2WjYBUt3WODfKRrXKvs7bozduFV+bks5\nRB+gu4IDlQKBgQDcjDvzzZJ8UT5GMEcwhwMeo4AtMXOaENT5JAAbIA/dXmKHBaD+\nxgU1hPdu2CXUltJD7nUH9FAjD2Yk+z8M9/BzaotTl9VkuDfKWkpdTVrjFqIK2bC4\nz+YsCfN8d3/n4v1YOIWcurMEQTC799srZHtc7P/DcD7tA/4X7GEUA+kKBQKBgQCy\ndk4iQD9SWdSLpJiKv4uxg5AuKEZJvDrzvE3wOgO/qyXebP+ZuFDPwY/jlvbWFrM2\nsfavzGfwnS1EBNT8cigxo973jatm/C/Sg/r2YONveDpPFuZS08DjzmLLneuV1NOx\nmR1JfqhJJngWUBAbYCR2SqOp0A6xUcKZGsFklyjsVQKBgHCTu1C3dhLlvTCLx8fc\nEh8AffW8nZUaUCKfzrQfwVeMCPwRnsV/0irl1AjI9unz1M5LEhXE8aeIjau4vLgN\nlAzvFrjajLHJGKvhR4c3Y/kPbIQAdpVLBJ07rKKI4GPrY4BNqkjwGp4UlkZEWW0O\nyPpdvAL9n1La1EoK4T/On4Uu\n-----END PRIVATE KEY-----";
#[test]
fn rsa_pem_accepts_pkcs1_and_pkcs8() {
let a = build_rsa_keypair(RSA_PKCS1).expect("PKCS1 RSA key should parse");
let b = build_rsa_keypair(RSA_PKCS8).expect("PKCS8 RSA key should parse");
assert_eq!(a.modulus, b.modulus);
assert_eq!(a.exponent, b.exponent);
assert_eq!(a.pkcs1_der, b.pkcs1_der);
assert_eq!(a.modulus.len(), 256);
}
#[test]
fn signing_keys_are_accepted_by_jsonwebtoken() {
use jsonwebtoken::{Algorithm, EncodingKey, Header};
#[derive(serde::Serialize)]
struct Claims {
sub: &'static str,
}
let claims = Claims { sub: "test" };
for (curve, pem, alg) in [
(EcKeyCurve::P256, P256_SEC1, Algorithm::ES256),
(EcKeyCurve::P256, P256_PKCS8, Algorithm::ES256),
(EcKeyCurve::P384, P384_SEC1, Algorithm::ES384),
(EcKeyCurve::P384, P384_PKCS8, Algorithm::ES384),
] {
let key = build_ecdsa_pem(curve, pem).expect("key should parse");
jsonwebtoken::encode(
&Header::new(alg),
&claims,
&EncodingKey::from_ec_der(&key.pkcs8_der),
)
.unwrap_or_else(|err| panic!("{alg:?} signing failed: {err}"));
}
for pem in [RSA_PKCS1, RSA_PKCS8] {
let key = build_rsa_keypair(pem).expect("key should parse");
for alg in [Algorithm::RS256, Algorithm::PS512] {
jsonwebtoken::encode(
&Header::new(alg),
&claims,
&EncodingKey::from_rsa_der(&key.pkcs1_der),
)
.unwrap_or_else(|err| panic!("{alg:?} signing failed: {err}"));
}
}
}
#[test]
fn ecdsa_pem_rejects_keyless_pem() {
let err = match build_ecdsa_pem(
EcKeyCurve::P256,
"-----BEGIN CERTIFICATE-----\nMIIB\n-----END CERTIFICATE-----",
) {
Ok(_) => panic!("expected a certificate-only PEM to be rejected"),
Err(err) => err,
};
assert!(err.contains("No usable ECDSA private key"), "{err}");
}
const P256_ENCRYPTED: &str = "-----BEGIN ENCRYPTED PRIVATE KEY-----\nMIH0MF8GCSqGSIb3DQEFDTBSMDEGCSqGSIb3DQEFDDAkBBCoW4qsep9YbFLRW2u4\nk8ljAgIIADAMBggqhkiG9w0CCQUAMB0GCWCGSAFlAwQBKgQQZYoEYHqh+y9uqT70\n6521jwSBkA9dcdq6hT/7Fzqcu0wX3QVr+8g1Kxc6tCV9dLShi8VU2ax8jG3zZt3h\nBp1CLyX8UfT98SujtoH36PEXPDDTralcP6vWViqGx5AagT4DRFjcI8yucTUXkLoD\n9ZIRBVPviTeznEHt3OvCCMuO76rsyu/gxNC7D46TBtq8JX1OFcaXPctpN8l5GKqH\nh4gA6av3og==\n-----END ENCRYPTED PRIVATE KEY-----";
#[test]
fn ecdsa_pem_reports_password_protected_key() {
let err = match build_ecdsa_pem(EcKeyCurve::P256, P256_ENCRYPTED) {
Ok(_) => panic!("expected an encrypted key to be rejected"),
Err(err) => err,
};
assert!(err.contains("password-protected"), "{err}");
}
#[test]
fn rsa_pem_rejects_undersized_modulus() {
const RSA_1024: &str = "-----BEGIN RSA PRIVATE KEY-----
MIICXAIBAAKBgQCz6gcAg0f+2/HFrudtMfRSylyzI8W/lNmPQZhUpz+R6D/7/+4/
HFKsUcZIi+nzdOnrzW/kw19nVOKk2ylAUNV9d2TR75HqrPBYsu0LCDRidb9XOyhY
bQJII1KuFlaWNjfxG28Tlg//FdVPPkn/oTQwnhvMcWCK3Hatho6cx9uzWwIDAQAB
AoGBAIPvjwr1OwrOyFIrnVMaWw2LkMdd6FpCEflYJRmPPLMHGkT2vgRSBN6RaVMy
J3J9vj1J/lBIZeIlAb/baDjeDnAj5GBzCB319oxnBuZSmpyYntW1DEsdhbK0Yeu+
7v05oXBXfzdZvGBWYrwlj5ipoHQo0R+WN4NVXqJFwiagaGBBAkEA6qwv5Pww54za
fHNUD1M6MKRBk5m0Y/GJ58sWmnmFJI6I3sHBIfcy5lylm5KecduzSKoVtAUUbNWf
KOKoZcKHMwJBAMRD2WDEd5+8q5ZxzYG0x5sEdz1lhJkt+YSbudNgfE1kPDDrCE0V
8+hgNdp6Mj1hfihwB0hTCcnaPsXLl9AyAzkCQDVU+HWD0uFso2LRGvN4qKrRSY3v
yo1EIWEqSHLG1zldo0FsqyW69jhgKcrXYWbi1TXYYaJN3Tx2t/skt7yYnv0CQDtD
NYdHq8tbAADcei5ZNRB058BtP/206SwGjbTq5H3F73rh7U7BezXGn1xKG5N3Nc3m
DfzjvgfqU5wMHtopz9kCQBw3AAiRCY8Y0UgejUtu8tXIK76qebaNcMCabBnFrAqV
A4Oj1c5BcOHVtww9W6NeiiRMJpUNN71gmyjsnOyT3cY=
-----END RSA PRIVATE KEY-----";
let err = match build_rsa_keypair(RSA_1024) {
Ok(_) => panic!("expected a 1024-bit RSA key to be rejected"),
Err(err) => err,
};
assert!(err.contains("1024 bits"), "{err}");
}
}
crates/common/src/config/network.rs
+555
View File
@@ -0,0 +1,555 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use super::*;
use crate::{
expr::if_block::{BootstrapExprExt, IfBlock},
network::{
autoconfig::pacc::{
Authentication, Configuration, HttpServer, Info, Logo, OAuthPublic, Protocols,
Provider, TextServer,
},
security::Security,
},
};
use mail_builder::mime::make_boundary;
use registry::schema::{
enums::{AcmeChallengeType, ClusterTaskType, ProviderInfo, ServiceProtocol},
prelude::{ObjectType, Property},
structs::{
self, AcmeProvider, Asn, ClusterTaskGroup, HttpForm, MailExchanger, Rate, Service,
SystemSettings, TaskManager,
},
};
use std::{str::FromStr, time::Duration};
use utils::map::vec_map::VecMap;
#[derive(Clone)]
pub struct Network {
pub node_id: u64,
pub roles: ClusterRoles,
pub server_name: String,
pub security: Security,
pub http: Http,
pub contact_form: Option<ContactForm>,
pub asn_geo_lookup: AsnGeoLookupConfig,
pub task_manager: TaskManager,
pub has_acme_tls_challenge: bool,
pub has_acme_http_challenge: bool,
pub info: NetworkInfo,
}
#[derive(Clone)]
pub struct NetworkInfo {
pub pacc: Pacc,
pub mxs: Vec<MailExchanger>,
pub services: VecMap<ServiceProtocol, Service>,
}
#[derive(Clone)]
pub struct Pacc {
pub prefix: String,
pub suffix: String,
}
#[derive(Clone)]
pub struct Http {
pub rate_authenticated: Option<Rate>,
pub rate_anonymous: Option<Rate>,
pub url_https: String,
pub allowed_endpoint: IfBlock,
pub response_headers: Vec<(hyper::header::HeaderName, hyper::header::HeaderValue)>,
pub use_forwarded: bool,
pub redirect_root: Option<String>,
}
#[derive(Clone)]
pub struct ContactForm {
pub rcpt_to: Vec<String>,
pub max_size: usize,
pub rate: Option<Rate>,
pub validate_domain: bool,
pub from_email: FieldOrDefault,
pub from_subject: FieldOrDefault,
pub from_name: FieldOrDefault,
pub field_honey_pot: Option<String>,
}
#[derive(Clone)]
pub struct ClusterRoles {
pub store_maintenance: bool,
pub account_maintenance: bool,
pub push_notifications: bool,
pub search_indexing: bool,
pub spam_training: bool,
pub metrics_calculate: bool,
pub metrics_push: bool,
pub outbound_mta: bool,
pub task_scheduler: bool,
pub task_manager: bool,
}
#[derive(Clone, Default)]
pub enum AsnGeoLookupConfig {
Resource {
expires: Duration,
timeout: Duration,
max_size: usize,
headers: HeaderMap,
asn_resources: Vec<String>,
geo_resources: Vec<String>,
},
Dns {
zone_ipv4: String,
zone_ipv6: String,
separator: String,
index_asn: usize,
index_asn_name: Option<usize>,
index_country: Option<usize>,
},
#[default]
Disabled,
}
#[derive(Clone)]
pub struct FieldOrDefault {
pub field: Option<String>,
pub default: String,
}
impl ContactForm {
pub async fn parse(bp: &mut Bootstrap) -> Option<Self> {
let form = bp.setting_infallible::<HttpForm>().await;
if !form.enable {
return None;
} else if form.deliver_to.is_empty() {
bp.build_error(
ObjectType::HttpForm.singleton(),
"Contact form is enabled but no recipient addresses are configured",
);
return None;
}
Some(ContactForm {
rcpt_to: form.deliver_to.into_inner(),
max_size: form.max_size as usize,
validate_domain: form.validate_domain,
from_email: FieldOrDefault {
field: form.field_email,
default: form.default_from_address,
},
from_subject: FieldOrDefault {
field: form.field_subject,
default: form.default_subject,
},
from_name: FieldOrDefault {
field: form.field_name,
default: form.default_name,
},
field_honey_pot: form.field_honey_pot,
rate: form.rate_limit,
})
}
}
impl Network {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let system = bp.setting_infallible::<SystemSettings>().await;
let mut has_acme_tls_challenge = false;
let mut has_acme_http_challenge = false;
let mut has_acme_challenges = false;
for provider in bp.list_infallible::<AcmeProvider>().await {
match provider.object.challenge_type {
AcmeChallengeType::Http01 => has_acme_http_challenge = true,
AcmeChallengeType::TlsAlpn01 => has_acme_tls_challenge = true,
_ => {}
}
has_acme_challenges = true;
}
if !has_acme_challenges {
// Assume this is an initial deployment and optimistically set both to true
// to avoid requiring a reload after ACME providers are added
has_acme_http_challenge = true;
has_acme_tls_challenge = true;
}
const SPLIT_HERE: &str = "$$__SPLIT_HERE__$$";
let mut pacc = Configuration {
protocols: Protocols::default(),
authentication: Some(Authentication {
oauth_public: Some(OAuthPublic {
issuer: SPLIT_HERE.to_string(),
}),
password: true,
}),
info: Info {
provider: Provider {
name: "Stalwart".into(),
..Default::default()
},
..Default::default()
},
};
let default_hostname = if !system.default_hostname.is_empty() {
system.default_hostname.as_str()
} else {
bp.registry.local_hostname()
};
let mut http_host = default_hostname.to_string();
for (service, details) in &system.services {
let hostname = details.hostname.as_deref().unwrap_or(default_hostname);
match service {
ServiceProtocol::Jmap => {
if hostname != http_host {
http_host = hostname.to_string();
}
pacc.protocols.jmap = HttpServer {
url: format!("https://{hostname}/jmap/session",),
}
.into();
}
ServiceProtocol::Caldav => {
pacc.protocols.caldav = HttpServer {
url: format!("https://{hostname}/dav/cal/",),
}
.into();
}
ServiceProtocol::Carddav => {
pacc.protocols.carddav = HttpServer {
url: format!("https://{hostname}/dav/card/",),
}
.into();
}
ServiceProtocol::Webdav => {
pacc.protocols.webdav = HttpServer {
url: format!("https://{hostname}/dav/file/",),
}
.into();
}
ServiceProtocol::Imap => {
pacc.protocols.imap = TextServer {
host: hostname.to_string(),
}
.into();
}
ServiceProtocol::Pop3 => {
pacc.protocols.pop3 = TextServer {
host: hostname.to_string(),
}
.into();
}
ServiceProtocol::Smtp => {
pacc.protocols.smtp = TextServer {
host: hostname.to_string(),
}
.into();
}
ServiceProtocol::Managesieve => {
pacc.protocols.managesieve = TextServer {
host: hostname.to_string(),
}
.into();
}
}
}
for (tag, text) in system.provider_info {
match tag {
ProviderInfo::ProviderName => pacc.info.provider.name = text,
ProviderInfo::ProviderShortName => pacc.info.provider.short_name = Some(text),
ProviderInfo::UserDocumentation => {
pacc.info.help.get_or_insert_default().documentation = Some(text)
}
ProviderInfo::DeveloperDocumentation => {
pacc.info.help.get_or_insert_default().developer = Some(text)
}
ProviderInfo::ContactUri => {
pacc.info
.help
.get_or_insert_default()
.contact
.get_or_insert_default()
.push(text);
}
ProviderInfo::LogoUrl => {
let logo = pacc.info.provider.logo.get_or_insert_default();
if logo.is_empty() {
logo.push(Logo {
url: text,
..Default::default()
});
} else {
logo[0].url = text;
}
}
ProviderInfo::LogoWidth => {
let logo = pacc.info.provider.logo.get_or_insert_default();
if logo.is_empty() {
logo.push(Logo {
width: text.parse().ok(),
..Default::default()
});
} else {
logo[0].width = text.parse().ok();
}
}
ProviderInfo::LogoHeight => {
let logo = pacc.info.provider.logo.get_or_insert_default();
if logo.is_empty() {
logo.push(Logo {
height: text.parse().ok(),
..Default::default()
});
} else {
logo[0].height = text.parse().ok();
}
}
}
}
let (prefix, suffix) = serde_json::to_string(&pacc)
.unwrap_or_default()
.rsplit_once(SPLIT_HERE)
.map(|(prefix, suffix)| (prefix.to_string(), suffix.to_string()))
.unwrap();
let mut network = Network {
node_id: bp.node_id() as u64,
server_name: default_hostname.to_string(),
security: Security::parse(bp).await,
contact_form: ContactForm::parse(bp).await,
asn_geo_lookup: AsnGeoLookupConfig::parse(bp).await.unwrap_or_default(),
roles: ClusterRoles::default(),
http: Http::parse(bp, &http_host).await,
task_manager: bp.setting_infallible::<TaskManager>().await,
has_acme_tls_challenge,
has_acme_http_challenge,
info: NetworkInfo {
mxs: system.mail_exchangers.into_iter().collect(),
services: system.services,
pacc: Pacc { prefix, suffix },
},
};
if let Some(role) = &bp.role {
match &role.tasks {
ClusterTaskGroup::EnableAll => {}
ClusterTaskGroup::DisableAll => {
for network_role in network.roles.all_mut() {
*network_role = false;
}
}
ClusterTaskGroup::EnableSome(group) => {
for network_role in network.roles.all_mut() {
*network_role = false;
}
for task_type in group.task_types.iter() {
network.roles.set_role(*task_type, true);
}
}
ClusterTaskGroup::DisableSome(group) => {
for task_type in group.task_types.iter() {
network.roles.set_role(*task_type, false);
}
}
}
}
network
}
pub fn message_id(&self) -> String {
format!("{}@{}", make_boundary("."), self.server_name)
}
}
impl Http {
#[cfg_attr(
any(feature = "dev_mode", feature = "test_mode"),
allow(unused_variables)
)]
pub async fn parse(bp: &mut Bootstrap, server_name: &str) -> Self {
let http = bp.setting_infallible::<structs::Http>().await;
// Parse HTTP headers
let mut http_headers = http
.response_headers
.iter()
.map(|(k, v)| {
Ok((
hyper::header::HeaderName::from_str(k.trim()).map_err(|err| {
format!("Invalid header found in property \"http.headers\": {}", err)
})?,
hyper::header::HeaderValue::from_str(v.trim()).map_err(|err| {
format!("Invalid header found in property \"http.headers\": {}", err)
})?,
))
})
.collect::<Result<Vec<_>, String>>()
.map_err(|e| {
bp.build_error(
ObjectType::Http.singleton(),
format!("Failed to parse HTTP headers: {}", e),
)
})
.unwrap_or_default();
// Add permissive CORS headers
#[cfg(feature = "dev_mode")]
let use_permissive_cors = true;
#[cfg(not(feature = "dev_mode"))]
let use_permissive_cors = http.use_permissive_cors || bp.registry.is_recovery_mode();
if use_permissive_cors {
http_headers.push((
hyper::header::ACCESS_CONTROL_ALLOW_ORIGIN,
hyper::header::HeaderValue::from_static("*"),
));
http_headers.push((
hyper::header::ACCESS_CONTROL_ALLOW_HEADERS,
hyper::header::HeaderValue::from_static(
"Authorization, Content-Type, Accept, X-Requested-With",
),
));
http_headers.push((
hyper::header::ACCESS_CONTROL_ALLOW_METHODS,
hyper::header::HeaderValue::from_static(
"POST, GET, PATCH, PUT, DELETE, HEAD, OPTIONS",
),
));
}
// Add HTTP Strict Transport Security
if http.enable_hsts {
http_headers.push((
hyper::header::STRICT_TRANSPORT_SECURITY,
hyper::header::HeaderValue::from_static("max-age=31536000; includeSubDomains"),
));
}
#[cfg(any(feature = "dev_mode", feature = "test_mode"))]
let server_name = "127.0.0.1";
Http {
url_https: if !bp.registry.is_recovery_mode() {
if let Some(url) = bp.registry.public_url() {
url.to_string()
} else {
format!("https://{server_name}")
}
} else {
String::new()
},
allowed_endpoint: if bp.registry.is_recovery_mode() {
IfBlock::empty(ObjectType::Http.singleton(), Property::AllowedEndpoints)
} else {
bp.compile_expr(ObjectType::Http.singleton(), &http.ctx_allowed_endpoints())
},
rate_authenticated: if bp.registry.is_recovery_mode() {
None
} else {
http.rate_limit_authenticated
},
rate_anonymous: if bp.registry.is_recovery_mode() {
None
} else {
http.rate_limit_anonymous
},
response_headers: http_headers,
use_forwarded: http.use_x_forwarded,
redirect_root: http.redirect_root,
}
}
}
impl AsnGeoLookupConfig {
pub async fn parse(bp: &mut Bootstrap) -> Option<Self> {
match bp.setting_infallible::<Asn>().await {
Asn::Resource(asn) => Some(AsnGeoLookupConfig::Resource {
expires: asn.expires.into_inner(),
timeout: asn.timeout.into_inner(),
max_size: asn.max_size as usize,
headers: asn
.http_auth
.build_headers(asn.http_headers, None)
.await
.map_err(|err| {
bp.build_error(
ObjectType::Asn.singleton(),
format!("Unable to build HTTP headers: {}", err),
)
})
.ok()?,
asn_resources: asn.asn_urls.into_inner(),
geo_resources: asn.geo_urls.into_inner(),
}),
Asn::Dns(asn) => Some(AsnGeoLookupConfig::Dns {
zone_ipv4: asn.zone_ip_v4,
zone_ipv6: asn.zone_ip_v6,
separator: asn.separator,
index_asn: asn.index_asn as usize,
index_asn_name: asn.index_asn_name.map(|v| v as usize),
index_country: asn.index_country.map(|v| v as usize),
}),
Asn::Disabled => None,
}
}
}
impl ClusterRoles {
fn all_mut(&mut self) -> impl Iterator<Item = &mut bool> {
[
&mut self.store_maintenance,
&mut self.account_maintenance,
&mut self.push_notifications,
&mut self.search_indexing,
&mut self.spam_training,
&mut self.outbound_mta,
&mut self.task_manager,
&mut self.task_scheduler,
&mut self.metrics_calculate,
&mut self.metrics_push,
]
.into_iter()
}
fn set_role(&mut self, role: ClusterTaskType, enabled: bool) {
match role {
ClusterTaskType::StoreMaintenance => self.store_maintenance = enabled,
ClusterTaskType::AccountMaintenance => self.account_maintenance = enabled,
ClusterTaskType::PushNotifications => self.push_notifications = enabled,
ClusterTaskType::SearchIndexing => self.search_indexing = enabled,
ClusterTaskType::SpamClassifierTraining => self.spam_training = enabled,
ClusterTaskType::MetricsCalculate => self.metrics_calculate = enabled,
ClusterTaskType::MetricsPush => self.metrics_push = enabled,
ClusterTaskType::OutboundMta => self.outbound_mta = enabled,
ClusterTaskType::TaskQueueProcessing => self.task_manager = enabled,
ClusterTaskType::TaskScheduler => self.task_scheduler = enabled,
}
}
}
impl Default for ClusterRoles {
fn default() -> Self {
ClusterRoles {
store_maintenance: true,
account_maintenance: true,
push_notifications: true,
search_indexing: true,
spam_training: true,
metrics_calculate: true,
metrics_push: true,
outbound_mta: true,
task_manager: true,
task_scheduler: true,
}
}
}
crates/common/src/config/server/listener.rs
+349
View File
@@ -0,0 +1,349 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use super::{
DEFAULT_TLS_TIMEOUT, Listener, Listeners, ServerProtocol, TcpListener,
tls::{TLS12_VERSION, TLS13_VERSION},
};
use crate::{
Inner,
network::{TcpAcceptor, tls::CertificateResolver},
};
use registry::{
schema::{
enums::{NetworkListenerProtocol, TlsCipherSuite, TlsVersion},
prelude::{ObjectType, SocketAddr},
structs::{ClusterListenerGroup, NetworkListener, SystemSettings},
},
types::{id::ObjectId, map::Map},
};
use rustls::{
ALL_VERSIONS, ServerConfig, SupportedCipherSuite,
crypto::aws_lc_rs::{ALL_CIPHER_SUITES, cipher_suite::*, default_provider},
};
use std::{
net::{IpAddr, Ipv4Addr, SocketAddr as StdSocketAddr},
str::FromStr,
sync::Arc,
};
use store::registry::{RegistryObject, bootstrap::Bootstrap};
use tokio::net::TcpSocket;
use tokio_rustls::TlsAcceptor;
use types::id::Id;
use utils::snowflake::SnowflakeIdGenerator;
impl Listeners {
pub async fn parse(bp: &mut Bootstrap) -> Self {
// Parse ACME managers
let mut servers = Listeners {
span_id_gen: Arc::new(SnowflakeIdGenerator::new()),
..Default::default()
};
// Parse servers
if !bp.registry.is_recovery_mode() {
let system = bp.setting_infallible::<SystemSettings>().await;
for listener in bp.list_infallible::<NetworkListener>().await {
if bp.role.as_ref().is_none_or(|r| match &r.listeners {
ClusterListenerGroup::EnableAll => true,
ClusterListenerGroup::DisableAll => false,
ClusterListenerGroup::EnableSome(group) => {
group.listener_ids.iter().any(|id| *id == listener.id.id())
}
ClusterListenerGroup::DisableSome(group) => {
!group.listener_ids.iter().any(|id| *id == listener.id.id())
}
}) {
servers.parse_server(bp, listener, &system);
}
}
} else {
servers.parse_server(
bp,
RegistryObject {
id: ObjectId::new(ObjectType::NetworkListener, Id::singleton()),
object: NetworkListener {
bind: Map::new(vec![
SocketAddr::from_str(&format!(
"[::]:{}",
std::env::var("STALWART_RECOVERY_MODE_PORT")
.ok()
.and_then(|p| p.parse::<u16>().ok())
.unwrap_or(8080)
))
.unwrap(),
]),
name: "http-recovery".to_string(),
protocol: NetworkListenerProtocol::Http,
tls_implicit: false,
..Default::default()
},
revision: 0,
},
&SystemSettings::default(),
);
}
servers
}
pub fn parse_server(
&mut self,
bp: &mut Bootstrap,
listener: RegistryObject<NetworkListener>,
system: &SystemSettings,
) {
let id = listener.id;
let revision = listener.revision;
let listener = listener.object;
// Parse protocol
let protocol = match listener.protocol {
NetworkListenerProtocol::Smtp => ServerProtocol::Smtp,
NetworkListenerProtocol::Lmtp => ServerProtocol::Lmtp,
NetworkListenerProtocol::Http => ServerProtocol::Http,
NetworkListenerProtocol::Imap => ServerProtocol::Imap,
NetworkListenerProtocol::Pop3 => ServerProtocol::Pop3,
NetworkListenerProtocol::ManageSieve => ServerProtocol::ManageSieve,
};
// Build listeners
let mut listeners = Vec::new();
for addr in listener.bind.iter() {
// Parse bind address and build socket
let mut addr = addr.0;
let socket = match if addr.is_ipv4() {
TcpSocket::new_v4()
} else {
TcpSocket::new_v6()
} {
Ok(socket) => socket,
Err(err)
if is_ipv6_unsupported(&err)
&& addr.is_ipv6()
&& addr.ip().is_unspecified() =>
{
let v4_addr =
StdSocketAddr::new(IpAddr::V4(Ipv4Addr::UNSPECIFIED), addr.port());
bp.build_warning(
id,
format!(
"IPv6 unavailable on this host ({err}); \
falling back from {addr} to {v4_addr}"
),
);
addr = v4_addr;
match TcpSocket::new_v4() {
Ok(socket) => socket,
Err(err) => {
bp.build_error(
id,
format!("Failed to create IPv4 fallback socket: {err}"),
);
return;
}
}
}
Err(err) => {
bp.build_error(id, format!("Failed to create socket: {err}"));
return;
}
};
#[cfg(windows)]
if addr.is_ipv6()
&& addr.ip().is_unspecified()
&& let Err(err) = socket2::SockRef::from(&socket).set_only_v6(false)
{
bp.build_warning(
id,
format!(
"Failed to disable IPV6_V6ONLY on {addr} ({err}); \
IPv4 clients will not be able to connect to this listener"
),
);
}
if let Err(err) = socket.set_reuseaddr(listener.socket_reuse_address) {
bp.build_error(id, format!("Failed to set SO_REUSEADDR: {err}"));
return;
}
#[cfg(not(target_env = "msvc"))]
if let Err(err) = socket.set_reuseport(listener.socket_reuse_port) {
bp.build_error(id, format!("Failed to set SO_REUSEPORT: {err}"));
return;
}
if let Some(send_size) = listener.socket_send_buffer_size
&& let Err(err) = socket.set_send_buffer_size(send_size as u32)
{
bp.build_error(id, format!("Failed to set SO_SNDBUF: {err}"));
return;
}
if let Some(recv_size) = listener.socket_receive_buffer_size
&& let Err(err) = socket.set_recv_buffer_size(recv_size as u32)
{
bp.build_error(id, format!("Failed to set SO_RCVBUF: {err}"));
return;
}
if let Some(tos) = listener.socket_tos_v4
&& let Err(err) = socket.set_tos_v4(tos as u32)
{
bp.build_error(id, format!("Failed to set IP_TOS: {err}"));
return;
}
listeners.push(TcpListener {
socket,
addr,
ttl: listener.socket_ttl.map(|v| v as u32),
backlog: listener.socket_backlog.map(|v| v as u32),
nodelay: listener.socket_no_delay,
});
}
let span_id_gen = self.span_id_gen.clone();
self.servers.push(Listener {
max_connections: listener.max_connections.unwrap_or(system.max_connections),
tls_timeout: listener
.tls_timeout
.map_or(DEFAULT_TLS_TIMEOUT, |timeout| timeout.into_inner()),
id: listener.name.clone(),
registry_id: id,
protocol,
listeners,
proxy_networks: if !listener.override_proxy_trusted_networks.is_empty() {
listener.override_proxy_trusted_networks.as_slice().to_vec()
} else {
system.proxy_trusted_networks.as_slice().to_vec()
},
span_id_gen,
});
self.parsed_listeners.push(RegistryObject {
id,
object: listener,
revision,
});
}
pub async fn parse_tcp_acceptors(&mut self, bp: &mut Bootstrap, inner: Arc<Inner>) {
let resolver = Arc::new(CertificateResolver::new(inner.clone()));
for listener in std::mem::take(&mut self.parsed_listeners) {
let id = listener.id;
let listener = listener.object;
// Build TLS config
let acceptor = if listener.use_tls {
// Parse protocol versions
let mut tls_v2 = true;
let mut tls_v3 = true;
for disabled in listener.tls_disable_protocols {
match disabled {
TlsVersion::Tls12 => {
tls_v2 = false;
}
TlsVersion::Tls13 => {
tls_v3 = false;
}
}
}
// Parse cipher suites
let mut disabled_ciphers: Vec<SupportedCipherSuite> = Vec::new();
for disabled in listener.tls_disable_cipher_suites {
disabled_ciphers.push(match disabled {
TlsCipherSuite::Tls13Aes256GcmSha384 => TLS13_AES_256_GCM_SHA384,
TlsCipherSuite::Tls13Aes128GcmSha256 => TLS13_AES_128_GCM_SHA256,
TlsCipherSuite::Tls13Chacha20Poly1305Sha256 => {
TLS13_CHACHA20_POLY1305_SHA256
}
TlsCipherSuite::TlsEcdheEcdsaWithAes256GcmSha384 => {
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
}
TlsCipherSuite::TlsEcdheEcdsaWithAes128GcmSha256 => {
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
}
TlsCipherSuite::TlsEcdheEcdsaWithChacha20Poly1305Sha256 => {
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
}
TlsCipherSuite::TlsEcdheRsaWithAes256GcmSha384 => {
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
}
TlsCipherSuite::TlsEcdheRsaWithAes128GcmSha256 => {
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
}
TlsCipherSuite::TlsEcdheRsaWithChacha20Poly1305Sha256 => {
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
}
});
}
// Build cert provider
let mut provider = default_provider();
if !disabled_ciphers.is_empty() {
provider.cipher_suites = ALL_CIPHER_SUITES
.iter()
.filter(|suite| !disabled_ciphers.contains(suite))
.copied()
.collect();
}
// Build server config
let mut server_config = match ServerConfig::builder_with_provider(provider.into())
.with_protocol_versions(if tls_v3 == tls_v2 {
ALL_VERSIONS
} else if tls_v3 {
TLS13_VERSION
} else {
TLS12_VERSION
}) {
Ok(server_config) => server_config
.with_no_client_auth()
.with_cert_resolver(resolver.clone()),
Err(err) => {
bp.build_error(id, format!("Failed to build TLS server config: {err}"));
return;
}
};
server_config.ignore_client_order = listener.tls_ignore_client_order;
// Build acceptor
let default_config = Arc::new(server_config);
TcpAcceptor::Tls {
acceptor: TlsAcceptor::from(default_config.clone()),
config: default_config,
implicit: listener.tls_implicit,
}
} else {
TcpAcceptor::Plain
};
self.tcp_acceptors.insert(listener.name, acceptor);
}
}
}
fn is_ipv6_unsupported(err: &std::io::Error) -> bool {
let code = err.raw_os_error();
#[cfg(unix)]
{
matches!(code, Some(libc::EAFNOSUPPORT) | Some(libc::EPROTONOSUPPORT))
}
#[cfg(windows)]
{
matches!(code, Some(10047) | Some(10043))
}
#[cfg(not(any(unix, windows)))]
{
false
}
}
crates/common/src/config/server/mod.rs
+83
View File
@@ -0,0 +1,83 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::network::TcpAcceptor;
use ahash::AHashMap;
use registry::{
schema::structs::NetworkListener,
types::{id::ObjectId, ipmask::IpAddrOrMask},
};
use serde::{Deserialize, Serialize};
use std::{fmt::Display, net::SocketAddr, sync::Arc, time::Duration};
use store::registry::RegistryObject;
use tokio::net::TcpSocket;
use utils::snowflake::SnowflakeIdGenerator;
pub mod listener;
pub mod tls;
#[derive(Default)]
pub struct Listeners {
pub servers: Vec<Listener>,
pub tcp_acceptors: AHashMap<String, TcpAcceptor>,
pub span_id_gen: Arc<SnowflakeIdGenerator>,
parsed_listeners: Vec<RegistryObject<NetworkListener>>,
}
#[derive(Debug, Default)]
pub struct Listener {
pub registry_id: ObjectId,
pub id: String,
pub protocol: ServerProtocol,
pub listeners: Vec<TcpListener>,
pub proxy_networks: Vec<IpAddrOrMask>,
pub max_connections: u64,
pub tls_timeout: Duration,
pub span_id_gen: Arc<SnowflakeIdGenerator>,
}
pub const DEFAULT_TLS_TIMEOUT: Duration = Duration::from_secs(60);
#[derive(Debug)]
pub struct TcpListener {
pub socket: TcpSocket,
pub addr: SocketAddr,
pub backlog: Option<u32>,
// TCP options
pub ttl: Option<u32>,
pub nodelay: bool,
}
#[derive(Debug, PartialEq, Eq, Hash, Clone, Copy, Default, Serialize, Deserialize)]
pub enum ServerProtocol {
#[default]
Smtp,
Lmtp,
Imap,
Pop3,
Http,
ManageSieve,
}
impl ServerProtocol {
pub fn as_str(&self) -> &'static str {
match self {
ServerProtocol::Smtp => "smtp",
ServerProtocol::Lmtp => "lmtp",
ServerProtocol::Imap => "imap",
ServerProtocol::Http => "http",
ServerProtocol::Pop3 => "pop3",
ServerProtocol::ManageSieve => "managesieve",
}
}
}
impl Display for ServerProtocol {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(self.as_str())
}
}
crates/common/src/config/server/tls.rs
+278
View File
@@ -0,0 +1,278 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::network::acme::ParsedCert;
use ahash::{AHashMap, AHashSet};
use rcgen::generate_simple_self_signed;
use registry::{
schema::{
prelude::Object,
structs::{Certificate, PublicText, SecretText, SystemSettings},
},
types::{datetime::UTCDateTime, map::Map},
};
use rustls::{
SupportedProtocolVersion,
crypto::aws_lc_rs::sign::any_supported_type,
sign::CertifiedKey,
version::{TLS12, TLS13},
};
use rustls_pemfile::{Item, certs, read_all};
use rustls_pki_types::PrivateKeyDer;
use std::{io::Cursor, sync::Arc};
use store::{
registry::{bootstrap::Bootstrap, write::RegistryWrite},
write::now,
};
pub static TLS13_VERSION: &[&SupportedProtocolVersion] = &[&TLS13];
pub static TLS12_VERSION: &[&SupportedProtocolVersion] = &[&TLS12];
pub(crate) async fn parse_certificates(
bp: &mut Bootstrap,
certificates: &mut AHashMap<Box<str>, Arc<CertifiedKey>>,
subject_names: &mut AHashSet<Box<str>>,
) {
let system = bp.setting_infallible::<SystemSettings>().await;
// Parse certificates
let now = now() as i64;
let mut certs_expired = Vec::new();
let mut certs_expirations = AHashMap::new();
for cert_obj in bp.list_infallible::<Certificate>().await {
let obj_id = cert_obj.id;
let revision = cert_obj.revision;
let mut cert = cert_obj.object;
let is_file_backed = matches!(cert.certificate, PublicText::File(_))
|| matches!(cert.private_key, SecretText::File(_));
let mut public = None;
let mut refreshed_meta = None;
if is_file_backed {
let pem = match cert.certificate.value().await {
Ok(value) => value.into_owned().into_bytes(),
Err(err) => {
bp.build_error(obj_id, format!("Failed to obtain certificate value: {err}"));
continue;
}
};
match ParsedCert::parse(&pem) {
Ok(parsed) => {
let not_valid_after =
UTCDateTime::from_timestamp(parsed.valid_not_after.timestamp());
let not_valid_before =
UTCDateTime::from_timestamp(parsed.valid_not_before.timestamp());
let sans = Map::new(parsed.sans);
if cert.not_valid_after != not_valid_after
|| cert.not_valid_before != not_valid_before
|| cert.issuer != parsed.issuer
|| cert.subject_alternative_names != sans
{
refreshed_meta =
Some((not_valid_after, not_valid_before, parsed.issuer, sans));
}
public = Some(pem);
}
Err(err) => {
bp.build_error(obj_id, format!("Invalid certificate: {err}"));
continue;
}
}
}
let (not_valid_after, not_valid_before) = match refreshed_meta.as_ref() {
Some((after, before, _, _)) => (after.timestamp(), before.timestamp()),
None => (
cert.not_valid_after.timestamp(),
cert.not_valid_before.timestamp(),
),
};
if not_valid_after <= now {
certs_expired.push((
obj_id,
cert.subject_alternative_names.clone().into_inner(),
Object {
inner: cert.into(),
revision,
},
));
continue;
} else if not_valid_before > now {
continue; // Skip certificates that are not yet valid
}
let secret = match cert.private_key.secret().await {
Ok(secret) => secret.into_owned().into_bytes(),
Err(err) => {
bp.build_error(
obj_id,
format!("Failed to obtain private key secret: {err}"),
);
continue;
}
};
let public = match public {
Some(public) => public,
None => match cert.certificate.value().await {
Ok(value) => value.into_owned().into_bytes(),
Err(err) => {
bp.build_error(obj_id, format!("Failed to obtain certificate value: {err}"));
continue;
}
},
};
if let Some((not_valid_after, not_valid_before, issuer, sans)) = refreshed_meta {
let old = Object {
inner: cert.clone().into(),
revision,
};
cert.not_valid_after = not_valid_after;
cert.not_valid_before = not_valid_before;
cert.issuer = issuer;
cert.subject_alternative_names = sans;
let new = Object {
inner: cert.clone().into(),
revision,
};
if let Err(err) = bp
.registry
.write(RegistryWrite::update(obj_id.id(), &new, &old))
.await
{
trc::error!(
err.details("Failed to refresh TLS certificate metadata in registry.")
.caused_by(trc::location!())
);
}
}
// Add default certificate
if system
.default_certificate_id
.as_ref()
.is_some_and(|id| *id == obj_id.id())
{
cert.subject_alternative_names
.push_unchecked("*".to_string());
}
// Ensure that the most up-to-date certificate is used
cert.subject_alternative_names.inner_mut().retain(|name| {
if certs_expirations
.get(name)
.is_none_or(|expires| *expires < not_valid_after)
{
certs_expirations.insert(name.clone(), not_valid_after);
true
} else {
false
}
});
match build_certified_key(public, secret) {
Ok(key) => {
// Add certificates
let key = Arc::new(key);
for name in cert.subject_alternative_names.into_inner() {
subject_names.insert(name.as_str().into());
certificates.insert(
name.strip_prefix("*.")
.map(Into::into)
.unwrap_or_else(|| name.into_boxed_str()),
key.clone(),
);
}
}
Err(err) => {
bp.build_error(obj_id, format!("Invalid certificate: {err}"));
}
}
}
// Remove expired certificates
if !certs_expired.is_empty() {
for (id, sans, object) in certs_expired {
if let Err(err) = bp
.registry
.write(RegistryWrite::delete_object(id, &object))
.await
{
trc::error!(
err.details("Failed to delete expired TLS certificate from registry.")
.caused_by(trc::location!())
);
} else {
trc::event!(
Tls(trc::TlsEvent::ExpiredCertificateRemoved),
Details = sans
);
}
}
}
}
pub(crate) fn build_certified_key(
cert: Vec<u8>,
pk_bytes: Vec<u8>,
) -> Result<CertifiedKey, String> {
let mut pk = None;
for item in read_all(&mut Cursor::new(pk_bytes)) {
match item.map_err(|err| format!("Failed to read private key PEM: {err}"))? {
Item::Pkcs8Key(key) => {
pk = Some(PrivateKeyDer::Pkcs8(key));
break;
}
Item::Pkcs1Key(key) => {
pk = Some(PrivateKeyDer::Pkcs1(key));
break;
}
Item::Sec1Key(key) => {
pk = Some(PrivateKeyDer::Sec1(key));
break;
}
_ => continue, // Skip certificates, DH params, etc.
}
}
let pk = pk.ok_or_else(|| "No private keys found.".to_string())?;
let cert = certs(&mut Cursor::new(cert))
.collect::<Result<Vec<_>, _>>()
.map_err(|err| format!("Failed to read certificates: {err}"))?;
if !cert.is_empty() {
Ok(CertifiedKey {
cert,
key: any_supported_type(&pk)
.map_err(|err| format!("Failed to sign certificate: {err}",))?,
ocsp: None,
})
} else {
Err("No certificates found.".to_string())
}
}
pub(crate) fn build_self_signed_cert(
domains: impl Into<Vec<String>>,
) -> Result<CertifiedKey, String> {
let domains = domains
.into()
.into_iter()
.map(|domain| {
if domain.is_ascii() {
domain
} else {
idna::domain_to_ascii(&domain).unwrap_or(domain)
}
})
.collect::<Vec<_>>();
let rcgen::CertifiedKey { cert, signing_key } = generate_simple_self_signed(domains)
.map_err(|err| format!("Failed to generate self-signed certificate: {err}",))?;
build_certified_key(
cert.pem().into_bytes(),
signing_key.serialize_pem().into_bytes(),
)
}
crates/common/src/config/smtp/auth.rs
+380
View File
@@ -0,0 +1,380 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::expr::{
self,
if_block::{BootstrapExprExt, IfBlock},
};
use mail_auth::{
common::crypto::{Ed25519Key, HashAlgorithm, RsaKey, Sha256, SigningKey},
dkim::{Canonicalization, Done},
dkim2::{Dkim2Signer, Done as Dkim2Done, Flag},
};
use mail_parser::decoders::base64::base64_decode;
use registry::{
schema::{
enums::{self, Dkim2Flag, ExpressionConstant},
prelude::ObjectType,
structs::{Dkim1Signature, DkimSignature, SenderAuth},
},
types::{ObjectImpl, map::Map},
};
use rustls_pki_types::{PrivateKeyDer, PrivatePkcs1KeyDer, PrivatePkcs8KeyDer, pem::PemObject};
use store::registry::bootstrap::Bootstrap;
use utils::cache::CacheItemWeight;
#[derive(Clone)]
pub struct MailAuthConfig {
pub dkim: DkimAuthConfig,
pub arc: ArcAuthConfig,
pub spf: SpfAuthConfig,
pub dmarc: DmarcAuthConfig,
pub iprev: IpRevAuthConfig,
}
#[derive(Clone)]
pub struct DkimAuthConfig {
pub verify: IfBlock,
pub sign: IfBlock,
pub strict: bool,
}
#[derive(Clone)]
pub struct ArcAuthConfig {
pub verify: IfBlock,
//pub seal: IfBlock,
}
#[derive(Clone)]
pub struct SpfAuthConfig {
pub verify_ehlo: IfBlock,
pub verify_mail_from: IfBlock,
}
#[derive(Clone)]
pub struct DmarcAuthConfig {
pub verify: IfBlock,
}
#[derive(Clone)]
pub struct IpRevAuthConfig {
pub verify: IfBlock,
}
#[derive(Debug, Clone, Copy, Default)]
pub enum VerifyStrategy {
#[default]
Relaxed,
Strict,
Disable,
}
pub enum Dkim1Signer {
RsaSha256(mail_auth::dkim::DkimSigner<RsaKey<Sha256>, Done>),
Ed25519Sha256(mail_auth::dkim::DkimSigner<Ed25519Key, Done>),
}
#[derive(Default)]
pub struct DkimSigners {
pub dkim1: Vec<Dkim1Signer>,
pub dkim2: Option<Dkim2Signer<Dkim2Done>>,
}
impl MailAuthConfig {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let auth = bp.setting_infallible::<SenderAuth>().await;
MailAuthConfig {
dkim: DkimAuthConfig {
verify: bp
.compile_expr(ObjectType::SenderAuth.singleton(), &auth.ctx_dkim_verify()),
sign: bp.compile_expr(
ObjectType::SenderAuth.singleton(),
&auth.ctx_dkim_sign_domain(),
),
strict: auth.dkim_strict,
},
arc: ArcAuthConfig {
verify: bp.compile_expr(ObjectType::SenderAuth.singleton(), &auth.ctx_arc_verify()),
//seal: bp.compile_expr(ObjectType::SenderAuth.singleton(), &auth.ctx_arc_seal_domain()),
},
spf: SpfAuthConfig {
verify_ehlo: bp.compile_expr(
ObjectType::SenderAuth.singleton(),
&auth.ctx_spf_ehlo_verify(),
),
verify_mail_from: bp.compile_expr(
ObjectType::SenderAuth.singleton(),
&auth.ctx_spf_from_verify(),
),
},
dmarc: DmarcAuthConfig {
verify: bp
.compile_expr(ObjectType::SenderAuth.singleton(), &auth.ctx_dmarc_verify()),
},
iprev: IpRevAuthConfig {
verify: bp.compile_expr(
ObjectType::SenderAuth.singleton(),
&auth.ctx_reverse_ip_verify(),
),
},
}
}
}
impl DkimSigners {
pub async fn insert(&mut self, domain: String, signature: DkimSignature) -> trc::Result<()> {
let mut errors = vec![];
if !signature.validate(&mut errors) {
return Err(trc::DkimEvent::BuildError
.reason("DKIM signature validation failed")
.details(
errors
.into_iter()
.map(|v| trc::Value::from(v.to_string()))
.collect::<Vec<_>>(),
));
}
match signature {
DkimSignature::Dkim1Ed25519Sha256(signature) => {
let private_key = signature
.private_key
.secret()
.await
.map_err(|err| trc::DkimEvent::BuildError.reason(err))?;
let private_key = simple_pem_parse(&private_key).ok_or_else(|| {
trc::DkimEvent::BuildError
.reason("Failed to parse ED25519 private key PEM")
.details("Invalid PEM format")
})?;
let key =
Ed25519Key::from_pkcs8_maybe_unchecked_der(&private_key).map_err(|err| {
trc::DkimEvent::BuildError
.reason(err)
.details("Failed to build ED25519 key")
})?;
self.dkim1
.push(Dkim1Signer::Ed25519Sha256(build_dkim1_signer(
domain, signature, key,
)));
}
DkimSignature::Dkim1RsaSha256(signature) => {
let private_key = signature
.private_key
.secret()
.await
.map_err(|err| trc::DkimEvent::BuildError.reason(err))?;
let key = rsa_key_parse(private_key.as_bytes())?;
self.dkim1.push(Dkim1Signer::RsaSha256(build_dkim1_signer(
domain, signature, key,
)));
}
DkimSignature::Dkim2Ed25519Sha256(signature) => {
let private_key = signature
.private_key
.secret()
.await
.map_err(|err| trc::DkimEvent::BuildError.reason(err))?;
let private_key = simple_pem_parse(&private_key).ok_or_else(|| {
trc::DkimEvent::BuildError
.reason("Failed to parse ED25519 private key PEM")
.details("Invalid PEM format")
})?;
let key =
Ed25519Key::from_pkcs8_maybe_unchecked_der(&private_key).map_err(|err| {
trc::DkimEvent::BuildError
.reason(err)
.details("Failed to build ED25519 key")
})?;
self.dkim2 = Some(match self.dkim2.take() {
None => Dkim2Signer::from_key(key)
.domain(domain)
.selector(signature.selector)
.flags(map_dkim2_flags(signature.flags)),
Some(signer) => signer
.additional_key(key, signature.selector)
.flags(map_dkim2_flags(signature.flags)),
});
}
DkimSignature::Dkim2RsaSha256(signature) => {
let private_key = signature
.private_key
.secret()
.await
.map_err(|err| trc::DkimEvent::BuildError.reason(err))?;
let key = rsa_key_parse(private_key.as_bytes())?;
self.dkim2 = Some(match self.dkim2.take() {
None => Dkim2Signer::from_key(key)
.domain(domain)
.selector(signature.selector)
.flags(map_dkim2_flags(signature.flags)),
Some(signer) => signer
.additional_key(key, signature.selector)
.flags(map_dkim2_flags(signature.flags)),
});
}
}
Ok(())
}
}
fn map_dkim2_flags(flags: Map<enums::Dkim2Flag>) -> impl Iterator<Item = Flag> {
flags.into_inner().into_iter().map(|flag| match flag {
Dkim2Flag::Donotmodify => Flag::DoNotModify,
Dkim2Flag::Donotexplode => Flag::DoNotExplode,
Dkim2Flag::Feedback => Flag::Feedback,
})
}
pub fn rsa_key_parse(private_key: &[u8]) -> trc::Result<RsaKey<Sha256>> {
PrivatePkcs1KeyDer::from_pem_slice(private_key)
.map(PrivateKeyDer::Pkcs1)
.or_else(|_| PrivatePkcs8KeyDer::from_pem_slice(private_key).map(PrivateKeyDer::Pkcs8))
.map_err(|err| {
trc::DkimEvent::BuildError
.reason(err)
.details("Failed to build RSA key")
})
.and_then(|key| {
RsaKey::<Sha256>::from_key_der(key).map_err(|err| {
trc::DkimEvent::BuildError
.reason(err)
.details("Failed to build RSA key")
})
})
}
pub fn simple_pem_parse(contents: &str) -> Option<Vec<u8>> {
let mut contents = contents.as_bytes().iter().copied();
let mut base64 = vec![];
'outer: while let Some(ch) = contents.next() {
if !ch.is_ascii_whitespace() {
if ch == b'-' {
for ch in contents.by_ref() {
if ch == b'\n' {
break;
}
}
} else {
base64.push(ch);
}
for ch in contents.by_ref() {
if ch == b'-' {
break 'outer;
} else if !ch.is_ascii_whitespace() {
base64.push(ch);
}
}
}
}
base64_decode(&base64)
}
fn build_dkim1_signer<T: SigningKey>(
domain: String,
signature: Dkim1Signature,
key: T,
) -> mail_auth::dkim::DkimSigner<T, Done> {
let mut signer = mail_auth::dkim::DkimSigner::from_key(key)
.domain(domain)
.selector(signature.selector)
.headers(signature.headers)
.reporting(signature.report);
match signature.canonicalization {
enums::DkimCanonicalization::RelaxedRelaxed => {
signer = signer
.body_canonicalization(Canonicalization::Relaxed)
.header_canonicalization(Canonicalization::Relaxed);
}
enums::DkimCanonicalization::SimpleSimple => {
signer = signer
.body_canonicalization(Canonicalization::Simple)
.header_canonicalization(Canonicalization::Simple);
}
enums::DkimCanonicalization::RelaxedSimple => {
signer = signer
.body_canonicalization(Canonicalization::Simple)
.header_canonicalization(Canonicalization::Relaxed);
}
enums::DkimCanonicalization::SimpleRelaxed => {
signer = signer
.body_canonicalization(Canonicalization::Relaxed)
.header_canonicalization(Canonicalization::Simple);
}
}
if let Some(expire) = signature.expire {
signer = signer.expiration(expire.into_inner().as_secs());
}
if let Some(auid) = signature.auid {
signer = signer.agent_user_identifier(auid);
}
if let Some(atps) = signature.third_party {
signer = signer.atps(atps);
}
if let Some(atpsh) = signature.third_party_hash {
signer = signer.atpsh(match atpsh {
enums::DkimHash::Sha256 => HashAlgorithm::Sha256,
enums::DkimHash::Sha1 => HashAlgorithm::Sha1,
});
}
signer
}
impl<'x> TryFrom<expr::Variable<'x>> for VerifyStrategy {
type Error = ();
fn try_from(value: expr::Variable<'x>) -> Result<Self, Self::Error> {
match value {
expr::Variable::Constant(c) => match c {
ExpressionConstant::Relaxed => Ok(VerifyStrategy::Relaxed),
ExpressionConstant::Strict => Ok(VerifyStrategy::Strict),
ExpressionConstant::Disable => Ok(VerifyStrategy::Disable),
_ => Err(()),
},
_ => Err(()),
}
}
}
impl VerifyStrategy {
#[inline(always)]
pub fn verify(&self) -> bool {
matches!(self, VerifyStrategy::Strict | VerifyStrategy::Relaxed)
}
#[inline(always)]
pub fn is_strict(&self) -> bool {
matches!(self, VerifyStrategy::Strict)
}
}
impl CacheItemWeight for Dkim1Signer {
fn weight(&self) -> u64 {
std::mem::size_of::<Self>() as u64
}
}
impl CacheItemWeight for DkimSigners {
fn weight(&self) -> u64 {
(std::mem::size_of::<Self>()
+ self.dkim1.len() * std::mem::size_of::<Dkim1Signer>()
+ std::mem::size_of::<Dkim2Signer<Dkim2Done>>()) as u64
}
}
crates/common/src/config/smtp/mod.rs
+92
View File
@@ -0,0 +1,92 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
pub mod auth;
pub mod queue;
pub mod report;
pub mod resolver;
pub mod session;
use self::{
auth::MailAuthConfig, queue::QueueConfig, report::ReportConfig, resolver::Resolvers,
session::SessionConfig,
};
use crate::{config::smtp::queue::RequireOptional, expr::if_block::IfBlock};
use registry::{
schema::{properties::ObjectType, structs::Rate},
types::id::ObjectId,
};
use store::registry::bootstrap::Bootstrap;
#[derive(Clone)]
pub struct SmtpConfig {
pub session: SessionConfig,
pub queue: QueueConfig,
pub resolvers: Resolvers,
pub mail_auth: MailAuthConfig,
pub report: ReportConfig,
pub mta_sts_client: reqwest::Client,
pub tls_report_client: reqwest::Client,
}
#[derive(Debug, Clone)]
//#[cfg_attr(feature = "test_mode", derive(PartialEq, Eq))]
pub struct QueueRateLimiter {
pub id: ObjectId,
pub expr: IfBlock,
pub keys: u16,
pub rate: Rate,
}
pub const THROTTLE_RCPT: u16 = 1 << 0;
pub const THROTTLE_RCPT_DOMAIN: u16 = 1 << 1;
pub const THROTTLE_SENDER: u16 = 1 << 2;
pub const THROTTLE_SENDER_DOMAIN: u16 = 1 << 3;
pub const THROTTLE_AUTH_AS: u16 = 1 << 4;
pub const THROTTLE_LISTENER: u16 = 1 << 5;
pub const THROTTLE_MX: u16 = 1 << 6;
pub const THROTTLE_REMOTE_IP: u16 = 1 << 7;
pub const THROTTLE_LOCAL_IP: u16 = 1 << 8;
pub const THROTTLE_HELO_DOMAIN: u16 = 1 << 9;
impl SmtpConfig {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let config = Self {
session: SessionConfig::parse(bp).await,
queue: QueueConfig::parse(bp).await,
resolvers: Resolvers::parse(bp).await,
mail_auth: MailAuthConfig::parse(bp).await,
report: ReportConfig::parse(bp).await,
mta_sts_client: utils::http::http_client_builder(false)
.pool_max_idle_per_host(0)
.user_agent(crate::USER_AGENT)
.redirect(reqwest::redirect::Policy::none())
.build()
.unwrap_or_default(),
tls_report_client: utils::http::unpooled_http_client(false),
};
if !config.resolvers.dnssec_available
&& (config.queue.tls_strategy.is_empty()
|| config
.queue
.tls_strategy
.values()
.any(|t| !matches!(t.dane, RequireOptional::Disable)))
{
bp.build_warning(
ObjectType::DnsResolver.singleton(),
concat!(
"The configured DNS resolver cannot validate DNSSEC. ",
"DANE has been disabled to avoid deferring mail. ",
"Ensure the resolver is DNSSEC-capable and reachable over TCP."
),
);
}
config
}
}
crates/common/src/config/smtp/queue.rs
+801
View File
@@ -0,0 +1,801 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use super::*;
use crate::{
config::server::ServerProtocol,
expr::{
if_block::{BootstrapExprExt, IfBlock},
*,
},
};
use ahash::AHashMap;
use directory::Credentials;
use mail_auth::IpLookupStrategy;
use registry::schema::{
enums::{self, ExpressionConstant, ExpressionVariable, MtaRequiredOrOptional},
prelude::ObjectType,
structs::{
DsnReportSettings, MtaConnectionStrategy, MtaDeliveryExpiration, MtaDeliverySchedule,
MtaDeliveryScheduleIntervalsOrDefault, MtaInboundThrottle, MtaOutboundStrategy,
MtaOutboundThrottle, MtaQueueQuota, MtaRoute, MtaTlsStrategy, MtaVirtualQueue,
},
};
use std::{
fmt::Display,
hash::{Hash, Hasher},
net::IpAddr,
time::Duration,
};
#[derive(
Debug,
Clone,
Copy,
PartialEq,
Eq,
Hash,
rkyv::Serialize,
rkyv::Deserialize,
rkyv::Archive,
serde::Deserialize,
)]
#[rkyv(derive(Debug, Clone, Copy, PartialEq), compare(PartialEq))]
#[repr(transparent)]
pub struct QueueName([u8; 8]);
pub const DEFAULT_QUEUE_NAME: QueueName = QueueName([b'd', b'e', b'f', b'a', b'u', b'l', b't', 0]);
#[derive(Clone)]
pub struct QueueConfig {
// Strategy resolver
pub route: IfBlock,
pub queue: IfBlock,
pub connection: IfBlock,
pub tls: IfBlock,
// DSN
pub dsn: Dsn,
// Rate limits
pub inbound_limiters: QueueRateLimiters,
pub outbound_limiters: QueueRateLimiters,
pub quota: QueueQuotas,
// Strategies
pub queue_strategy: AHashMap<String, QueueStrategy>,
pub connection_strategy: AHashMap<String, ConnectionStrategy>,
pub routing_strategy: AHashMap<String, RoutingStrategy>,
pub tls_strategy: AHashMap<String, TlsStrategy>,
pub virtual_queues: AHashMap<QueueName, VirtualQueue>,
}
#[derive(Clone, Hash, PartialEq, Eq, Debug)]
pub enum RoutingStrategy {
Local,
Mx(MxConfig),
Relay(RelayConfig),
}
#[derive(Clone, Debug)]
pub struct MxConfig {
pub max_mx: usize,
pub max_multi_homed: usize,
pub ip_lookup_strategy: IpLookupStrategy,
}
#[derive(Clone)]
pub struct Dsn {
pub name: IfBlock,
pub address: IfBlock,
pub sign: IfBlock,
}
#[derive(Clone, Debug)]
pub struct VirtualQueue {
pub threads: usize,
}
#[derive(Clone, Debug)]
pub struct QueueStrategy {
pub retry: Vec<u64>,
pub notify: Vec<u64>,
pub expiry: QueueExpiry,
pub virtual_queue: QueueName,
}
#[derive(
rkyv::Serialize,
rkyv::Deserialize,
rkyv::Archive,
Debug,
Clone,
Copy,
PartialEq,
Eq,
serde::Deserialize,
)]
pub enum QueueExpiry {
Ttl(u64),
Attempts(u32),
}
#[derive(Clone, Debug)]
pub struct TlsStrategy {
pub dane: RequireOptional,
pub mta_sts: RequireOptional,
pub tls: RequireOptional,
pub allow_invalid_certs: bool,
pub timeout_tls: Duration,
pub timeout_mta_sts: Duration,
}
#[derive(Clone, Debug)]
pub struct ConnectionStrategy {
pub source_ipv4: Vec<IpAndHost>,
pub source_ipv6: Vec<IpAndHost>,
pub ehlo_hostname: Option<String>,
pub timeout_connect: Duration,
pub timeout_greeting: Duration,
pub timeout_ehlo: Duration,
pub timeout_mail: Duration,
pub timeout_rcpt: Duration,
pub timeout_data: Duration,
}
#[derive(Clone, Debug)]
pub struct IpAndHost {
pub ip: IpAddr,
pub host: Option<String>,
}
#[derive(Debug, Clone, Default)]
pub struct QueueRateLimiters {
pub sender: Vec<QueueRateLimiter>,
pub rcpt: Vec<QueueRateLimiter>,
pub remote: Vec<QueueRateLimiter>,
}
#[derive(Clone, Default)]
pub struct QueueQuotas {
pub sender: Vec<QueueQuota>,
pub rcpt: Vec<QueueQuota>,
pub rcpt_domain: Vec<QueueQuota>,
}
#[derive(Clone)]
pub struct QueueQuota {
pub id: ObjectId,
pub expr: IfBlock,
pub keys: u16,
pub size: Option<u64>,
pub messages: Option<u64>,
}
#[derive(Clone, Hash, PartialEq, Eq)]
pub struct RelayConfig {
pub address: HostOrIp<Box<str>, IpStr>,
pub port: u16,
pub protocol: ServerProtocol,
pub auth: Option<Credentials>,
pub tls_implicit: bool,
pub tls_allow_invalid_certs: bool,
}
#[derive(Clone, Debug, Hash, PartialEq, Eq)]
pub enum HostOrIp<N, I> {
Host(N),
Ip(I),
}
#[derive(Clone, Debug, Hash, PartialEq, Eq)]
pub struct IpStr {
pub ip: IpAddr,
pub ip_str: Box<str>,
}
#[derive(Debug, Clone, Copy, Default)]
pub enum RequireOptional {
#[default]
Optional,
Require,
Disable,
}
impl QueueConfig {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let st = bp.setting_infallible::<MtaOutboundStrategy>().await;
let dsn = bp.setting_infallible::<DsnReportSettings>().await;
let mut queue = QueueConfig {
route: bp.compile_expr(ObjectType::MtaOutboundStrategy.singleton(), &st.ctx_route()),
queue: bp.compile_expr(
ObjectType::MtaOutboundStrategy.singleton(),
&st.ctx_schedule(),
),
connection: bp.compile_expr(
ObjectType::MtaOutboundStrategy.singleton(),
&st.ctx_connection(),
),
tls: bp.compile_expr(ObjectType::MtaOutboundStrategy.singleton(), &st.ctx_tls()),
dsn: Dsn {
name: bp.compile_expr(
ObjectType::DsnReportSettings.singleton(),
&dsn.ctx_from_name(),
),
address: bp.compile_expr(
ObjectType::DsnReportSettings.singleton(),
&dsn.ctx_from_address(),
),
sign: bp.compile_expr(
ObjectType::DsnReportSettings.singleton(),
&dsn.ctx_dkim_sign_domain(),
),
},
inbound_limiters: QueueRateLimiters::parse_inbound(bp).await,
outbound_limiters: QueueRateLimiters::parse_outbound(bp).await,
quota: QueueQuotas::parse(bp).await,
queue_strategy: Default::default(),
connection_strategy: Default::default(),
routing_strategy: Default::default(),
tls_strategy: Default::default(),
virtual_queues: Default::default(),
};
// Parse virtual queues
let mut queue_id_to_name = AHashMap::new();
for obj in bp.list_infallible::<MtaVirtualQueue>().await {
if let Some(queue_name) = QueueName::new(&obj.object.name) {
queue_id_to_name.insert(obj.id.id(), queue_name);
queue.virtual_queues.insert(
queue_name,
VirtualQueue {
threads: obj.object.threads_per_node as usize,
},
);
}
}
// Parse queue strategies
for obj in bp.list_infallible::<MtaDeliverySchedule>().await {
let virtual_queue = if let Some(name) = queue_id_to_name.get(&obj.object.queue_id) {
*name
} else {
bp.build_error(
obj.id,
format!("Virtual queue ID '{}' does not exist.", obj.object.queue_id),
);
continue;
};
queue.queue_strategy.insert(
obj.object.name,
QueueStrategy {
retry: match obj.object.retry {
MtaDeliveryScheduleIntervalsOrDefault::Default => vec![
2 * 60,
5 * 60,
10 * 60,
15 * 60,
30 * 60,
60 * 60,
2 * 60 * 60,
24 * 60 * 60,
3 * 24 * 60 * 60,
],
MtaDeliveryScheduleIntervalsOrDefault::Custom(intervals) => intervals
.intervals
.into_iter()
.map(|d| d.duration.as_secs())
.collect(),
},
notify: match obj.object.notify {
MtaDeliveryScheduleIntervalsOrDefault::Default => {
vec![24 * 60 * 60, 3 * 24 * 60 * 60]
}
MtaDeliveryScheduleIntervalsOrDefault::Custom(intervals) => intervals
.intervals
.into_iter()
.map(|d| d.duration.as_secs())
.collect(),
},
expiry: match obj.object.expiry {
MtaDeliveryExpiration::Ttl(exp) => {
QueueExpiry::Ttl(exp.expire.into_inner().as_secs())
}
MtaDeliveryExpiration::Attempts(exp) => {
QueueExpiry::Attempts(exp.max_attempts as u32)
}
},
virtual_queue,
},
);
}
// Parse connection strategies
for obj in bp.list_infallible::<MtaConnectionStrategy>().await {
let mut source_ipv4 = Vec::new();
let mut source_ipv6 = Vec::new();
for ip_host in obj.object.source_ips {
let ip_host = IpAndHost {
ip: ip_host.source_ip.into_inner(),
host: ip_host.ehlo_hostname,
};
if ip_host.ip.is_ipv4() {
source_ipv4.push(ip_host);
} else {
source_ipv6.push(ip_host);
}
}
queue.connection_strategy.insert(
obj.object.name,
ConnectionStrategy {
source_ipv4,
source_ipv6,
ehlo_hostname: obj.object.ehlo_hostname,
timeout_connect: obj.object.connect_timeout.into_inner(),
timeout_greeting: obj.object.greeting_timeout.into_inner(),
timeout_ehlo: obj.object.ehlo_timeout.into_inner(),
timeout_mail: obj.object.mail_from_timeout.into_inner(),
timeout_rcpt: obj.object.rcpt_to_timeout.into_inner(),
timeout_data: obj.object.data_timeout.into_inner(),
},
);
}
// Parse routing strategies
for obj in bp.list_infallible::<MtaRoute>().await {
match obj.object {
MtaRoute::Mx(route) => {
queue.routing_strategy.insert(
route.name,
RoutingStrategy::Mx(MxConfig {
max_mx: route.max_mx_hosts as usize,
max_multi_homed: route.max_multihomed as usize,
ip_lookup_strategy: match route.ip_lookup_strategy {
enums::MtaIpStrategy::V4ThenV6 => IpLookupStrategy::Ipv4thenIpv6,
enums::MtaIpStrategy::V6ThenV4 => IpLookupStrategy::Ipv6thenIpv4,
enums::MtaIpStrategy::V4Only => IpLookupStrategy::Ipv4Only,
enums::MtaIpStrategy::V6Only => IpLookupStrategy::Ipv6Only,
},
}),
);
}
MtaRoute::Relay(route) => {
let secret = route
.auth_secret
.secret()
.await
.map_err(|err| {
bp.build_error(obj.id, err);
})
.unwrap_or_default();
queue.routing_strategy.insert(
route.name,
RoutingStrategy::Relay(RelayConfig {
address: if let Ok(ip) = route.address.parse() {
HostOrIp::Ip(IpStr {
ip,
ip_str: route.address.into(),
})
} else {
HostOrIp::Host(route.address.into())
},
port: route.port as u16,
protocol: match route.protocol {
enums::MtaProtocol::Smtp => ServerProtocol::Smtp,
enums::MtaProtocol::Lmtp => ServerProtocol::Lmtp,
},
auth: route.auth_username.zip(secret).map(|(user, secret)| {
Credentials::Basic {
username: user,
secret: secret.into_owned(),
mfa_token: None,
}
}),
tls_implicit: route.implicit_tls,
tls_allow_invalid_certs: route.allow_invalid_certs,
}),
);
}
MtaRoute::Local(route) => {
queue
.routing_strategy
.insert(route.name, RoutingStrategy::Local);
}
}
}
// Parse TLS strategies
for obj in bp.list_infallible::<MtaTlsStrategy>().await {
queue.tls_strategy.insert(
obj.object.name,
TlsStrategy {
dane: match obj.object.dane {
MtaRequiredOrOptional::Optional => RequireOptional::Optional,
MtaRequiredOrOptional::Require => RequireOptional::Require,
MtaRequiredOrOptional::Disable => RequireOptional::Disable,
},
mta_sts: match obj.object.mta_sts {
MtaRequiredOrOptional::Optional => RequireOptional::Optional,
MtaRequiredOrOptional::Require => RequireOptional::Require,
MtaRequiredOrOptional::Disable => RequireOptional::Disable,
},
tls: match obj.object.start_tls {
MtaRequiredOrOptional::Optional => RequireOptional::Optional,
MtaRequiredOrOptional::Require => RequireOptional::Require,
MtaRequiredOrOptional::Disable => RequireOptional::Disable,
},
allow_invalid_certs: obj.object.allow_invalid_certs,
timeout_tls: obj.object.tls_timeout.into_inner(),
timeout_mta_sts: obj.object.mta_sts_timeout.into_inner(),
},
);
}
queue
}
}
impl QueueRateLimiters {
async fn parse_inbound(bp: &mut Bootstrap) -> QueueRateLimiters {
let mut throttle = QueueRateLimiters::default();
for obj in bp.list_infallible::<MtaInboundThrottle>().await {
if !obj.object.enable {
continue;
}
let limiter = QueueRateLimiter {
expr: bp.compile_expr(obj.id, &obj.object.ctx_match_()),
id: obj.id,
keys: obj
.object
.key
.iter()
.map(|key| match key {
enums::MtaInboundThrottleKey::Rcpt => THROTTLE_RCPT,
enums::MtaInboundThrottleKey::RcptDomain => THROTTLE_RCPT_DOMAIN,
enums::MtaInboundThrottleKey::Sender => THROTTLE_SENDER,
enums::MtaInboundThrottleKey::SenderDomain => THROTTLE_SENDER_DOMAIN,
enums::MtaInboundThrottleKey::AuthenticatedAs => THROTTLE_AUTH_AS,
enums::MtaInboundThrottleKey::Listener => THROTTLE_LISTENER,
enums::MtaInboundThrottleKey::RemoteIp => THROTTLE_REMOTE_IP,
enums::MtaInboundThrottleKey::LocalIp => THROTTLE_LOCAL_IP,
enums::MtaInboundThrottleKey::HeloDomain => THROTTLE_HELO_DOMAIN,
})
.fold(0, |acc, key| acc | key),
rate: obj.object.rate,
};
if (limiter.keys & (THROTTLE_RCPT | THROTTLE_RCPT_DOMAIN)) != 0
|| limiter.expr.all_items().any(|c| {
matches!(
c,
ExpressionItem::Variable(
ExpressionVariable::Rcpt | ExpressionVariable::RcptDomain
)
)
})
{
throttle.rcpt.push(limiter);
} else if (limiter.keys
& (THROTTLE_SENDER
| THROTTLE_SENDER_DOMAIN
| THROTTLE_HELO_DOMAIN
| THROTTLE_AUTH_AS))
!= 0
|| limiter.expr.all_items().any(|c| {
matches!(
c,
ExpressionItem::Variable(
ExpressionVariable::Sender
| ExpressionVariable::SenderDomain
| ExpressionVariable::HeloDomain
| ExpressionVariable::AuthenticatedAs
)
)
})
{
throttle.sender.push(limiter);
} else {
throttle.remote.push(limiter);
}
}
throttle
}
async fn parse_outbound(bp: &mut Bootstrap) -> QueueRateLimiters {
// Parse throttle
let mut throttle = QueueRateLimiters::default();
for obj in bp.list_infallible::<MtaOutboundThrottle>().await {
if !obj.object.enable {
continue;
}
let limiter = QueueRateLimiter {
expr: bp.compile_expr(obj.id, &obj.object.ctx_match_()),
id: obj.id,
keys: obj
.object
.key
.iter()
.map(|key| match key {
enums::MtaOutboundThrottleKey::RcptDomain => THROTTLE_RCPT_DOMAIN,
enums::MtaOutboundThrottleKey::Sender => THROTTLE_SENDER,
enums::MtaOutboundThrottleKey::SenderDomain => THROTTLE_SENDER_DOMAIN,
enums::MtaOutboundThrottleKey::Mx => THROTTLE_MX,
enums::MtaOutboundThrottleKey::RemoteIp => THROTTLE_REMOTE_IP,
enums::MtaOutboundThrottleKey::LocalIp => THROTTLE_LOCAL_IP,
})
.fold(0, |acc, key| acc | key),
rate: obj.object.rate,
};
if (limiter.keys & (THROTTLE_MX | THROTTLE_REMOTE_IP | THROTTLE_LOCAL_IP)) != 0
|| limiter.expr.all_items().any(|c| {
matches!(
c,
ExpressionItem::Variable(
ExpressionVariable::Mx
| ExpressionVariable::RemoteIp
| ExpressionVariable::LocalIp
)
)
})
{
throttle.remote.push(limiter);
} else if (limiter.keys & (THROTTLE_RCPT_DOMAIN)) != 0
|| limiter
.expr
.all_items()
.any(|c| matches!(c, ExpressionItem::Variable(ExpressionVariable::RcptDomain)))
{
throttle.rcpt.push(limiter);
} else {
throttle.sender.push(limiter);
}
}
throttle
}
}
impl QueueQuotas {
async fn parse(bp: &mut Bootstrap) -> QueueQuotas {
let mut capacities = QueueQuotas {
sender: Vec::new(),
rcpt: Vec::new(),
rcpt_domain: Vec::new(),
};
for obj in bp.list_infallible::<MtaQueueQuota>().await {
if !obj.object.enable {
continue;
}
let quota = QueueQuota {
expr: bp.compile_expr(obj.id, &obj.object.ctx_match_()),
id: obj.id,
keys: obj
.object
.key
.iter()
.map(|key| match key {
enums::MtaQueueQuotaKey::Rcpt => THROTTLE_RCPT,
enums::MtaQueueQuotaKey::RcptDomain => THROTTLE_RCPT_DOMAIN,
enums::MtaQueueQuotaKey::Sender => THROTTLE_SENDER,
enums::MtaQueueQuotaKey::SenderDomain => THROTTLE_SENDER_DOMAIN,
})
.fold(0, |acc, key| acc | key),
size: obj.object.size,
messages: obj.object.messages,
};
if (quota.keys & THROTTLE_RCPT) != 0
|| quota
.expr
.all_items()
.any(|c| matches!(c, ExpressionItem::Variable(ExpressionVariable::Rcpt)))
{
capacities.rcpt.push(quota);
} else if (quota.keys & THROTTLE_RCPT_DOMAIN) != 0
|| quota
.expr
.all_items()
.any(|c| matches!(c, ExpressionItem::Variable(ExpressionVariable::RcptDomain)))
{
capacities.rcpt_domain.push(quota);
} else {
capacities.sender.push(quota);
}
}
capacities
}
}
impl<'x> TryFrom<Variable<'x>> for RequireOptional {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
match value {
Variable::Constant(ExpressionConstant::Optional) => Ok(RequireOptional::Optional),
Variable::Constant(ExpressionConstant::Require) => Ok(RequireOptional::Require),
Variable::Constant(ExpressionConstant::Disable) => Ok(RequireOptional::Disable),
_ => Err(()),
}
}
}
impl<'x> TryFrom<Variable<'x>> for IpLookupStrategy {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
match value {
Variable::Constant(value) => match value {
ExpressionConstant::Ipv4Only => Ok(IpLookupStrategy::Ipv4Only),
ExpressionConstant::Ipv6Only => Ok(IpLookupStrategy::Ipv6Only),
ExpressionConstant::Ipv6ThenIpv4 => Ok(IpLookupStrategy::Ipv6thenIpv4),
ExpressionConstant::Ipv4ThenIpv6 => Ok(IpLookupStrategy::Ipv4thenIpv6),
_ => Err(()),
},
Variable::String(value) => {
match value.as_str() {
"ipv4_only" => Ok(IpLookupStrategy::Ipv4Only),
"ipv6_only" => Ok(IpLookupStrategy::Ipv6Only),
//"ipv4_and_ipv6" => IpLookupStrategy::Ipv4AndIpv6,
"ipv6_then_ipv4" => Ok(IpLookupStrategy::Ipv6thenIpv4),
"ipv4_then_ipv6" => Ok(IpLookupStrategy::Ipv4thenIpv6),
_ => Err(()),
}
}
_ => Err(()),
}
}
}
impl std::fmt::Debug for RelayConfig {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("RelayConfig")
.field("address", &self.address)
.field("port", &self.port)
.field("protocol", &self.protocol)
.field("tls_implicit", &self.tls_implicit)
.field("tls_allow_invalid_certs", &self.tls_allow_invalid_certs)
.finish()
}
}
impl TlsStrategy {
#[inline(always)]
pub fn try_dane(&self) -> bool {
matches!(
self.dane,
RequireOptional::Require | RequireOptional::Optional
)
}
#[inline(always)]
pub fn try_start_tls(&self) -> bool {
matches!(
self.tls,
RequireOptional::Require | RequireOptional::Optional
)
}
#[inline(always)]
pub fn is_dane_required(&self) -> bool {
matches!(self.dane, RequireOptional::Require)
}
#[inline(always)]
pub fn try_mta_sts(&self) -> bool {
matches!(
self.mta_sts,
RequireOptional::Require | RequireOptional::Optional
)
}
#[inline(always)]
pub fn is_mta_sts_required(&self) -> bool {
matches!(self.mta_sts, RequireOptional::Require)
}
#[inline(always)]
pub fn is_tls_required(&self) -> bool {
matches!(self.tls, RequireOptional::Require)
|| self.is_dane_required()
|| self.is_mta_sts_required()
}
}
impl Hash for MxConfig {
fn hash<H: Hasher>(&self, state: &mut H) {
self.max_mx.hash(state);
self.max_multi_homed.hash(state);
}
}
impl PartialEq for MxConfig {
fn eq(&self, other: &Self) -> bool {
self.max_mx == other.max_mx && self.max_multi_homed == other.max_multi_homed
}
}
impl Eq for MxConfig {}
impl QueueName {
pub fn new(name: impl AsRef<[u8]>) -> Option<Self> {
let name_bytes = name.as_ref();
if (1..=8).contains(&name_bytes.len()) {
let mut bytes = [0; 8];
bytes[..name_bytes.len()].copy_from_slice(name_bytes);
QueueName(bytes).into()
} else {
None
}
}
pub fn from_bytes(name: &[u8]) -> Option<Self> {
name.try_into().ok().map(|bytes: [u8; 8]| QueueName(bytes))
}
pub fn as_str(&self) -> &str {
std::str::from_utf8(&self.0)
.unwrap_or_default()
.trim_end_matches('\0')
}
pub fn into_inner(self) -> [u8; 8] {
self.0
}
pub fn as_slice(&self) -> &[u8] {
&self.0
}
}
impl ArchivedQueueName {
pub fn as_str(&self) -> &str {
std::str::from_utf8(self.0.as_ref())
.unwrap_or_default()
.trim_end_matches('\0')
}
pub fn as_slice(&self) -> &[u8] {
self.0.as_ref()
}
}
impl Default for QueueName {
fn default() -> Self {
DEFAULT_QUEUE_NAME
}
}
impl Display for QueueName {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
self.as_str().fmt(f)
}
}
impl Display for ArchivedQueueName {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
self.as_str().fmt(f)
}
}
impl AsRef<[u8]> for QueueName {
fn as_ref(&self) -> &[u8] {
&self.0
}
}
crates/common/src/config/smtp/report.rs
+273
View File
@@ -0,0 +1,273 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use super::*;
use crate::expr::{
Variable,
if_block::{BootstrapExprExt, IfBlock},
};
use registry::schema::{
enums::ExpressionConstant,
prelude::ObjectType,
structs::{
DataRetention, DkimReportSettings, DmarcReportSettings, ReportSettings, SpfReportSettings,
TlsReportSettings,
},
};
use std::{str::FromStr, time::Duration};
#[derive(Clone)]
pub struct ReportConfig {
pub submitter: IfBlock,
pub analysis: ReportAnalysis,
pub dkim: Report,
pub spf: Report,
pub dmarc: Report,
pub dmarc_aggregate: AggregateReport,
pub tls: AggregateReport,
}
#[derive(Clone)]
pub struct ReportAnalysis {
pub addresses: Vec<AddressMatch>,
pub forward: bool,
pub store: Option<Duration>,
pub max_size: usize,
}
#[derive(Clone)]
pub enum AddressMatch {
StartsWith(String),
EndsWith(String),
Equals(String),
}
#[derive(Clone)]
pub struct AggregateReport {
pub name: IfBlock,
pub address: IfBlock,
pub org_name: IfBlock,
pub contact_info: IfBlock,
pub send: IfBlock,
pub sign: IfBlock,
pub max_size: IfBlock,
}
#[derive(Clone)]
pub struct Report {
pub name: IfBlock,
pub address: IfBlock,
pub subject: IfBlock,
pub sign: IfBlock,
pub send: IfBlock,
}
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub enum AggregateFrequency {
Hourly,
Daily,
Weekly,
#[default]
Never,
}
impl ReportConfig {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let report = bp.setting_infallible::<ReportSettings>().await;
let dkim = bp.setting_infallible::<DkimReportSettings>().await;
let spf = bp.setting_infallible::<SpfReportSettings>().await;
let dmarc = bp.setting_infallible::<DmarcReportSettings>().await;
let tls = bp.setting_infallible::<TlsReportSettings>().await;
let dr = bp.setting_infallible::<DataRetention>().await;
ReportConfig {
submitter: bp.compile_expr(
ObjectType::ReportSettings.singleton(),
&report.ctx_outbound_report_submitter(),
),
analysis: ReportAnalysis {
addresses: report
.inbound_report_addresses
.iter()
.filter_map(|addr| AddressMatch::from_str(addr).ok())
.collect(),
forward: report.inbound_report_forwarding,
store: dr.hold_mta_reports_for.map(|d| d.into_inner()),
max_size: std::cmp::max(report.inbound_report_max_size, 1024) as usize,
},
dkim: Report {
name: bp.compile_expr(
ObjectType::DkimReportSettings.singleton(),
&dkim.ctx_from_name(),
),
address: bp.compile_expr(
ObjectType::DkimReportSettings.singleton(),
&dkim.ctx_from_address(),
),
subject: bp.compile_expr(
ObjectType::DkimReportSettings.singleton(),
&dkim.ctx_subject(),
),
sign: bp.compile_expr(
ObjectType::DkimReportSettings.singleton(),
&dkim.ctx_dkim_sign_domain(),
),
send: bp.compile_expr(
ObjectType::DkimReportSettings.singleton(),
&dkim.ctx_send_frequency(),
),
},
spf: Report {
name: bp.compile_expr(
ObjectType::SpfReportSettings.singleton(),
&spf.ctx_from_name(),
),
address: bp.compile_expr(
ObjectType::SpfReportSettings.singleton(),
&spf.ctx_from_address(),
),
subject: bp.compile_expr(
ObjectType::SpfReportSettings.singleton(),
&spf.ctx_subject(),
),
sign: bp.compile_expr(
ObjectType::SpfReportSettings.singleton(),
&spf.ctx_dkim_sign_domain(),
),
send: bp.compile_expr(
ObjectType::SpfReportSettings.singleton(),
&spf.ctx_send_frequency(),
),
},
dmarc: Report {
name: bp.compile_expr(
ObjectType::DmarcReportSettings.singleton(),
&dmarc.ctx_failure_from_name(),
),
address: bp.compile_expr(
ObjectType::DmarcReportSettings.singleton(),
&dmarc.ctx_failure_from_address(),
),
subject: bp.compile_expr(
ObjectType::DmarcReportSettings.singleton(),
&dmarc.ctx_failure_subject(),
),
sign: bp.compile_expr(
ObjectType::DmarcReportSettings.singleton(),
&dmarc.ctx_failure_dkim_sign_domain(),
),
send: bp.compile_expr(
ObjectType::DmarcReportSettings.singleton(),
&dmarc.ctx_failure_send_frequency(),
),
},
dmarc_aggregate: AggregateReport {
name: bp.compile_expr(
ObjectType::DmarcReportSettings.singleton(),
&dmarc.ctx_aggregate_from_name(),
),
address: bp.compile_expr(
ObjectType::DmarcReportSettings.singleton(),
&dmarc.ctx_aggregate_from_address(),
),
org_name: bp.compile_expr(
ObjectType::DmarcReportSettings.singleton(),
&dmarc.ctx_aggregate_org_name(),
),
contact_info: bp.compile_expr(
ObjectType::DmarcReportSettings.singleton(),
&dmarc.ctx_aggregate_contact_info(),
),
send: bp.compile_expr(
ObjectType::DmarcReportSettings.singleton(),
&dmarc.ctx_aggregate_send_frequency(),
),
sign: bp.compile_expr(
ObjectType::DmarcReportSettings.singleton(),
&dmarc.ctx_aggregate_dkim_sign_domain(),
),
max_size: bp.compile_expr(
ObjectType::DmarcReportSettings.singleton(),
&dmarc.ctx_aggregate_max_report_size(),
),
},
tls: AggregateReport {
name: bp.compile_expr(
ObjectType::TlsReportSettings.singleton(),
&tls.ctx_from_name(),
),
address: bp.compile_expr(
ObjectType::TlsReportSettings.singleton(),
&tls.ctx_from_address(),
),
org_name: bp.compile_expr(
ObjectType::TlsReportSettings.singleton(),
&tls.ctx_org_name(),
),
contact_info: bp.compile_expr(
ObjectType::TlsReportSettings.singleton(),
&tls.ctx_contact_info(),
),
send: bp.compile_expr(
ObjectType::TlsReportSettings.singleton(),
&tls.ctx_send_frequency(),
),
sign: bp.compile_expr(
ObjectType::TlsReportSettings.singleton(),
&tls.ctx_dkim_sign_domain(),
),
max_size: bp.compile_expr(
ObjectType::TlsReportSettings.singleton(),
&tls.ctx_max_report_size(),
),
},
}
}
}
impl<'x> TryFrom<Variable<'x>> for AggregateFrequency {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
match value {
Variable::Constant(ExpressionConstant::Disable) => Ok(AggregateFrequency::Never),
Variable::Constant(ExpressionConstant::Hourly) => Ok(AggregateFrequency::Hourly),
Variable::Constant(ExpressionConstant::Daily) => Ok(AggregateFrequency::Daily),
Variable::Constant(ExpressionConstant::Weekly) => Ok(AggregateFrequency::Weekly),
_ => Err(()),
}
}
}
impl ReportAnalysis {
pub fn is_report_address(&self, address: &str) -> bool {
self.addresses.iter().any(|addr_match| match addr_match {
AddressMatch::StartsWith(prefix) => address.starts_with(prefix),
AddressMatch::EndsWith(suffix) => address.ends_with(suffix),
AddressMatch::Equals(value) => address == value,
})
}
}
impl FromStr for AddressMatch {
type Err = String;
fn from_str(value: &str) -> Result<Self, Self::Err> {
if let Some(value) = value.strip_prefix('*').map(|v| v.trim()) {
if !value.is_empty() {
return Ok(AddressMatch::EndsWith(value.to_lowercase()));
}
} else if let Some(value) = value.strip_suffix('*').map(|v| v.trim()) {
if !value.is_empty() {
return Ok(AddressMatch::StartsWith(value.to_lowercase()));
}
} else if value.contains('@') {
return Ok(AddressMatch::Equals(value.trim().to_lowercase()));
}
Err(format!("Invalid address match value {:?}.", value,))
}
}
crates/common/src/config/smtp/resolver.rs
+406
View File
@@ -0,0 +1,406 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use ahash::AHashMap;
use mail_auth::{
MessageAuthenticator,
hickory_resolver::{
TokioResolver,
config::{
CLOUDFLARE, ConnectionConfig, GOOGLE, NameServerConfig, ProtocolConfig, QUAD9,
ResolverConfig, ResolverOpts,
},
net::runtime::TokioRuntimeProvider,
system_conf::read_system_conf,
},
};
use registry::schema::{
enums::{DnsResolverProtocol, PolicyEnforcement},
prelude::ObjectType,
structs::{DnsResolver, MtaSts, SystemSettings},
};
use serde::{Deserialize, Serialize};
use std::{
fmt::Display,
hash::{DefaultHasher, Hash, Hasher},
net::IpAddr,
str::FromStr,
sync::Arc,
};
use store::registry::bootstrap::Bootstrap;
use utils::cache::CacheItemWeight;
pub struct Resolvers {
pub dns: MessageAuthenticator,
pub dnssec: DnssecResolver,
pub dnssec_available: bool,
}
#[derive(Clone)]
pub struct DnssecResolver {
pub resolver: TokioResolver,
}
#[derive(Debug, Clone, Copy, Hash, PartialEq, Eq, Serialize, Deserialize)]
pub enum TlsaMatching {
Full,
Sha256,
Sha512,
}
#[derive(Debug, Clone, Hash, PartialEq, Eq, Serialize, Deserialize)]
pub struct TlsaEntry {
pub is_end_entity: bool,
pub is_spki: bool,
pub matching: TlsaMatching,
pub data: Vec<u8>,
}
#[derive(Debug, Clone, Hash, PartialEq, Eq, Serialize, Deserialize)]
pub struct Tlsa {
pub entries: Vec<TlsaEntry>,
pub has_end_entities: bool,
pub has_intermediates: bool,
}
#[derive(Debug, PartialEq, Eq, Hash, Default, Clone, Copy, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum Mode {
Enforce,
Testing,
#[default]
None,
}
#[derive(Debug, PartialEq, Eq, Hash, PartialOrd, Ord, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum MxPattern {
Equals(String),
StartsWith(String),
}
#[derive(Debug, PartialEq, Eq, Hash, Clone, Serialize, Deserialize)]
pub struct Policy {
pub id: String,
pub mode: Mode,
pub mx: Box<[MxPattern]>,
pub max_age: u64,
}
impl CacheItemWeight for Tlsa {
fn weight(&self) -> u64 {
self.entries
.iter()
.map(|entry| (entry.data.len() + std::mem::size_of::<TlsaEntry>()) as u64)
.sum::<u64>()
+ std::mem::size_of::<Tlsa>() as u64
}
}
impl CacheItemWeight for Policy {
fn weight(&self) -> u64 {
(std::mem::size_of::<Policy>()
+ self
.mx
.iter()
.map(|mx| match mx {
MxPattern::Equals(t) => t.len(),
MxPattern::StartsWith(t) => t.len(),
})
.sum::<usize>()) as u64
}
}
impl Resolvers {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let mut resolver_config: ResolverConfig;
let mut opts = ResolverOpts::default();
match bp.setting_infallible::<DnsResolver>().await {
DnsResolver::System(resolver) => match read_system_conf() {
Ok((config, options)) => {
resolver_config = config;
opts = options;
opts.num_concurrent_reqs = resolver.concurrency as usize;
opts.timeout = resolver.timeout.into_inner();
opts.preserve_intermediates = resolver.preserve_intermediates;
opts.try_tcp_on_error = resolver.tcp_on_error;
opts.attempts = resolver.attempts as usize;
opts.edns0 = resolver.enable_edns;
}
Err(err) => {
bp.build_error(
ObjectType::DnsResolver.singleton(),
format!("Failed to read system DNS config: {err}"),
);
resolver_config = ResolverConfig::udp_and_tcp(&CLOUDFLARE);
}
},
DnsResolver::Custom(resolver) => {
resolver_config = ResolverConfig::default();
let mut nameservers: AHashMap<IpAddr, Vec<ConnectionConfig>> = AHashMap::new();
for server in resolver.servers {
let ip = server.address.into_inner();
let port = server.port as u16;
let protocol = match server.protocol {
DnsResolverProtocol::Udp => ProtocolConfig::Udp,
DnsResolverProtocol::Tcp => ProtocolConfig::Tcp,
DnsResolverProtocol::Tls => ProtocolConfig::Tls {
server_name: Arc::from(server.address.to_string()),
},
};
let mut connection = ConnectionConfig::new(protocol);
connection.port = port;
nameservers.entry(ip).or_default().push(connection);
}
for (ip, connections) in nameservers {
resolver_config.add_name_server(NameServerConfig::new(ip, true, connections));
}
opts.num_concurrent_reqs = resolver.concurrency as usize;
opts.timeout = resolver.timeout.into_inner();
opts.preserve_intermediates = resolver.preserve_intermediates;
opts.try_tcp_on_error = resolver.tcp_on_error;
opts.attempts = resolver.attempts as usize;
opts.edns0 = resolver.enable_edns;
}
DnsResolver::Cloudflare(resolver) => {
resolver_config = if resolver.use_tls {
ResolverConfig::tls(&CLOUDFLARE)
} else {
ResolverConfig::udp_and_tcp(&CLOUDFLARE)
};
opts.num_concurrent_reqs = resolver.concurrency as usize;
opts.timeout = resolver.timeout.into_inner();
opts.preserve_intermediates = resolver.preserve_intermediates;
opts.try_tcp_on_error = resolver.tcp_on_error;
opts.attempts = resolver.attempts as usize;
opts.edns0 = resolver.enable_edns;
}
DnsResolver::Quad9(resolver) => {
resolver_config = if resolver.use_tls {
ResolverConfig::tls(&QUAD9)
} else {
ResolverConfig::udp_and_tcp(&QUAD9)
};
opts.num_concurrent_reqs = resolver.concurrency as usize;
opts.timeout = resolver.timeout.into_inner();
opts.preserve_intermediates = resolver.preserve_intermediates;
opts.try_tcp_on_error = resolver.tcp_on_error;
opts.attempts = resolver.attempts as usize;
opts.edns0 = resolver.enable_edns;
}
DnsResolver::Google(resolver) => {
resolver_config = ResolverConfig::udp_and_tcp(&GOOGLE);
opts.num_concurrent_reqs = resolver.concurrency as usize;
opts.timeout = resolver.timeout.into_inner();
opts.preserve_intermediates = resolver.preserve_intermediates;
opts.try_tcp_on_error = resolver.tcp_on_error;
opts.attempts = resolver.attempts as usize;
opts.edns0 = resolver.enable_edns;
}
}
// We already have a cache, so disable the built-in cache
opts.cache_size = 0;
// Prepare DNSSEC resolver options
let config_dnssec = resolver_config.clone();
let mut opts_dnssec = opts.clone();
opts_dnssec.validate = true;
let dnssec = DnssecResolver {
resolver: TokioResolver::builder_with_config(
config_dnssec,
TokioRuntimeProvider::default(),
)
.with_options(opts_dnssec)
.build()
.expect("Failed to build DNSSEC resolver"),
};
Resolvers {
#[cfg(not(feature = "test_mode"))]
dnssec_available: ensure_dnssec(&resolver_config, &dnssec.resolver).await,
#[cfg(feature = "test_mode")]
dnssec_available: true,
dns: MessageAuthenticator::new(resolver_config, opts).unwrap(),
dnssec,
}
}
}
#[cfg(not(feature = "test_mode"))]
async fn ensure_dnssec(config: &ResolverConfig, resolver: &TokioResolver) -> bool {
config.name_servers().iter().any(|name_server| {
name_server
.connections
.iter()
.any(|connection| !matches!(connection.protocol, ProtocolConfig::Udp))
}) && resolver
.lookup(
hickory_proto::rr::Name::root(),
hickory_proto::rr::RecordType::DNSKEY,
)
.await
.is_ok_and(|lookup| {
lookup
.answers()
.iter()
.any(|record| record.proof.is_secure())
})
}
impl Policy {
pub async fn try_parse(bp: &mut Bootstrap) -> Option<Self> {
let mta = bp.setting_infallible::<MtaSts>().await;
if matches!(mta.mode, PolicyEnforcement::Disable) {
return None;
}
let mut mx_hosts = mta.mx_hosts.into_inner();
if mx_hosts.is_empty() {
let settings = bp.setting_infallible::<SystemSettings>().await;
let default_host = settings.default_hostname.as_str();
mx_hosts = settings
.mail_exchangers
.iter()
.map(|mx| mx.hostname.as_deref().unwrap_or(default_host).to_string())
.collect();
}
if !mx_hosts.is_empty() {
mx_hosts.sort_unstable();
mx_hosts.dedup();
let mut policy = Policy {
id: Default::default(),
mode: match mta.mode {
PolicyEnforcement::Enforce => Mode::Enforce,
PolicyEnforcement::Testing => Mode::Testing,
PolicyEnforcement::Disable => Mode::None,
},
mx: mx_hosts
.into_iter()
.map(|mx| {
if let Some(mx) = mx.strip_prefix("*.") {
MxPattern::StartsWith(mx.to_string())
} else {
MxPattern::Equals(mx)
}
})
.collect(),
max_age: mta.max_age.into_inner().as_secs(),
};
policy.id = policy.hash().to_string();
Some(policy)
} else {
None
}
}
fn hash(&self) -> u64 {
let mut s = DefaultHasher::new();
self.mode.hash(&mut s);
self.max_age.hash(&mut s);
self.mx.hash(&mut s);
s.finish()
}
}
impl FromStr for Mode {
type Err = String;
fn from_str(value: &str) -> Result<Self, Self::Err> {
match value {
"enforce" => Ok(Self::Enforce),
"testing" | "test" => Ok(Self::Testing),
"none" => Ok(Self::None),
_ => Err(format!("Invalid mode value {value:?}")),
}
}
}
impl Default for Resolvers {
fn default() -> Self {
let (config, opts) = match read_system_conf() {
Ok(conf) => conf,
Err(_) => (
ResolverConfig::udp_and_tcp(&CLOUDFLARE),
ResolverOpts::default(),
),
};
let config_dnssec = config.clone();
let mut opts_dnssec = opts.clone();
opts_dnssec.validate = true;
Self {
dns: MessageAuthenticator::new(config, opts).expect("Failed to build DNS resolver"),
dnssec: DnssecResolver {
resolver: TokioResolver::builder_with_config(
config_dnssec,
TokioRuntimeProvider::default(),
)
.with_options(opts_dnssec)
.build()
.expect("Failed to build DNSSEC resolver"),
},
dnssec_available: true,
}
}
}
impl Display for Policy {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("version: STSv1\r\n")?;
f.write_str("mode: ")?;
match self.mode {
Mode::Enforce => f.write_str("enforce")?,
Mode::Testing => f.write_str("testing")?,
Mode::None => f.write_str("none")?,
}
f.write_str("\r\nmax_age: ")?;
self.max_age.fmt(f)?;
f.write_str("\r\n")?;
for mx in &self.mx {
f.write_str("mx: ")?;
mx.fmt(f)?;
f.write_str("\r\n")?;
}
Ok(())
}
}
impl Display for MxPattern {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
MxPattern::Equals(mx) => f.write_str(mx),
MxPattern::StartsWith(mx) => {
f.write_str("*.")?;
f.write_str(mx)
}
}
}
}
impl Clone for Resolvers {
fn clone(&self) -> Self {
Self {
dns: self.dns.clone(),
dnssec: self.dnssec.clone(),
dnssec_available: self.dnssec_available,
}
}
}
crates/common/src/config/smtp/session.rs
+567
View File
@@ -0,0 +1,567 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use self::resolver::Policy;
use super::*;
use crate::expr::{
Variable,
if_block::{BootstrapExprExt, IfBlock},
};
use ahash::AHashSet;
use hyper::HeaderMap;
use registry::schema::{
enums::{self, ExpressionConstant, MtaStage},
prelude::ObjectType,
structs::{
MtaExtensions, MtaHook, MtaInboundSession, MtaMilter, MtaStageAuth, MtaStageConnect,
MtaStageData, MtaStageEhlo, MtaStageMail, MtaStageRcpt,
},
};
use smtp_proto::*;
use std::{
net::{SocketAddr, ToSocketAddrs},
str::FromStr,
time::Duration,
};
#[derive(Clone)]
pub struct SessionConfig {
pub timeout: IfBlock,
pub duration: IfBlock,
pub transfer_limit: IfBlock,
pub connect: Connect,
pub ehlo: Ehlo,
pub auth: Auth,
pub mail: Mail,
pub rcpt: Rcpt,
pub data: Data,
pub extensions: Extensions,
pub mta_sts_policy: Option<Policy>,
pub milters: Vec<Milter>,
pub hooks: Vec<MTAHook>,
}
#[derive(Clone)]
pub struct Connect {
pub hostname: IfBlock,
pub script: IfBlock,
pub greeting: IfBlock,
}
#[derive(Clone)]
pub struct Ehlo {
pub script: IfBlock,
pub require: IfBlock,
pub reject_non_fqdn: IfBlock,
}
#[derive(Clone)]
pub struct Extensions {
pub pipelining: IfBlock,
pub chunking: IfBlock,
pub requiretls: IfBlock,
pub dsn: IfBlock,
pub vrfy: IfBlock,
pub expn: IfBlock,
pub no_soliciting: IfBlock,
pub future_release: IfBlock,
pub deliver_by: IfBlock,
pub mt_priority: IfBlock,
}
#[derive(Clone)]
pub struct Auth {
pub mechanisms: IfBlock,
pub require: IfBlock,
pub must_match_sender: IfBlock,
pub errors_max: IfBlock,
pub errors_wait: IfBlock,
}
#[derive(Clone)]
pub struct Mail {
pub script: IfBlock,
pub rewrite: IfBlock,
pub is_allowed: IfBlock,
}
#[derive(Clone)]
pub struct Rcpt {
pub script: IfBlock,
pub relay: IfBlock,
pub rewrite: IfBlock,
pub errors_max: IfBlock,
pub errors_wait: IfBlock,
pub max_recipients: IfBlock,
}
#[derive(Debug, Default, Clone)]
pub enum AddressMapping {
Enable,
Custom(IfBlock),
#[default]
Disable,
}
#[derive(Clone)]
pub struct Data {
pub script: IfBlock,
pub spam_filter: IfBlock,
pub max_messages: IfBlock,
pub max_message_size: IfBlock,
pub max_received_headers: IfBlock,
pub add_received: IfBlock,
pub add_received_spf: IfBlock,
pub add_return_path: IfBlock,
pub add_auth_results: IfBlock,
pub add_message_id: IfBlock,
pub add_date: IfBlock,
pub add_delivered_to: bool,
}
#[derive(Clone)]
pub struct Milter {
pub enable: IfBlock,
pub id: ObjectId,
pub addrs: Vec<SocketAddr>,
pub hostname: String,
pub port: u16,
pub timeout_connect: Duration,
pub timeout_command: Duration,
pub timeout_data: Duration,
pub tls: bool,
pub tls_allow_invalid_certs: bool,
pub tempfail_on_error: bool,
pub max_frame_len: usize,
pub protocol_version: MilterVersion,
pub flags_actions: Option<u32>,
pub flags_protocol: Option<u32>,
pub run_on_stage: AHashSet<Stage>,
}
#[derive(Clone, Copy)]
pub enum MilterVersion {
V2,
V6,
}
#[derive(Clone)]
pub struct MTAHook {
pub enable: IfBlock,
pub id: ObjectId,
pub url: String,
pub timeout: Duration,
pub headers: HeaderMap,
pub tls_allow_invalid_certs: bool,
pub tempfail_on_error: bool,
pub run_on_stage: AHashSet<Stage>,
pub max_response_size: usize,
pub client: reqwest::Client,
}
#[derive(Clone, Copy, PartialEq, Eq, Hash)]
pub enum Stage {
Connect,
Ehlo,
Auth,
Mail,
Rcpt,
Data,
}
impl SessionConfig {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let session = bp.setting_infallible::<MtaInboundSession>().await;
let connect = bp.setting_infallible::<MtaStageConnect>().await;
let auth = bp.setting_infallible::<MtaStageAuth>().await;
let ehlo = bp.setting_infallible::<MtaStageEhlo>().await;
let mail = bp.setting_infallible::<MtaStageMail>().await;
let rcpt = bp.setting_infallible::<MtaStageRcpt>().await;
let data = bp.setting_infallible::<MtaStageData>().await;
let ext = bp.setting_infallible::<MtaExtensions>().await;
let mut hooks = Vec::new();
for hook in bp.list_infallible::<MtaHook>().await {
let id = hook.id;
let hook = hook.object;
let enable = bp.compile_expr(id, &hook.ctx_enable());
let headers = match hook
.http_auth
.build_headers(hook.http_headers, "application/json".into())
.await
{
Ok(headers) => headers,
Err(err) => {
bp.build_error(id, format!("Unable to build HTTP headers: {}", err));
continue;
}
};
hooks.push(MTAHook {
enable,
id,
url: hook.url,
timeout: hook.timeout.into_inner(),
headers,
tls_allow_invalid_certs: hook.allow_invalid_certs,
tempfail_on_error: hook.temp_fail_on_error,
run_on_stage: hook.stages.into_iter().map(Stage::from).collect(),
max_response_size: hook.max_response_size as usize,
client: utils::http::http_client_builder(hook.allow_invalid_certs)
.build()
.unwrap_or_default(),
});
}
SessionConfig {
timeout: bp.compile_expr(
ObjectType::MtaInboundSession.singleton(),
&session.ctx_timeout(),
),
duration: bp.compile_expr(
ObjectType::MtaInboundSession.singleton(),
&session.ctx_max_duration(),
),
transfer_limit: bp.compile_expr(
ObjectType::MtaInboundSession.singleton(),
&session.ctx_transfer_limit(),
),
connect: Connect {
hostname: bp.compile_expr(
ObjectType::MtaStageConnect.singleton(),
&connect.ctx_hostname(),
),
script: bp.compile_expr(
ObjectType::MtaStageConnect.singleton(),
&connect.ctx_script(),
),
greeting: bp.compile_expr(
ObjectType::MtaStageConnect.singleton(),
&connect.ctx_smtp_greeting(),
),
},
ehlo: Ehlo {
script: bp.compile_expr(ObjectType::MtaStageEhlo.singleton(), &ehlo.ctx_script()),
require: bp.compile_expr(ObjectType::MtaStageEhlo.singleton(), &ehlo.ctx_require()),
reject_non_fqdn: bp.compile_expr(
ObjectType::MtaStageEhlo.singleton(),
&ehlo.ctx_reject_non_fqdn(),
),
},
auth: Auth {
mechanisms: bp.compile_expr(
ObjectType::MtaStageAuth.singleton(),
&auth.ctx_sasl_mechanisms(),
),
require: bp.compile_expr(ObjectType::MtaStageAuth.singleton(), &auth.ctx_require()),
must_match_sender: bp.compile_expr(
ObjectType::MtaStageAuth.singleton(),
&auth.ctx_must_match_sender(),
),
errors_max: bp.compile_expr(
ObjectType::MtaStageAuth.singleton(),
&auth.ctx_max_failures(),
),
errors_wait: bp.compile_expr(
ObjectType::MtaStageAuth.singleton(),
&auth.ctx_wait_on_fail(),
),
},
mail: Mail {
script: bp.compile_expr(ObjectType::MtaStageMail.singleton(), &mail.ctx_script()),
rewrite: bp.compile_expr(ObjectType::MtaStageMail.singleton(), &mail.ctx_rewrite()),
is_allowed: bp.compile_expr(
ObjectType::MtaStageMail.singleton(),
&mail.ctx_is_sender_allowed(),
),
},
rcpt: Rcpt {
script: bp.compile_expr(ObjectType::MtaStageRcpt.singleton(), &rcpt.ctx_script()),
relay: bp.compile_expr(
ObjectType::MtaStageRcpt.singleton(),
&rcpt.ctx_allow_relaying(),
),
rewrite: bp.compile_expr(ObjectType::MtaStageRcpt.singleton(), &rcpt.ctx_rewrite()),
errors_max: bp.compile_expr(
ObjectType::MtaStageRcpt.singleton(),
&rcpt.ctx_max_failures(),
),
errors_wait: bp.compile_expr(
ObjectType::MtaStageRcpt.singleton(),
&rcpt.ctx_wait_on_fail(),
),
max_recipients: bp.compile_expr(
ObjectType::MtaStageRcpt.singleton(),
&rcpt.ctx_max_recipients(),
),
},
data: Data {
script: bp.compile_expr(ObjectType::MtaStageData.singleton(), &data.ctx_script()),
spam_filter: bp.compile_expr(
ObjectType::MtaStageData.singleton(),
&data.ctx_enable_spam_filter(),
),
max_messages: bp.compile_expr(
ObjectType::MtaStageData.singleton(),
&data.ctx_max_messages(),
),
max_message_size: bp.compile_expr(
ObjectType::MtaStageData.singleton(),
&data.ctx_max_message_size(),
),
max_received_headers: bp.compile_expr(
ObjectType::MtaStageData.singleton(),
&data.ctx_max_received_headers(),
),
add_received: bp.compile_expr(
ObjectType::MtaStageData.singleton(),
&data.ctx_add_received_header(),
),
add_received_spf: bp.compile_expr(
ObjectType::MtaStageData.singleton(),
&data.ctx_add_received_spf_header(),
),
add_return_path: bp.compile_expr(
ObjectType::MtaStageData.singleton(),
&data.ctx_add_return_path_header(),
),
add_auth_results: bp.compile_expr(
ObjectType::MtaStageData.singleton(),
&data.ctx_add_auth_results_header(),
),
add_message_id: bp.compile_expr(
ObjectType::MtaStageData.singleton(),
&data.ctx_add_message_id_header(),
),
add_date: bp.compile_expr(
ObjectType::MtaStageData.singleton(),
&data.ctx_add_date_header(),
),
add_delivered_to: data.add_delivered_to_header,
},
extensions: Extensions {
pipelining: bp
.compile_expr(ObjectType::MtaExtensions.singleton(), &ext.ctx_pipelining()),
chunking: bp
.compile_expr(ObjectType::MtaExtensions.singleton(), &ext.ctx_chunking()),
requiretls: bp.compile_expr(
ObjectType::MtaExtensions.singleton(),
&ext.ctx_require_tls(),
),
dsn: bp.compile_expr(ObjectType::MtaExtensions.singleton(), &ext.ctx_dsn()),
vrfy: bp.compile_expr(ObjectType::MtaExtensions.singleton(), &ext.ctx_vrfy()),
expn: bp.compile_expr(ObjectType::MtaExtensions.singleton(), &ext.ctx_expn()),
no_soliciting: bp.compile_expr(
ObjectType::MtaExtensions.singleton(),
&ext.ctx_no_soliciting(),
),
future_release: bp.compile_expr(
ObjectType::MtaExtensions.singleton(),
&ext.ctx_future_release(),
),
deliver_by: bp
.compile_expr(ObjectType::MtaExtensions.singleton(), &ext.ctx_deliver_by()),
mt_priority: bp.compile_expr(
ObjectType::MtaExtensions.singleton(),
&ext.ctx_mt_priority(),
),
},
mta_sts_policy: Policy::try_parse(bp).await,
milters: bp
.list_infallible::<MtaMilter>()
.await
.into_iter()
.filter_map(|milter| {
let id = milter.id;
let milter = milter.object;
Some(Milter {
enable: bp.compile_expr(id, &milter.ctx_enable()),
id,
addrs: format!("{}:{}", milter.hostname, milter.port)
.to_socket_addrs()
.map_err(|err| {
bp.build_error(
id,
format!(
"Unable to resolve milter hostname {}: {}",
milter.hostname, err
),
)
})
.ok()?
.collect(),
hostname: milter.hostname,
port: milter.port as u16,
timeout_connect: milter.timeout_connect.into_inner(),
timeout_command: milter.timeout_command.into_inner(),
timeout_data: milter.timeout_data.into_inner(),
tls: milter.use_tls,
tls_allow_invalid_certs: milter.allow_invalid_certs,
tempfail_on_error: milter.temp_fail_on_error,
max_frame_len: milter.max_response_size as usize,
protocol_version: match milter.protocol_version {
enums::MilterVersion::V2 => MilterVersion::V2,
enums::MilterVersion::V6 => MilterVersion::V6,
},
flags_actions: milter.flags_action.map(|v| v as u32),
flags_protocol: milter.flags_protocol.map(|v| v as u32),
run_on_stage: milter.stages.into_iter().map(Stage::from).collect(),
})
})
.collect(),
hooks,
}
}
}
#[derive(Default)]
pub struct Mechanism(u64);
impl FromStr for Mechanism {
type Err = String;
fn from_str(value: &str) -> Result<Self, Self::Err> {
Ok(Mechanism(match value.to_ascii_uppercase().as_str() {
"LOGIN" => AUTH_LOGIN,
"PLAIN" => AUTH_PLAIN,
"XOAUTH2" => AUTH_XOAUTH2,
"OAUTHBEARER" => AUTH_OAUTHBEARER,
/*"SCRAM-SHA-256-PLUS" => AUTH_SCRAM_SHA_256_PLUS,
"SCRAM-SHA-256" => AUTH_SCRAM_SHA_256,
"SCRAM-SHA-1-PLUS" => AUTH_SCRAM_SHA_1_PLUS,
"SCRAM-SHA-1" => AUTH_SCRAM_SHA_1,
"XOAUTH" => AUTH_XOAUTH,
"9798-M-DSA-SHA1" => AUTH_9798_M_DSA_SHA1,
"9798-M-ECDSA-SHA1" => AUTH_9798_M_ECDSA_SHA1,
"9798-M-RSA-SHA1-ENC" => AUTH_9798_M_RSA_SHA1_ENC,
"9798-U-DSA-SHA1" => AUTH_9798_U_DSA_SHA1,
"9798-U-ECDSA-SHA1" => AUTH_9798_U_ECDSA_SHA1,
"9798-U-RSA-SHA1-ENC" => AUTH_9798_U_RSA_SHA1_ENC,
"EAP-AES128" => AUTH_EAP_AES128,
"EAP-AES128-PLUS" => AUTH_EAP_AES128_PLUS,
"ECDH-X25519-CHALLENGE" => AUTH_ECDH_X25519_CHALLENGE,
"ECDSA-NIST256P-CHALLENGE" => AUTH_ECDSA_NIST256P_CHALLENGE,
"EXTERNAL" => AUTH_EXTERNAL,
"GS2-KRB5" => AUTH_GS2_KRB5,
"GS2-KRB5-PLUS" => AUTH_GS2_KRB5_PLUS,
"GSS-SPNEGO" => AUTH_GSS_SPNEGO,
"GSSAPI" => AUTH_GSSAPI,
"KERBEROS_V4" => AUTH_KERBEROS_V4,
"KERBEROS_V5" => AUTH_KERBEROS_V5,
"NMAS-SAMBA-AUTH" => AUTH_NMAS_SAMBA_AUTH,
"NMAS_AUTHEN" => AUTH_NMAS_AUTHEN,
"NMAS_LOGIN" => AUTH_NMAS_LOGIN,
"NTLM" => AUTH_NTLM,
"OAUTH10A" => AUTH_OAUTH10A,
"OPENID20" => AUTH_OPENID20,
"OTP" => AUTH_OTP,
"SAML20" => AUTH_SAML20,
"SECURID" => AUTH_SECURID,
"SKEY" => AUTH_SKEY,
"SPNEGO" => AUTH_SPNEGO,
"SPNEGO-PLUS" => AUTH_SPNEGO_PLUS,
"SXOVER-PLUS" => AUTH_SXOVER_PLUS,
"CRAM-MD5" => AUTH_CRAM_MD5,
"DIGEST-MD5" => AUTH_DIGEST_MD5,
"ANONYMOUS" => AUTH_ANONYMOUS,*/
_ => return Err(format!("Unsupported mechanism {:?}.", value)),
}))
}
}
impl<'x> TryFrom<Variable<'x>> for Mechanism {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
match value {
Variable::Constant(value) => Mechanism::try_from(value),
Variable::Array(items) => {
let mut mechanism = 0;
for item in items {
match item {
Variable::Constant(value) => mechanism |= Mechanism::try_from(value)?.0,
_ => return Err(()),
}
}
Ok(Mechanism(mechanism))
}
_ => Err(()),
}
}
}
impl TryFrom<ExpressionConstant> for Mechanism {
type Error = ();
fn try_from(value: ExpressionConstant) -> Result<Self, Self::Error> {
match value {
ExpressionConstant::Login => Ok(Mechanism(AUTH_LOGIN)),
ExpressionConstant::Plain => Ok(Mechanism(AUTH_PLAIN)),
ExpressionConstant::Xoauth2 => Ok(Mechanism(AUTH_XOAUTH2)),
ExpressionConstant::Oauthbearer => Ok(Mechanism(AUTH_OAUTHBEARER)),
_ => Err(()),
}
}
}
impl From<Mechanism> for u64 {
fn from(value: Mechanism) -> Self {
value.0
}
}
impl From<u64> for Mechanism {
fn from(value: u64) -> Self {
Mechanism(value)
}
}
impl<'x> TryFrom<Variable<'x>> for MtPriority {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
match value {
Variable::Constant(value) => match value {
ExpressionConstant::Mixer => Ok(MtPriority::Mixer),
ExpressionConstant::Stanag4406 => Ok(MtPriority::Stanag4406),
ExpressionConstant::Nsep => Ok(MtPriority::Nsep),
_ => Err(()),
},
Variable::String(value) => {
let value = value.as_str();
if value.eq_ignore_ascii_case("MIXER") {
Ok(MtPriority::Mixer)
} else if value.eq_ignore_ascii_case("STANAG4406") {
Ok(MtPriority::Stanag4406)
} else if value.eq_ignore_ascii_case("NSEP") {
Ok(MtPriority::Nsep)
} else {
Err(())
}
}
_ => Err(()),
}
}
}
impl From<MtaStage> for Stage {
fn from(value: MtaStage) -> Self {
match value {
MtaStage::Connect => Stage::Connect,
MtaStage::Ehlo => Stage::Ehlo,
MtaStage::Auth => Stage::Auth,
MtaStage::Mail => Stage::Mail,
MtaStage::Rcpt => Stage::Rcpt,
MtaStage::Data => Stage::Data,
}
}
}
crates/common/src/config/storage.rs
+57
View File
@@ -0,0 +1,57 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use coordinator::Coordinator;
use directory::{Directories, Directory};
use registry::schema::prelude::ObjectType;
use std::{collections::HashMap, sync::Arc};
use store::{
BlobStore, InMemoryStore, RegistryStore, SearchStore, Store, registry::bootstrap::Bootstrap,
};
pub type IdMap<V> = HashMap<u32, Arc<V>, nohash_hasher::BuildNoHashHasher<u32>>;
#[derive(Clone)]
pub struct Storage {
pub registry: RegistryStore,
pub data: Store,
pub blob: BlobStore,
pub search: SearchStore,
pub memory: InMemoryStore,
pub metrics: Store,
pub tracing: Store,
pub coordinator: Coordinator,
pub directory: Option<Arc<Directory>>,
pub directories: IdMap<Directory>,
}
impl Storage {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let memory = InMemoryStore::build(bp).await.unwrap_or_default();
let directory = Directories::build(bp).await;
let search = SearchStore::build(bp).await.unwrap_or_default();
if let Err(err) = search.create_indexes().await {
bp.build_warning(
ObjectType::SearchStore.singleton(),
format!("Failed to create search indexes: {err}"),
);
}
Storage {
registry: bp.registry.clone(),
data: bp.data_store.clone(),
blob: BlobStore::build(bp).await.unwrap_or_default(),
search,
coordinator: Coordinator::build(bp, &memory).await.unwrap_or_default(),
memory,
tracing: Store::build_tracing(bp).await.unwrap_or_default(),
metrics: Store::build_metrics(bp).await.unwrap_or_default(),
directory: directory.default_directory,
directories: directory.directories,
}
}
}
crates/common/src/config/telemetry.rs
+716
View File
@@ -0,0 +1,716 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::config::storage::Storage;
use ahash::{AHashMap, AHashSet};
use base64::{Engine, engine::general_purpose::STANDARD};
use hyper::HeaderMap;
use opentelemetry::{InstrumentationScope, KeyValue};
use opentelemetry_otlp::{
LogExporter, MetricExporter, SpanExporter, WithExportConfig, WithHttpConfig,
};
use opentelemetry_sdk::{Resource, metrics::Temporality};
use opentelemetry_semantic_conventions::resource::SERVICE_VERSION;
use registry::schema::{
enums::{EventPolicy, LogRotateFrequency},
prelude::ObjectType,
structs::{self, EventTracingLevel, MetricsPrometheus, Tracer, WebHook},
};
use std::{collections::HashMap, str::FromStr, sync::Arc, time::Duration};
use store::registry::bootstrap::Bootstrap;
use trc::{EventType, Level, MetricType, TelemetryEvent, ipc::subscriber::Interests};
#[derive(Debug)]
pub struct TelemetrySubscriber {
pub id: String,
pub interests: Interests,
pub typ: TelemetrySubscriberType,
pub lossy: bool,
}
#[allow(clippy::large_enum_variant)]
#[derive(Debug)]
pub enum TelemetrySubscriberType {
ConsoleTracer(ConsoleTracer),
LogTracer(LogTracer),
OtelTracer(OtelTracer),
Webhook(WebhookTracer),
#[cfg(unix)]
JournalTracer(crate::telemetry::tracers::journald::Subscriber),
}
#[derive(Debug)]
pub struct OtelTracer {
pub span_exporter: SpanExporter,
pub span_exporter_enable: bool,
pub log_exporter: LogExporter,
pub log_exporter_enable: bool,
pub throttle: Duration,
}
pub struct OtelMetrics {
pub resource: Resource,
pub instrumentation: InstrumentationScope,
pub exporter: MetricExporter,
pub interval: Duration,
}
#[derive(Debug)]
pub struct ConsoleTracer {
pub ansi: bool,
pub multiline: bool,
pub buffered: bool,
}
#[derive(Debug)]
pub struct LogTracer {
pub path: String,
pub prefix: String,
pub rotate: RotationStrategy,
pub ansi: bool,
pub multiline: bool,
}
#[derive(Debug)]
pub struct WebhookTracer {
pub url: String,
pub key: String,
pub timeout: Duration,
pub throttle: Duration,
pub discard_after: Duration,
pub tls_allow_invalid_certs: bool,
pub headers: HeaderMap,
pub client: reqwest::Client,
}
#[derive(Debug)]
pub enum RotationStrategy {
Daily,
Hourly,
Minutely,
Never,
}
#[derive(Debug)]
pub struct Telemetry {
pub tracers: Tracers,
pub metrics: Interests,
}
#[derive(Debug)]
pub struct Tracers {
pub interests: Interests,
pub levels: AHashMap<EventType, Level>,
pub subscribers: Vec<TelemetrySubscriber>,
}
#[derive(Debug, Clone, Default)]
pub struct Metrics {
pub prometheus: Option<PrometheusMetrics>,
pub otel: Option<Arc<OtelMetrics>>,
pub log_path: Option<String>,
}
#[derive(Debug, Clone, Default)]
pub struct PrometheusMetrics {
pub auth: Option<String>,
}
impl Telemetry {
pub async fn parse(bp: &mut Bootstrap, storage: &Storage) -> Self {
let mut telemetry = Telemetry {
tracers: Tracers::parse(bp, storage).await,
metrics: Interests::default(),
};
// Parse metrics
let metrics = bp.setting_infallible::<structs::Metrics>().await;
apply_metrics(metrics.metrics, metrics.metrics_policy, |metric_type| {
let event_id = metric_type.event_id();
if event_id != usize::MAX {
telemetry.metrics.set(event_id);
}
});
telemetry
}
}
impl Tracers {
pub async fn parse(bp: &mut Bootstrap, storage: &Storage) -> Self {
let mut custom_levels = AHashMap::new();
let mut tracers: Vec<TelemetrySubscriber> = Vec::new();
let mut global_interests = Interests::default();
if !bp.registry.is_recovery_mode() {
// Parse custom logging levels
for level in bp.list_infallible::<EventTracingLevel>().await {
custom_levels.insert(level.object.event, level.object.level.into());
}
// Parse tracers
for tracer in bp.list_infallible::<Tracer>().await {
let id = tracer.id;
let tracer = tracer.object;
let level;
let lossy;
let events;
let events_policy;
let enable;
let typ = match tracer {
Tracer::Log(tracer) if tracer.enable => {
level = Level::from(tracer.level);
lossy = tracer.lossy;
events = tracer.events;
events_policy = tracer.events_policy;
enable = tracer.enable;
TelemetrySubscriberType::LogTracer(LogTracer {
path: tracer.path,
prefix: tracer.prefix,
rotate: match tracer.rotate {
LogRotateFrequency::Daily => RotationStrategy::Daily,
LogRotateFrequency::Hourly => RotationStrategy::Hourly,
LogRotateFrequency::Minutely => RotationStrategy::Minutely,
LogRotateFrequency::Never => RotationStrategy::Never,
},
ansi: tracer.ansi,
multiline: tracer.multiline,
})
}
Tracer::Stdout(tracer) if tracer.enable => {
level = Level::from(tracer.level);
lossy = tracer.lossy;
events = tracer.events;
events_policy = tracer.events_policy;
enable = tracer.enable;
if !tracers
.iter()
.any(|t| matches!(t.typ, TelemetrySubscriberType::ConsoleTracer(_)))
{
TelemetrySubscriberType::ConsoleTracer(ConsoleTracer {
ansi: tracer.ansi,
multiline: tracer.multiline,
buffered: tracer.buffered,
})
} else {
bp.build_error(id, "Only one console tracer is allowed");
continue;
}
}
Tracer::Journal(tracer) if tracer.enable => {
#[cfg(unix)]
{
level = Level::from(tracer.level);
lossy = tracer.lossy;
events = tracer.events;
events_policy = tracer.events_policy;
enable = tracer.enable;
if !tracers
.iter()
.any(|t| matches!(t.typ, TelemetrySubscriberType::JournalTracer(_)))
{
match crate::telemetry::tracers::journald::Subscriber::new() {
Ok(subscriber) => {
TelemetrySubscriberType::JournalTracer(subscriber)
}
Err(e) => {
bp.build_error(
id,
format!("Failed to create journald subscriber: {e}"),
);
continue;
}
}
} else {
bp.build_error(id, "Only one journal tracer is allowed");
continue;
}
}
#[cfg(not(unix))]
{
bp.build_error(id, "Journald is only available on Unix systems.");
continue;
}
}
Tracer::OtelHttp(tracer) if tracer.enable => {
level = Level::from(tracer.level);
lossy = tracer.lossy;
events = tracer.events;
events_policy = tracer.events_policy;
enable = tracer.enable;
let headers = match tracer
.http_auth
.build_headers(tracer.http_headers, None)
.await
{
Ok(headers) => headers
.into_iter()
.filter_map(|(k, v)| {
k.and_then(|k| {
Some((k.to_string(), v.to_str().ok()?.to_string()))
})
})
.collect::<HashMap<String, String>>(),
Err(err) => {
bp.build_error(
id,
format!("Failed to build OpenTelemetry HTTP headers: {err}"),
);
continue;
}
};
let mut span_exporter = SpanExporter::builder()
.with_http()
.with_endpoint(tracer.endpoint.clone())
.with_timeout(tracer.timeout.into_inner());
let mut log_exporter = LogExporter::builder()
.with_http()
.with_endpoint(tracer.endpoint)
.with_timeout(tracer.timeout.into_inner());
if !headers.is_empty() {
span_exporter = span_exporter.with_headers(headers.clone());
log_exporter = log_exporter.with_headers(headers);
}
match (span_exporter.build(), log_exporter.build()) {
(Ok(span_exporter), Ok(log_exporter)) => {
TelemetrySubscriberType::OtelTracer(OtelTracer {
span_exporter,
log_exporter,
throttle: tracer.throttle.into_inner(),
span_exporter_enable: tracer.enable_span_exporter,
log_exporter_enable: tracer.enable_log_exporter,
})
}
(Err(err), _) => {
bp.build_error(
id,
format!("Failed to build OpenTelemetry span exporter: {err}"),
);
continue;
}
(_, Err(err)) => {
bp.build_error(
id,
format!("Failed to build OpenTelemetry log exporter: {err}"),
);
continue;
}
}
}
Tracer::OtelGrpc(tracer) if tracer.enable => {
level = Level::from(tracer.level);
lossy = tracer.lossy;
events = tracer.events;
events_policy = tracer.events_policy;
enable = tracer.enable;
let mut span_exporter = SpanExporter::builder()
.with_tonic()
.with_protocol(opentelemetry_otlp::Protocol::Grpc)
.with_timeout(tracer.timeout.into_inner());
let mut log_exporter = LogExporter::builder()
.with_tonic()
.with_protocol(opentelemetry_otlp::Protocol::Grpc)
.with_timeout(tracer.timeout.into_inner());
if let Some(endpoint) = tracer.endpoint {
span_exporter = span_exporter.with_endpoint(endpoint.clone());
log_exporter = log_exporter.with_endpoint(endpoint);
}
match (span_exporter.build(), log_exporter.build()) {
(Ok(span_exporter), Ok(log_exporter)) => {
TelemetrySubscriberType::OtelTracer(OtelTracer {
span_exporter,
log_exporter,
throttle: tracer.throttle.into_inner(),
span_exporter_enable: tracer.enable_span_exporter,
log_exporter_enable: tracer.enable_log_exporter,
})
}
(Err(err), _) => {
bp.build_error(
id,
format!("Failed to build OpenTelemetry span exporter: {err}"),
);
continue;
}
(_, Err(err)) => {
bp.build_error(
id,
format!("Failed to build OpenTelemetry log exporter: {err}"),
);
continue;
}
}
}
_ => continue,
};
if !enable {
continue;
}
// Create tracer
let mut tracer = TelemetrySubscriber {
id: format!("t_{}", id.id()),
interests: Default::default(),
lossy,
typ,
};
// Parse disabled events
let exclude_event = match &tracer.typ {
TelemetrySubscriberType::ConsoleTracer(_) => None,
TelemetrySubscriberType::LogTracer(_) => {
EventType::Telemetry(TelemetryEvent::LogError).into()
}
TelemetrySubscriberType::OtelTracer(_) => {
EventType::Telemetry(TelemetryEvent::OtelExporterError).into()
}
TelemetrySubscriberType::Webhook(_) => {
EventType::Telemetry(TelemetryEvent::WebhookError).into()
}
#[cfg(unix)]
TelemetrySubscriberType::JournalTracer(_) => {
EventType::Telemetry(TelemetryEvent::JournalError).into()
}
};
// Parse disabled events
apply_events(events, events_policy, |event_type| {
if exclude_event != Some(event_type) {
let event_level = custom_levels
.get(&event_type)
.copied()
.unwrap_or(event_type.level());
if level.is_contained(event_level) {
tracer.interests.set(event_type);
global_interests.set(event_type);
}
}
});
if !tracer.interests.is_empty() {
tracers.push(tracer);
} else {
bp.build_warning(id, "No events enabled for tracer");
}
}
// Parse webhooks
for hook in bp.list_infallible::<WebHook>().await {
let id = hook.id;
let hook = hook.object;
if !hook.enable {
continue;
}
let headers = match hook
.http_auth
.build_headers(hook.http_headers, "application/json".into())
.await
{
Ok(headers) => headers,
Err(err) => {
bp.build_error(id, format!("Unable to build HTTP headers: {}", err));
continue;
}
};
// Build tracer
let mut tracer = TelemetrySubscriber {
id: format!("w_{}", id.id()),
interests: Default::default(),
lossy: hook.lossy,
typ: TelemetrySubscriberType::Webhook(WebhookTracer {
url: hook.url,
timeout: hook.timeout.into_inner(),
tls_allow_invalid_certs: hook.allow_invalid_certs,
client: utils::http::http_client_builder(hook.allow_invalid_certs)
.build()
.unwrap_or_default(),
headers,
key: hook
.signature_key
.secret()
.await
.map_err(|err| {
bp.build_error(
id,
format!("Unable to retrieve signature key: {}", err),
);
})
.unwrap_or_default()
.unwrap_or_default()
.into_owned(),
throttle: hook.throttle.into_inner(),
discard_after: hook.discard_after.into_inner(),
}),
};
// Parse webhook events
apply_events(hook.events, hook.events_policy, |event_type| {
if event_type != EventType::Telemetry(TelemetryEvent::WebhookError) {
tracer.interests.set(event_type);
global_interests.set(event_type);
}
});
if !tracer.interests.is_empty() {
tracers.push(tracer);
} else {
bp.build_error(id, "No events enabled for webhook");
}
}
#[cfg(feature = "dev_mode")]
if let Ok(level) = std::env::var("LOG") {
let level = Level::from_str(&level).expect("Invalid LOG level");
for event_type in EventType::variants() {
let event_level = custom_levels
.get(event_type)
.copied()
.unwrap_or(event_type.level());
if level.is_contained(event_level) {
global_interests.set(event_type.to_id() as usize);
}
}
tracers.push(TelemetrySubscriber {
id: "default".to_string(),
interests: global_interests.clone(),
typ: TelemetrySubscriberType::ConsoleTracer(ConsoleTracer {
ansi: true,
multiline: false,
buffered: true,
}),
lossy: false,
});
}
} else {
// Add default tracer if none were found
let level = std::env::var("STALWART_RECOVERY_MODE_LOG_LEVEL")
.ok()
.and_then(|level| Level::from_str(&level).ok())
.unwrap_or(Level::Info);
for event_type in EventType::variants() {
let event_level = custom_levels
.get(event_type)
.copied()
.unwrap_or(event_type.level());
if level.is_contained(event_level) {
global_interests.set(event_type.to_id() as usize);
}
}
tracers.push(TelemetrySubscriber {
id: "recover-log".to_string(),
interests: global_interests.clone(),
typ: TelemetrySubscriberType::ConsoleTracer(ConsoleTracer {
ansi: true,
multiline: false,
buffered: true,
}),
lossy: false,
});
}
Tracers {
subscribers: tracers,
interests: global_interests,
levels: custom_levels,
}
}
}
impl Metrics {
pub async fn parse(bp: &mut Bootstrap) -> Self {
let metrics = bp.setting_infallible::<structs::Metrics>().await;
let resource = Resource::builder()
.with_service_name("stalwart")
.with_attribute(KeyValue::new(SERVICE_VERSION, env!("CARGO_PKG_VERSION")))
.build();
let instrumentation = InstrumentationScope::builder("stalwart")
.with_version(env!("CARGO_PKG_VERSION"))
.build();
Metrics {
prometheus: match metrics.prometheus {
MetricsPrometheus::Enabled(prom) => {
let secret = prom
.auth_secret
.secret()
.await
.map_err(|err| {
bp.build_error(
ObjectType::Metrics.singleton(),
format!("Unable to retrieve Prometheus auth secret: {err}"),
);
})
.unwrap_or_default();
Some(PrometheusMetrics {
auth: prom.auth_username.and_then(|user| {
secret.map(|secret| STANDARD.encode(format!("{user}:{secret}")))
}),
})
}
MetricsPrometheus::Disabled => None,
},
otel: match metrics.open_telemetry {
structs::MetricsOtel::Http(otel) => {
let headers = match otel.http_auth.build_headers(otel.http_headers, None).await
{
Ok(headers) => headers
.into_iter()
.filter_map(|(k, v)| {
k.and_then(|k| Some((k.to_string(), v.to_str().ok()?.to_string())))
})
.collect::<HashMap<String, String>>(),
Err(err) => {
bp.build_error(
ObjectType::Metrics.singleton(),
format!("Failed to build OpenTelemetry HTTP headers: {err}"),
);
Default::default()
}
};
let mut exporter = MetricExporter::builder()
.with_temporality(Temporality::Delta)
.with_http()
.with_endpoint(otel.endpoint)
.with_timeout(otel.timeout.into_inner());
if !headers.is_empty() {
exporter = exporter.with_headers(headers);
}
match exporter.build() {
Ok(exporter) => Some(Arc::new(OtelMetrics {
exporter,
interval: otel.interval.into_inner(),
resource,
instrumentation,
})),
Err(err) => {
bp.build_error(
ObjectType::Metrics.singleton(),
format!("Failed to build OpenTelemetry metrics exporter: {err}"),
);
None
}
}
}
structs::MetricsOtel::Grpc(otel) => {
let mut exporter = MetricExporter::builder()
.with_temporality(Temporality::Delta)
.with_tonic()
.with_protocol(opentelemetry_otlp::Protocol::Grpc)
.with_timeout(otel.timeout.into_inner());
if let Some(endpoint) = otel.endpoint {
exporter = exporter.with_endpoint(endpoint);
}
match exporter.build() {
Ok(exporter) => Some(Arc::new(OtelMetrics {
exporter,
interval: otel.interval.into_inner(),
resource,
instrumentation,
})),
Err(err) => {
bp.build_error(
ObjectType::Metrics.singleton(),
format!("Failed to build OpenTelemetry metrics exporter: {err}"),
);
None
}
}
}
structs::MetricsOtel::Disabled => None,
},
log_path: bp
.list_infallible::<Tracer>()
.await
.into_iter()
.find_map(|tracer| {
if let Tracer::Log(log_tracer) = tracer.object
&& log_tracer.enable
{
Some(log_tracer.path)
} else {
None
}
}),
}
}
}
fn apply_events(
event_types: impl IntoIterator<Item = EventType>,
policy: EventPolicy,
mut apply_fn: impl FnMut(EventType),
) {
let mut exclude_events = AHashSet::new();
for event_type in event_types {
if policy == EventPolicy::Include {
apply_fn(event_type);
} else {
exclude_events.insert(event_type);
}
}
if policy != EventPolicy::Include {
for event_type in EventType::variants() {
if !exclude_events.contains(event_type) {
apply_fn(*event_type);
}
}
}
}
fn apply_metrics(
event_types: impl IntoIterator<Item = MetricType>,
policy: EventPolicy,
mut apply_fn: impl FnMut(MetricType),
) {
let mut exclude_events = AHashSet::new();
for event_type in event_types {
if policy == EventPolicy::Include {
apply_fn(event_type);
} else {
exclude_events.insert(event_type);
}
}
if policy != EventPolicy::Include {
for event_type in MetricType::variants() {
if !exclude_events.contains(event_type) {
apply_fn(*event_type);
}
}
}
}
impl std::fmt::Debug for OtelMetrics {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("OtelMetrics")
.field("interval", &self.interval)
.finish()
}
}
crates/common/src/expr/eval.rs
+787
View File
@@ -0,0 +1,787 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use super::{
BinaryOperator, Constant, Expression, ExpressionItem, StringCow, SystemVariable, UnaryOperator,
Variable,
functions::{FUNCTIONS, ResolveVariable},
if_block::IfBlock,
};
use crate::Server;
use compact_str::{CompactString, ToCompactString, format_compact};
use hyper::StatusCode;
use registry::{
schema::prelude::Property,
types::{EnumImpl, id::ObjectId},
};
use std::{cmp::Ordering, fmt::Display};
use trc::{Collector, EvalEvent};
impl Server {
pub async fn eval_if<'x, R: TryFrom<Variable<'x>>, V: ResolveVariable>(
&'x self,
if_block: &'x IfBlock,
resolver: &'x V,
session_id: u64,
) -> Option<R> {
if if_block.is_empty() {
trc::event!(
Eval(EvalEvent::Result),
SpanId = session_id,
Id = if_block.id.to_string(),
Key = if_block.property.as_str(),
Result = ""
);
return None;
}
match (EvalContext {
resolver,
core: self,
expr: if_block,
captures: Vec::new(),
session_id,
})
.eval()
.await
{
Ok(result) => {
trc::event!(
Eval(EvalEvent::Result),
SpanId = session_id,
Id = if_block.id.to_string(),
Key = if_block.property.as_str(),
Result = format!("{result:?}"),
);
match result.try_into() {
Ok(value) => Some(value),
Err(_) => {
trc::event!(
Eval(EvalEvent::Result),
SpanId = session_id,
Id = if_block.id.to_string(),
Key = if_block.property.as_str(),
Result = "",
);
None
}
}
}
Err(err) => {
trc::event!(
Eval(EvalEvent::Error),
SpanId = session_id,
Id = if_block.id.to_string(),
Key = if_block.property.as_str(),
CausedBy = err,
);
None
}
}
}
pub async fn eval_expr<'x, R: TryFrom<Variable<'x>>, V: ResolveVariable>(
&'x self,
expr: &'x Expression,
resolver: &'x V,
obj_id: ObjectId,
property: Property,
session_id: u64,
) -> Option<R> {
if expr.is_empty() {
return None;
}
match (EvalContext {
resolver,
core: self,
expr,
captures: &mut Vec::new(),
session_id,
})
.eval()
.await
{
Ok(result) => {
trc::event!(
Eval(EvalEvent::Result),
SpanId = session_id,
Id = obj_id.to_string(),
Key = property.as_str(),
Result = format!("{result:?}"),
);
match result.try_into() {
Ok(value) => Some(value),
Err(_) => {
trc::event!(
Eval(EvalEvent::Error),
SpanId = session_id,
Id = obj_id.to_string(),
Key = property.as_str(),
Details = "Failed to convert result",
);
None
}
}
}
Err(err) => {
trc::event!(
Eval(EvalEvent::Error),
SpanId = session_id,
Id = obj_id.to_string(),
Key = property.as_str(),
CausedBy = err,
);
None
}
}
}
}
struct EvalContext<'x, V: ResolveVariable, T, C> {
resolver: &'x V,
core: &'x Server,
expr: &'x T,
captures: C,
session_id: u64,
}
impl<'x, V: ResolveVariable> EvalContext<'x, V, IfBlock, Vec<CompactString>> {
async fn eval(&mut self) -> trc::Result<Variable<'x>> {
for if_then in &self.expr.if_then {
if (EvalContext {
resolver: self.resolver,
core: self.core,
expr: &if_then.expr,
captures: &mut self.captures,
session_id: self.session_id,
})
.eval()
.await?
.to_bool()
{
return (EvalContext {
resolver: self.resolver,
core: self.core,
expr: &if_then.then,
captures: &mut self.captures,
session_id: self.session_id,
})
.eval()
.await;
}
}
(EvalContext {
resolver: self.resolver,
core: self.core,
expr: &self.expr.default,
captures: &mut self.captures,
session_id: self.session_id,
})
.eval()
.await
}
}
impl<'x, V: ResolveVariable> EvalContext<'x, V, Expression, &mut Vec<CompactString>> {
async fn eval(&mut self) -> trc::Result<Variable<'x>> {
let mut stack = Vec::new();
let mut exprs = self.expr.items.iter();
while let Some(expr) = exprs.next() {
match expr {
ExpressionItem::Variable(v) => {
stack.push(self.resolver.resolve_variable(*v));
}
ExpressionItem::Global(v) => {
stack.push(self.resolver.resolve_global(v));
}
ExpressionItem::Constant(val) => {
stack.push(Variable::from(val));
}
ExpressionItem::Capture(v) => {
stack.push(Variable::String(StringCow::Owned(
self.captures
.get(*v as usize)
.map(|v| v.as_str())
.unwrap_or_default()
.to_compact_string(),
)));
}
ExpressionItem::System(setting) => match setting {
SystemVariable::Hostname => {
stack.push(self.core.core.network.server_name.as_str().into())
}
SystemVariable::Domain => {
stack.push(self.core.core.email.default_domain_name.as_str().into())
}
SystemVariable::NodeId => stack.push(self.core.core.network.node_id.into()),
SystemVariable::NodeHostname => {
stack.push(self.core.registry().local_hostname().into())
}
SystemVariable::NodeRole => stack.push(
self.core
.registry()
.cluster_role()
.unwrap_or_default()
.into(),
),
SystemVariable::Metric(variable) => {
stack.push(Variable::Float(Collector::read_metric(*variable)));
}
},
ExpressionItem::UnaryOperator(op) => {
let value = stack.pop().unwrap_or_default();
stack.push(match op {
UnaryOperator::Not => value.op_not(),
UnaryOperator::Minus => value.op_minus(),
});
}
ExpressionItem::BinaryOperator(op) => {
let right = stack.pop().unwrap_or_default();
let left = stack.pop().unwrap_or_default();
stack.push(match op {
BinaryOperator::Add => left.op_add(right),
BinaryOperator::Subtract => left.op_subtract(right),
BinaryOperator::Multiply => left.op_multiply(right),
BinaryOperator::Divide => left.op_divide(right),
BinaryOperator::And => left.op_and(right),
BinaryOperator::Or => left.op_or(right),
BinaryOperator::Xor => left.op_xor(right),
BinaryOperator::Eq => left.op_eq(right),
BinaryOperator::Ne => left.op_ne(right),
BinaryOperator::Lt => left.op_lt(right),
BinaryOperator::Le => left.op_le(right),
BinaryOperator::Gt => left.op_gt(right),
BinaryOperator::Ge => left.op_ge(right),
});
}
ExpressionItem::Function { id, num_args } => {
let num_args = *num_args as usize;
let mut arguments = Variable::array(num_args);
for arg_num in 0..num_args {
arguments[num_args - arg_num - 1] = stack.pop().unwrap_or_default();
}
let result = if let Some((_, fnc, _)) = FUNCTIONS.get(*id as usize) {
(fnc)(arguments)
} else {
Box::pin(self.core.eval_fnc(
*id - FUNCTIONS.len() as u32,
arguments,
self.session_id,
))
.await?
};
stack.push(result);
}
ExpressionItem::JmpIf { val, pos } => {
if stack.last().is_some_and(|v| v.to_bool()) == *val {
for _ in 0..*pos {
exprs.next();
}
}
}
ExpressionItem::ArrayAccess => {
let index = stack
.pop()
.unwrap_or_default()
.to_usize()
.unwrap_or_default();
let array = stack.pop().unwrap_or_default().into_array();
stack.push(array.into_iter().nth(index).unwrap_or_default());
}
ExpressionItem::ArrayBuild(num_items) => {
let num_items = *num_items as usize;
let mut items = Variable::array(num_items);
for arg_num in 0..num_items {
items[num_items - arg_num - 1] = stack.pop().unwrap_or_default();
}
stack.push(Variable::Array(items));
}
ExpressionItem::Regex(regex) => {
self.captures.clear();
let value = stack.pop().unwrap_or_default().into_string();
if let Some(captures_) = regex.captures(value.as_ref()) {
for capture in captures_.iter() {
self.captures
.push(capture.map_or("", |m| m.as_str()).to_compact_string());
}
}
stack.push(Variable::Integer(!self.captures.is_empty() as i64));
}
}
}
Ok(stack.pop().unwrap_or_default())
}
}
impl Expression {
pub fn is_empty(&self) -> bool {
self.items.is_empty()
}
pub fn items(&self) -> &[ExpressionItem] {
&self.items
}
}
impl<'x> Variable<'x> {
pub fn op_add(self, other: Variable<'x>) -> Variable<'x> {
match (self, other) {
(Variable::Integer(a), Variable::Integer(b)) => Variable::Integer(a.saturating_add(b)),
(Variable::Float(a), Variable::Float(b)) => Variable::Float(a + b),
(Variable::Integer(i), Variable::Float(f))
| (Variable::Float(f), Variable::Integer(i)) => Variable::Float(i as f64 + f),
(Variable::Array(a), Variable::Array(b)) => {
Variable::Array(a.into_iter().chain(b).collect::<Vec<_>>())
}
(Variable::Array(a), b) => {
Variable::Array(a.into_iter().chain([b]).collect::<Vec<_>>())
}
(a, Variable::Array(b)) => {
Variable::Array([a].into_iter().chain(b).collect::<Vec<_>>())
}
(Variable::String(a), b) => {
if !a.is_empty() {
Variable::String(StringCow::Owned(format_compact!("{}{}", a, b)))
} else {
b
}
}
(a, Variable::String(b)) => {
if !b.is_empty() {
Variable::String(StringCow::Owned(format_compact!("{}{}", a, b)))
} else {
a
}
}
(a, Variable::Constant(_)) => a,
(Variable::Constant(_), b) => b,
}
}
pub fn op_subtract(self, other: Variable<'x>) -> Variable<'x> {
match (self, other) {
(Variable::Integer(a), Variable::Integer(b)) => Variable::Integer(a.saturating_sub(b)),
(Variable::Float(a), Variable::Float(b)) => Variable::Float(a - b),
(Variable::Integer(a), Variable::Float(b)) => Variable::Float(a as f64 - b),
(Variable::Float(a), Variable::Integer(b)) => Variable::Float(a - b as f64),
(Variable::Array(a), b) | (b, Variable::Array(a)) => {
Variable::Array(a.into_iter().filter(|v| v != &b).collect::<Vec<_>>())
}
(a, b) => a.parse_number().op_subtract(b.parse_number()),
}
}
pub fn op_multiply(self, other: Variable<'x>) -> Variable<'x> {
match (self, other) {
(Variable::Integer(a), Variable::Integer(b)) => Variable::Integer(a.saturating_mul(b)),
(Variable::Float(a), Variable::Float(b)) => Variable::Float(a * b),
(Variable::Integer(i), Variable::Float(f))
| (Variable::Float(f), Variable::Integer(i)) => Variable::Float(i as f64 * f),
(a, b) => a.parse_number().op_multiply(b.parse_number()),
}
}
pub fn op_divide(self, other: Variable<'x>) -> Variable<'x> {
match (self, other) {
(Variable::Integer(a), Variable::Integer(b)) => {
Variable::Float(if b != 0 { a as f64 / b as f64 } else { 0.0 })
}
(Variable::Float(a), Variable::Float(b)) => {
Variable::Float(if b != 0.0 { a / b } else { 0.0 })
}
(Variable::Integer(a), Variable::Float(b)) => {
Variable::Float(if b != 0.0 { a as f64 / b } else { 0.0 })
}
(Variable::Float(a), Variable::Integer(b)) => {
Variable::Float(if b != 0 { a / b as f64 } else { 0.0 })
}
(a, b) => a.parse_number().op_divide(b.parse_number()),
}
}
pub fn op_and(self, other: Variable) -> Variable {
Variable::Integer(i64::from(self.to_bool() & other.to_bool()))
}
pub fn op_or(self, other: Variable) -> Variable {
Variable::Integer(i64::from(self.to_bool() | other.to_bool()))
}
pub fn op_xor(self, other: Variable) -> Variable {
Variable::Integer(i64::from(self.to_bool() ^ other.to_bool()))
}
pub fn op_eq(self, other: Variable) -> Variable {
Variable::Integer(i64::from(self == other))
}
pub fn op_ne(self, other: Variable) -> Variable {
Variable::Integer(i64::from(self != other))
}
pub fn op_lt(self, other: Variable) -> Variable {
Variable::Integer(i64::from(self < other))
}
pub fn op_le(self, other: Variable) -> Variable {
Variable::Integer(i64::from(self <= other))
}
pub fn op_gt(self, other: Variable) -> Variable {
Variable::Integer(i64::from(self > other))
}
pub fn op_ge(self, other: Variable) -> Variable {
Variable::Integer(i64::from(self >= other))
}
pub fn op_not(self) -> Variable<'static> {
Variable::Integer(i64::from(!self.to_bool()))
}
pub fn op_minus(self) -> Variable<'static> {
match self {
Variable::Integer(n) => Variable::Integer(-n),
Variable::Float(n) => Variable::Float(-n),
_ => self.parse_number().op_minus(),
}
}
pub fn parse_number(&self) -> Variable<'static> {
match self {
Variable::String(s) if !s.is_empty() => {
if let Ok(n) = s.as_str().parse::<i64>() {
Variable::Integer(n)
} else if let Ok(n) = s.as_str().parse::<f64>() {
Variable::Float(n)
} else {
Variable::Integer(0)
}
}
Variable::Integer(n) => Variable::Integer(*n),
Variable::Float(n) => Variable::Float(*n),
Variable::Array(l) => Variable::Integer(l.is_empty() as i64),
_ => Variable::Integer(0),
}
}
#[inline(always)]
fn array(num_items: usize) -> Vec<Variable<'static>> {
let mut items = Vec::with_capacity(num_items);
for _ in 0..num_items {
items.push(Variable::Integer(0));
}
items
}
pub fn to_ref<'y: 'x>(&'y self) -> Variable<'x> {
match self {
Variable::String(s) => Variable::String(StringCow::Borrowed(s.as_str())),
Variable::Integer(n) => Variable::Integer(*n),
Variable::Float(n) => Variable::Float(*n),
Variable::Constant(c) => Variable::Constant(*c),
Variable::Array(l) => Variable::Array(l.iter().map(|v| v.to_ref()).collect::<Vec<_>>()),
}
}
pub fn to_bool(&self) -> bool {
match self {
Variable::Float(f) => *f != 0.0,
Variable::Integer(n) => *n != 0,
Variable::String(s) => !s.is_empty(),
Variable::Array(a) => !a.is_empty(),
Variable::Constant(_) => true,
}
}
pub fn to_string(&'_ self) -> StringCow<'_> {
match self {
Variable::String(s) => StringCow::Borrowed(s.as_str()),
Variable::Integer(n) => StringCow::Owned(n.to_compact_string()),
Variable::Float(n) => StringCow::Owned(n.to_compact_string()),
Variable::Array(l) => {
let mut result = CompactString::with_capacity(self.len() * 10);
for item in l {
if !result.is_empty() {
result.push_str("\r\n");
}
match item {
Variable::String(v) => result.push_str(v.as_str()),
Variable::Integer(v) => result.push_str(&v.to_compact_string()),
Variable::Float(v) => result.push_str(&v.to_compact_string()),
Variable::Array(_) => {}
Variable::Constant(c) => result.push_str(c.as_str()),
}
}
StringCow::Owned(result)
}
Variable::Constant(c) => StringCow::Borrowed(c.as_str()),
}
}
pub fn into_string(self) -> StringCow<'x> {
match self {
Variable::String(s) => s,
Variable::Integer(n) => StringCow::Owned(n.to_compact_string()),
Variable::Float(n) => StringCow::Owned(n.to_compact_string()),
Variable::Array(l) => {
let mut result = CompactString::with_capacity(l.len() * 10);
for item in l {
if !result.is_empty() {
result.push_str("\r\n");
}
match item {
Variable::String(v) => result.push_str(v.as_ref()),
Variable::Integer(v) => result.push_str(&v.to_compact_string()),
Variable::Float(v) => result.push_str(&v.to_compact_string()),
Variable::Array(_) => {}
Variable::Constant(c) => result.push_str(c.as_str()),
}
}
StringCow::Owned(result)
}
Variable::Constant(c) => StringCow::Borrowed(c.as_str()),
}
}
pub fn to_integer(&self) -> Option<i64> {
match self {
Variable::Integer(n) => Some(*n),
Variable::Float(n) => Some(*n as i64),
Variable::String(s) if !s.is_empty() => s.as_str().parse::<i64>().ok(),
_ => None,
}
}
pub fn to_usize(&self) -> Option<usize> {
match self {
Variable::Integer(n) => Some(*n as usize),
Variable::Float(n) => Some(*n as usize),
Variable::String(s) if !s.is_empty() => s.as_str().parse::<usize>().ok(),
_ => None,
}
}
pub fn len(&self) -> usize {
match self {
Variable::String(s) => s.len(),
Variable::Integer(_) | Variable::Float(_) => 2,
Variable::Array(l) => l.iter().map(|v| v.len() + 2).sum(),
Variable::Constant(c) => c.as_str().len(),
}
}
pub fn is_empty(&self) -> bool {
match self {
Variable::String(s) => s.is_empty(),
_ => false,
}
}
pub fn as_array(&'_ self) -> Option<&'_ [Variable<'_>]> {
match self {
Variable::Array(l) => Some(l),
_ => None,
}
}
pub fn into_array(self) -> Vec<Variable<'x>> {
match self {
Variable::Array(l) => l,
v if !v.is_empty() => vec![v],
_ => vec![],
}
}
pub fn to_array(&self) -> Vec<Variable<'_>> {
match self {
Variable::Array(l) => l.iter().map(|v| v.to_ref()).collect::<Vec<_>>(),
v if !v.is_empty() => vec![v.to_ref()],
_ => vec![],
}
}
pub fn into_owned(self) -> Variable<'static> {
match self {
Variable::String(s) => Variable::String(StringCow::Owned(s.into_owned())),
Variable::Integer(n) => Variable::Integer(n),
Variable::Float(n) => Variable::Float(n),
Variable::Constant(c) => Variable::Constant(c),
Variable::Array(l) => Variable::Array(l.into_iter().map(|v| v.into_owned()).collect()),
}
}
}
impl PartialEq for Variable<'_> {
fn eq(&self, other: &Self) -> bool {
match (self, other) {
(Self::Integer(a), Self::Integer(b)) => a == b,
(Self::Float(a), Self::Float(b)) => a == b,
(Self::Integer(a), Self::Float(b)) | (Self::Float(b), Self::Integer(a)) => {
*a as f64 == *b
}
(Self::String(a), Self::String(b)) => a.as_str() == b.as_str(),
(Self::String(_), Self::Integer(_) | Self::Float(_)) => &self.parse_number() == other,
(Self::Integer(_) | Self::Float(_), Self::String(_)) => self == &other.parse_number(),
(Self::Array(a), Self::Array(b)) => a == b,
_ => false,
}
}
}
impl Eq for Variable<'_> {}
#[allow(clippy::non_canonical_partial_ord_impl)]
impl PartialOrd for Variable<'_> {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
match (self, other) {
(Self::Integer(a), Self::Integer(b)) => a.partial_cmp(b),
(Self::Float(a), Self::Float(b)) => a.partial_cmp(b),
(Self::Integer(a), Self::Float(b)) => (*a as f64).partial_cmp(b),
(Self::Float(a), Self::Integer(b)) => a.partial_cmp(&(*b as f64)),
(Self::String(a), Self::String(b)) => a.as_str().partial_cmp(b.as_str()),
(Self::String(_), Self::Integer(_) | Self::Float(_)) => {
self.parse_number().partial_cmp(other)
}
(Self::Integer(_) | Self::Float(_), Self::String(_)) => {
self.partial_cmp(&other.parse_number())
}
(Self::Array(a), Self::Array(b)) => a.partial_cmp(b),
(Self::Array(_) | Self::String(_), _) => Ordering::Greater.into(),
(Self::Constant(a), Self::Constant(b)) => a.to_id().partial_cmp(&b.to_id()),
(_, Self::Array(_) | Self::Constant(_)) | (Self::Constant(_), _) => {
Ordering::Less.into()
}
}
}
}
impl Ord for Variable<'_> {
fn cmp(&self, other: &Self) -> std::cmp::Ordering {
self.partial_cmp(other).unwrap_or(Ordering::Greater)
}
}
impl Display for Variable<'_> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Variable::String(v) => v.fmt(f),
Variable::Integer(v) => v.fmt(f),
Variable::Float(v) => v.fmt(f),
Variable::Array(v) => {
for (i, v) in v.iter().enumerate() {
if i > 0 {
f.write_str("\n")?;
}
v.fmt(f)?;
}
Ok(())
}
Variable::Constant(c) => c.as_str().fmt(f),
}
}
}
impl<'x> From<&'x Constant> for Variable<'x> {
fn from(value: &'x Constant) -> Self {
match value {
Constant::Integer(i) => Variable::Integer(*i),
Constant::Float(f) => Variable::Float(*f),
Constant::String(s) => Variable::String(StringCow::Borrowed(s.as_str())),
Constant::Static(c) => Variable::Constant(*c),
}
}
}
impl<'x> TryFrom<Variable<'x>> for CompactString {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
if let Variable::String(s) = value {
Ok(match s {
StringCow::Borrowed(v) => v.into(),
StringCow::Owned(v) => v,
})
} else {
Err(())
}
}
}
impl<'x> TryFrom<Variable<'x>> for String {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
if let Variable::String(s) = value {
Ok(match s {
StringCow::Borrowed(v) => v.to_string(),
StringCow::Owned(v) => v.into_string(),
})
} else {
Err(())
}
}
}
impl<'x> From<Variable<'x>> for bool {
fn from(val: Variable<'x>) -> Self {
val.to_bool()
}
}
impl<'x> TryFrom<Variable<'x>> for i64 {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
value.to_integer().ok_or(())
}
}
impl<'x> TryFrom<Variable<'x>> for u64 {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
value.to_integer().map(|v| v as u64).ok_or(())
}
}
impl<'x> TryFrom<Variable<'x>> for usize {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
value.to_usize().ok_or(())
}
}
impl<'x> TryFrom<Variable<'x>> for StatusCode {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
match value.to_integer() {
Some(v) => match StatusCode::from_u16(v as u16) {
Ok(status) => Ok(status),
Err(_) => Err(()),
},
None => Err(()),
}
}
}
crates/common/src/expr/functions/array.rs
+65
View File
@@ -0,0 +1,65 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::expr::Variable;
pub(crate) fn fn_count(v: Vec<Variable>) -> Variable {
match &v[0] {
Variable::Array(a) => a.len(),
v => {
if !v.is_empty() {
1
} else {
0
}
}
}
.into()
}
pub(crate) fn fn_sort(mut v: Vec<Variable>) -> Variable {
let is_asc = v[1].to_bool();
let mut arr = v.remove(0).into_array();
if is_asc {
arr.sort_unstable();
} else {
arr.sort_unstable_by(|a, b| b.cmp(a));
}
arr.into()
}
pub(crate) fn fn_dedup(mut v: Vec<Variable>) -> Variable {
let arr = v.remove(0).into_array();
let mut result = Vec::with_capacity(arr.len());
for item in arr {
if !result.contains(&item) {
result.push(item);
}
}
result.into()
}
pub(crate) fn fn_is_intersect(v: Vec<Variable>) -> Variable {
match (&v[0], &v[1]) {
(Variable::Array(a), Variable::Array(b)) => a.iter().any(|x| b.contains(x)),
(Variable::Array(a), item) | (item, Variable::Array(a)) => a.contains(item),
_ => false,
}
.into()
}
pub(crate) fn fn_winnow(mut v: Vec<Variable>) -> Variable {
match v.remove(0) {
Variable::Array(a) => a
.into_iter()
.filter(|i| !i.is_empty())
.collect::<Vec<_>>()
.into(),
v => v,
}
}
crates/common/src/expr/functions/asynch.rs
+385
View File
@@ -0,0 +1,385 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use super::*;
use crate::{Server, expr::StringCow};
use compact_str::{CompactString, ToCompactString};
use mail_auth::IpLookupStrategy;
use std::{cmp::Ordering, net::IpAddr, vec::IntoIter};
use store::{Deserialize, Rows, Value, dispatch::lookup::KeyValue};
use trc::AddContext;
impl Server {
pub(crate) async fn eval_fnc<'x>(
&self,
fnc_id: u32,
params: Vec<Variable<'x>>,
session_id: u64,
) -> trc::Result<Variable<'x>> {
let mut params = FncParams::new(params);
match fnc_id {
F_IS_LOCAL_DOMAIN => {
let domain = params.next_as_string();
self.domain(domain.as_str())
.await
.caused_by(trc::location!())
.map(|v| v.is_some().into())
}
F_IS_LOCAL_ADDRESS => {
let address = params.next_as_string();
self.rcpt_id_from_email(address.as_ref())
.await
.caused_by(trc::location!())
.map(|v| v.is_some().into())
}
F_KEY_GET => {
let Some(store) = self.get_lookup_store(params.next_as_string().as_str()) else {
return Ok(Variable::default());
};
let key = params.next_as_string();
store
.key_get::<VariableWrapper>(key.as_str())
.await
.map(|value| value.map(|v| v.into_inner()).unwrap_or_default())
.caused_by(trc::location!())
}
F_KEY_EXISTS => {
let Some(store) = self.get_lookup_store(params.next_as_string().as_str()) else {
return Ok(Variable::default());
};
let key = params.next_as_string();
store
.key_exists(key.as_str())
.await
.caused_by(trc::location!())
.map(|v| v.into())
}
F_KEY_SET => {
let Some(store) = self.get_lookup_store(params.next_as_string().as_str()) else {
return Ok(Variable::default());
};
let key = params.next_as_string();
let value = params.next_as_string();
store
.key_set(KeyValue::new(
key.as_bytes().to_vec(),
value.as_bytes().to_vec(),
))
.await
.map(|_| true)
.caused_by(trc::location!())
.map(|v| v.into())
}
F_COUNTER_INCR => {
let Some(store) = self.get_lookup_store(params.next_as_string().as_str()) else {
return Ok(Variable::default());
};
let key = params.next_as_string();
let value = params.next_as_integer();
store
.counter_incr(KeyValue::new(key.into_owned(), value), true)
.await
.map(Variable::Integer)
.caused_by(trc::location!())
}
F_COUNTER_GET => {
let Some(store) = self.get_lookup_store(params.next_as_string().as_str()) else {
return Ok(Variable::default());
};
let key = params.next_as_string();
store
.counter_get(key.as_bytes().to_vec())
.await
.map(Variable::Integer)
.caused_by(trc::location!())
}
F_DNS_QUERY => self.dns_query(params).await,
F_SQL_QUERY => self.sql_query(params, session_id).await,
_ => Ok(Variable::default()),
}
}
async fn sql_query<'x>(
&self,
mut arguments: FncParams<'x>,
session_id: u64,
) -> trc::Result<Variable<'x>> {
let store_name = arguments.next_as_string();
let Some(store) = self
.get_lookup_store(store_name.as_ref())
.and_then(|v| v.into_store())
else {
return Err(trc::EventType::Eval(trc::EvalEvent::Error)
.into_err()
.id(store_name.into_owned())
.span_id(session_id)
.details("Store not found or is not a SQL store"));
};
let query = arguments.next_as_string();
if query.is_empty() {
return Err(trc::EventType::Eval(trc::EvalEvent::Error)
.into_err()
.details("Empty query string")
.span_id(session_id));
}
// Obtain arguments
let arguments = match arguments.next() {
Variable::Array(l) => l.into_iter().map(to_store_value).collect(),
v => vec![to_store_value(v)],
};
// Run query
if query
.as_bytes()
.get(..6)
.is_some_and(|q| q.eq_ignore_ascii_case(b"SELECT"))
{
let mut rows = store
.sql_query::<Rows>(query.as_str(), arguments)
.await
.caused_by(trc::location!())?;
Ok(match rows.rows.len().cmp(&1) {
Ordering::Equal => {
let mut row = rows.rows.pop().unwrap().values;
match row.len().cmp(&1) {
Ordering::Equal if !matches!(row.first(), Some(Value::Null)) => {
row.pop().map(into_variable).unwrap()
}
Ordering::Less => Variable::default(),
_ => {
Variable::Array(row.into_iter().map(into_variable).collect::<Vec<_>>())
}
}
}
Ordering::Less => Variable::default(),
Ordering::Greater => rows
.rows
.into_iter()
.map(|r| {
Variable::Array(r.values.into_iter().map(into_variable).collect::<Vec<_>>())
})
.collect::<Vec<_>>()
.into(),
})
} else {
store
.sql_query::<usize>(query.as_str(), arguments)
.await
.caused_by(trc::location!())
.map(|v| v.into())
}
}
async fn dns_query<'x>(&self, mut arguments: FncParams<'x>) -> trc::Result<Variable<'x>> {
let entry = arguments.next_as_string();
let record_type = arguments.next_as_string();
if record_type.as_str().eq_ignore_ascii_case("ip") {
self.core
.smtp
.resolvers
.dns
.ip_lookup(
entry.as_ref(),
IpLookupStrategy::Ipv4thenIpv6,
10,
Some(&self.inner.cache.dns_ipv4),
Some(&self.inner.cache.dns_ipv6),
)
.await
.map_err(|err| trc::Error::from(err).caused_by(trc::location!()))
.map(|result| {
result
.iter()
.map(|ip| Variable::from(ip.to_compact_string()))
.collect::<Vec<_>>()
.into()
})
} else if record_type.as_str().eq_ignore_ascii_case("mx") {
self.core
.smtp
.resolvers
.dns
.mx_lookup(entry.as_str(), Some(&self.inner.cache.dns_mx))
.await
.map_err(|err| trc::Error::from(err).caused_by(trc::location!()))
.map(|result| {
result
.rrset
.iter()
.flat_map(|mx| {
mx.exchanges.iter().map(|host| {
Variable::String(StringCow::Owned(
host.strip_suffix('.').unwrap_or(host).to_compact_string(),
))
})
})
.collect::<Vec<_>>()
.into()
})
} else if record_type.as_str().eq_ignore_ascii_case("txt") {
self.core
.smtp
.resolvers
.dns
.txt_raw_lookup(entry.as_str())
.await
.map_err(|err| trc::Error::from(err).caused_by(trc::location!()))
.map(|result| Variable::from(CompactString::from_utf8(result).unwrap_or_default()))
} else if record_type.as_str().eq_ignore_ascii_case("ptr") {
self.core
.smtp
.resolvers
.dns
.ptr_lookup(
entry.as_str().parse::<IpAddr>().map_err(|err| {
trc::EventType::Eval(trc::EvalEvent::Error)
.into_err()
.details("Failed to parse IP address")
.reason(err)
})?,
Some(&self.inner.cache.dns_ptr),
)
.await
.map_err(|err| trc::Error::from(err).caused_by(trc::location!()))
.map(|result| {
result
.rrset
.iter()
.map(|host| Variable::from(host.to_compact_string()))
.collect::<Vec<_>>()
.into()
})
} else if record_type.as_str().eq_ignore_ascii_case("ipv4") {
self.core
.smtp
.resolvers
.dns
.ipv4_lookup(entry.as_str(), Some(&self.inner.cache.dns_ipv4))
.await
.map_err(|err| trc::Error::from(err).caused_by(trc::location!()))
.map(|result| {
result
.rrset
.iter()
.map(|ip| Variable::from(ip.to_compact_string()))
.collect::<Vec<_>>()
.into()
})
} else if record_type.as_str().eq_ignore_ascii_case("ipv6") {
self.core
.smtp
.resolvers
.dns
.ipv6_lookup(entry.as_str(), Some(&self.inner.cache.dns_ipv6))
.await
.map_err(|err| trc::Error::from(err).caused_by(trc::location!()))
.map(|result| {
result
.rrset
.iter()
.map(|ip| Variable::from(ip.to_compact_string()))
.collect::<Vec<_>>()
.into()
})
} else {
Ok(Variable::default())
}
}
}
struct FncParams<'x> {
params: IntoIter<Variable<'x>>,
}
impl<'x> FncParams<'x> {
pub fn new(params: Vec<Variable<'x>>) -> Self {
Self {
params: params.into_iter(),
}
}
pub fn next_as_string(&mut self) -> StringCow<'x> {
self.params.next().unwrap().into_string()
}
pub fn next_as_integer(&mut self) -> i64 {
self.params.next().unwrap().to_integer().unwrap_or_default()
}
pub fn next(&mut self) -> Variable<'x> {
self.params.next().unwrap()
}
}
#[derive(Debug)]
struct VariableWrapper(Variable<'static>);
impl From<i64> for VariableWrapper {
fn from(value: i64) -> Self {
VariableWrapper(Variable::Integer(value))
}
}
impl Deserialize for VariableWrapper {
fn deserialize(bytes: &[u8]) -> trc::Result<Self> {
Ok(VariableWrapper(Variable::String(StringCow::Owned(
CompactString::from_utf8_lossy(bytes),
))))
}
}
impl From<store::Value<'static>> for VariableWrapper {
fn from(value: store::Value<'static>) -> Self {
VariableWrapper(match value {
Value::Integer(v) => Variable::Integer(v),
Value::Bool(v) => Variable::Integer(v as i64),
Value::Float(v) => Variable::Float(v),
Value::Text(v) => Variable::String(StringCow::Owned(v.into())),
Value::Blob(v) => Variable::String(StringCow::Owned(match v {
std::borrow::Cow::Borrowed(v) => CompactString::from_utf8_lossy(v),
std::borrow::Cow::Owned(v) => CompactString::from_utf8_lossy(&v),
})),
Value::Null => Variable::String(StringCow::Borrowed("")),
})
}
}
impl VariableWrapper {
pub fn into_inner(self) -> Variable<'static> {
self.0
}
}
fn to_store_value(value: Variable) -> Value {
match value {
Variable::String(v) => Value::Text(v.to_string().into()),
Variable::Integer(v) => Value::Integer(v),
Variable::Float(v) => Value::Float(v),
v => Value::Text(v.to_string().into_owned().into()),
}
}
fn into_variable(value: Value) -> Variable {
match value {
Value::Integer(v) => Variable::Integer(v),
Value::Bool(v) => Variable::Integer(i64::from(v)),
Value::Float(v) => Variable::Float(v),
Value::Text(v) => Variable::String(v.into()),
Value::Blob(v) => Variable::String(StringCow::Owned(CompactString::from_utf8_lossy(&v))),
Value::Null => Variable::default(),
}
}
crates/common/src/expr/functions/email.rs
+104
View File
@@ -0,0 +1,104 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use compact_str::CompactString;
use crate::expr::{StringCow, Variable};
pub(crate) fn fn_is_email(v: Vec<Variable>) -> Variable {
let mut last_ch = 0;
let mut in_quote = false;
let mut at_count = 0;
let mut dot_count = 0;
let mut lp_len = 0;
let mut value = 0;
for &ch in v[0].to_string().as_bytes() {
match ch {
b'0'..=b'9'
| b'a'..=b'z'
| b'A'..=b'Z'
| b'!'
| b'#'
| b'$'
| b'%'
| b'&'
| b'\''
| b'*'
| b'+'
| b'-'
| b'/'
| b'='
| b'?'
| b'^'
| b'_'
| b'`'
| b'{'
| b'|'
| b'}'
| b'~'
| 0x7f..=u8::MAX => {
value += 1;
}
b'.' if !in_quote => {
if last_ch != b'.' && last_ch != b'@' && value != 0 {
value += 1;
if at_count == 1 {
dot_count += 1;
}
} else {
return false.into();
}
}
b'@' if !in_quote => {
at_count += 1;
lp_len = value;
value = 0;
}
b'>' | b':' | b',' | b' ' if in_quote => {
value += 1;
}
b'\"' if !in_quote || last_ch != b'\\' => {
in_quote = !in_quote;
}
b'\\' if in_quote && last_ch != b'\\' => (),
_ => {
if !in_quote {
return false.into();
}
}
}
last_ch = ch;
}
(at_count == 1 && dot_count > 0 && lp_len > 0 && value > 0).into()
}
pub(crate) fn fn_email_part(v: Vec<Variable>) -> Variable {
let mut v = v.into_iter();
let value = v.next().unwrap();
let part = v.next().unwrap().into_string();
value.transform(|s| match s {
StringCow::Borrowed(s) => s
.rsplit_once('@')
.map(|(u, d)| match part.as_str() {
"local" => Variable::from(u.trim()),
"domain" => Variable::from(d.trim()),
_ => Variable::default(),
})
.unwrap_or_default(),
StringCow::Owned(s) => s
.rsplit_once('@')
.map(|(u, d)| match part.as_str() {
"local" => Variable::from(CompactString::new(u.trim())),
"domain" => Variable::from(CompactString::new(d.trim())),
_ => Variable::default(),
})
.unwrap_or_default(),
})
}
crates/common/src/expr/functions/misc.rs
+78
View File
@@ -0,0 +1,78 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::expr::Variable;
use compact_str::CompactString;
use mail_auth::common::resolver::ToReverseName;
use registry::types::ipmask::IpAddrOrMask;
use std::{net::IpAddr, str::FromStr};
pub(crate) fn fn_is_empty(v: Vec<Variable>) -> Variable {
match &v[0] {
Variable::String(s) => s.is_empty(),
Variable::Integer(_) | Variable::Float(_) | Variable::Constant(_) => false,
Variable::Array(a) => a.is_empty(),
}
.into()
}
pub(crate) fn fn_is_number(v: Vec<Variable>) -> Variable {
matches!(&v[0], Variable::Integer(_) | Variable::Float(_)).into()
}
pub(crate) fn fn_is_ip_addr(v: Vec<Variable>) -> Variable {
v[0].to_string()
.as_str()
.parse::<std::net::IpAddr>()
.is_ok()
.into()
}
pub(crate) fn fn_is_ipv4_addr(v: Vec<Variable>) -> Variable {
v[0].to_string()
.as_str()
.parse::<std::net::IpAddr>()
.is_ok_and(|ip| matches!(ip, IpAddr::V4(_)))
.into()
}
pub(crate) fn fn_is_ipv6_addr(v: Vec<Variable>) -> Variable {
v[0].to_string()
.as_str()
.parse::<std::net::IpAddr>()
.is_ok_and(|ip| matches!(ip, IpAddr::V6(_)))
.into()
}
pub(crate) fn fn_is_ip_in_cidr(v: Vec<Variable>) -> Variable {
let Ok(ip) = v[0].to_string().as_str().parse::<IpAddr>() else {
return false.into();
};
IpAddrOrMask::from_str(v[1].to_string().as_str())
.map(|mask| mask.matches(&ip))
.unwrap_or(false)
.into()
}
pub(crate) fn fn_ip_reverse_name(v: Vec<Variable>) -> Variable {
CompactString::new(
v[0].to_string()
.as_str()
.parse::<std::net::IpAddr>()
.map(|ip| ip.to_reverse_name())
.unwrap_or_default(),
)
.into()
}
pub(crate) fn fn_if_then(v: Vec<Variable>) -> Variable {
let mut v = v.into_iter();
let condition = v.next().unwrap();
let iff = v.next().unwrap();
let then = v.next().unwrap();
if condition.to_bool() { iff } else { then }
}
crates/common/src/expr/functions/mod.rs
+118
View File
@@ -0,0 +1,118 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use super::{StringCow, Variable};
use registry::schema::enums::ExpressionVariable;
pub mod array;
pub mod asynch;
pub mod email;
pub mod misc;
pub mod text;
pub trait ResolveVariable: Sync + Send {
fn resolve_variable(&self, variable: ExpressionVariable) -> Variable<'_>;
fn resolve_global(&self, variable: &str) -> Variable<'_>;
}
impl<'x> Variable<'x> {
fn transform(self, f: impl Fn(StringCow<'x>) -> Variable<'x>) -> Variable<'x> {
match self {
Variable::String(s) => f(s),
Variable::Array(list) => Variable::Array(
list.into_iter()
.map(|v| match v {
Variable::String(s) => f(s),
v => f(v.into_string()),
})
.collect::<Vec<_>>(),
),
v => f(v.into_string()),
}
}
}
#[allow(clippy::type_complexity)]
pub(crate) const FUNCTIONS: &[(&str, fn(Vec<Variable>) -> Variable, u32)] = &[
("count", array::fn_count, 1),
("sort", array::fn_sort, 2),
("dedup", array::fn_dedup, 1),
("winnow", array::fn_winnow, 1),
("is_intersect", array::fn_is_intersect, 2),
("is_email", email::fn_is_email, 1),
("email_part", email::fn_email_part, 2),
("is_empty", misc::fn_is_empty, 1),
("is_number", misc::fn_is_number, 1),
("is_ip_addr", misc::fn_is_ip_addr, 1),
("is_ipv4_addr", misc::fn_is_ipv4_addr, 1),
("is_ipv6_addr", misc::fn_is_ipv6_addr, 1),
("is_ip_in_cidr", misc::fn_is_ip_in_cidr, 2),
("ip_reverse_name", misc::fn_ip_reverse_name, 1),
("trim", text::fn_trim, 1),
("trim_end", text::fn_trim_end, 1),
("trim_start", text::fn_trim_start, 1),
("len", text::fn_len, 1),
("to_lowercase", text::fn_to_lowercase, 1),
("to_uppercase", text::fn_to_uppercase, 1),
("is_uppercase", text::fn_is_uppercase, 1),
("is_lowercase", text::fn_is_lowercase, 1),
("has_digits", text::fn_has_digits, 1),
("count_spaces", text::fn_count_spaces, 1),
("count_uppercase", text::fn_count_uppercase, 1),
("count_lowercase", text::fn_count_lowercase, 1),
("count_chars", text::fn_count_chars, 1),
("contains", text::fn_contains, 2),
("contains_ignore_case", text::fn_contains_ignore_case, 2),
("eq_ignore_case", text::fn_eq_ignore_case, 2),
("starts_with", text::fn_starts_with, 2),
("ends_with", text::fn_ends_with, 2),
("lines", text::fn_lines, 1),
("substring", text::fn_substring, 3),
("strip_prefix", text::fn_strip_prefix, 2),
("strip_suffix", text::fn_strip_suffix, 2),
("split", text::fn_split, 2),
("rsplit", text::fn_rsplit, 2),
("split_once", text::fn_split_once, 2),
("rsplit_once", text::fn_rsplit_once, 2),
("split_n", text::fn_split_n, 3),
("split_words", text::fn_split_words, 1),
("hash", text::fn_hash, 2),
("if_then", misc::fn_if_then, 3),
];
pub const F_IS_LOCAL_DOMAIN: u32 = 0;
pub const F_IS_LOCAL_ADDRESS: u32 = 1;
pub const F_KEY_GET: u32 = 2;
pub const F_KEY_EXISTS: u32 = 3;
pub const F_KEY_SET: u32 = 4;
pub const F_COUNTER_INCR: u32 = 5;
pub const F_COUNTER_GET: u32 = 6;
pub const F_SQL_QUERY: u32 = 7;
pub const F_DNS_QUERY: u32 = 8;
pub const ASYNC_FUNCTIONS: &[(&str, u32, u32)] = &[
("is_local_domain", F_IS_LOCAL_DOMAIN, 1),
("is_local_address", F_IS_LOCAL_ADDRESS, 1),
("key_get", F_KEY_GET, 2),
("key_exists", F_KEY_EXISTS, 2),
("key_set", F_KEY_SET, 3),
("counter_incr", F_COUNTER_INCR, 3),
("counter_get", F_COUNTER_GET, 2),
("dns_query", F_DNS_QUERY, 2),
("sql_query", F_SQL_QUERY, 3),
];
pub struct EmptyResolver;
impl ResolveVariable for EmptyResolver {
fn resolve_variable(&self, _: ExpressionVariable) -> Variable<'_> {
Variable::Integer(0)
}
fn resolve_global(&self, _: &str) -> Variable<'_> {
Variable::Integer(0)
}
}
crates/common/src/expr/functions/text.rs
+359
View File
@@ -0,0 +1,359 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use compact_str::{CompactString, ToCompactString, format_compact};
use sha1::Sha1;
use sha2::{Sha256, Sha512};
use utils::HexEncode;
use crate::expr::{StringCow, Variable};
pub(crate) fn fn_trim(mut v: Vec<Variable>) -> Variable {
v.remove(0).transform(|s| match s {
StringCow::Borrowed(s) => Variable::from(s.trim()),
StringCow::Owned(s) => Variable::from(s.trim().to_compact_string()),
})
}
pub(crate) fn fn_trim_end(mut v: Vec<Variable>) -> Variable {
v.remove(0).transform(|s| match s {
StringCow::Borrowed(s) => Variable::from(s.trim_end()),
StringCow::Owned(s) => Variable::from(s.trim_end().to_compact_string()),
})
}
pub(crate) fn fn_trim_start(mut v: Vec<Variable>) -> Variable {
v.remove(0).transform(|s| match s {
StringCow::Borrowed(s) => Variable::from(s.trim_start()),
StringCow::Owned(s) => Variable::from(s.trim_start().to_compact_string()),
})
}
pub(crate) fn fn_len(v: Vec<Variable>) -> Variable {
match &v[0] {
Variable::String(s) => s.len(),
Variable::Array(a) => a.len(),
v => v.to_string().len(),
}
.into()
}
pub(crate) fn fn_to_lowercase(mut v: Vec<Variable>) -> Variable {
v.remove(0)
.transform(|s| Variable::from(CompactString::from_str_to_lowercase(s.as_str())))
}
pub(crate) fn fn_to_uppercase(mut v: Vec<Variable>) -> Variable {
v.remove(0)
.transform(|s| Variable::from(CompactString::from_str_to_uppercase(s.as_str())))
}
pub(crate) fn fn_is_uppercase(mut v: Vec<Variable>) -> Variable {
v.remove(0).transform(|s| {
s.as_str()
.chars()
.filter(|c| c.is_alphabetic())
.all(|c| c.is_uppercase())
.into()
})
}
pub(crate) fn fn_is_lowercase(mut v: Vec<Variable>) -> Variable {
v.remove(0).transform(|s| {
s.as_str()
.chars()
.filter(|c| c.is_alphabetic())
.all(|c| c.is_lowercase())
.into()
})
}
pub(crate) fn fn_has_digits(mut v: Vec<Variable>) -> Variable {
v.remove(0)
.transform(|s| s.as_str().chars().any(|c| c.is_ascii_digit()).into())
}
pub(crate) fn fn_split_words(v: Vec<Variable>) -> Variable {
v[0].to_string()
.as_str()
.split_whitespace()
.filter(|word| word.chars().all(|c| c.is_alphanumeric()))
.map(|word| Variable::from(CompactString::new(word)))
.collect::<Vec<_>>()
.into()
}
pub(crate) fn fn_count_spaces(v: Vec<Variable>) -> Variable {
v[0].to_string()
.as_str()
.chars()
.filter(|c| c.is_whitespace())
.count()
.into()
}
pub(crate) fn fn_count_uppercase(v: Vec<Variable>) -> Variable {
v[0].to_string()
.as_str()
.chars()
.filter(|c| c.is_alphabetic() && c.is_uppercase())
.count()
.into()
}
pub(crate) fn fn_count_lowercase(v: Vec<Variable>) -> Variable {
v[0].to_string()
.as_str()
.chars()
.filter(|c| c.is_alphabetic() && c.is_lowercase())
.count()
.into()
}
pub(crate) fn fn_count_chars(v: Vec<Variable>) -> Variable {
v[0].to_string().as_str().chars().count().into()
}
pub(crate) fn fn_eq_ignore_case(v: Vec<Variable>) -> Variable {
v[0].to_string()
.as_str()
.eq_ignore_ascii_case(v[1].to_string().as_str())
.into()
}
pub(crate) fn fn_contains(v: Vec<Variable>) -> Variable {
match &v[0] {
Variable::String(s) => s.as_str().contains(v[1].to_string().as_str()),
Variable::Array(arr) => arr.contains(&v[1]),
val => val.to_string().as_str().contains(v[1].to_string().as_str()),
}
.into()
}
pub(crate) fn fn_contains_ignore_case(v: Vec<Variable>) -> Variable {
let needle = v[1].to_string();
match &v[0] {
Variable::String(s) => s
.as_str()
.to_lowercase()
.contains(&needle.as_str().to_lowercase()),
Variable::Array(arr) => arr.iter().any(|v| match v {
Variable::String(s) => s.as_str().eq_ignore_ascii_case(needle.as_str()),
_ => false,
}),
val => val.to_string().as_str().contains(needle.as_str()),
}
.into()
}
pub(crate) fn fn_starts_with(v: Vec<Variable>) -> Variable {
v[0].to_string()
.as_str()
.starts_with(v[1].to_string().as_str())
.into()
}
pub(crate) fn fn_ends_with(v: Vec<Variable>) -> Variable {
v[0].to_string()
.as_str()
.ends_with(v[1].to_string().as_str())
.into()
}
pub(crate) fn fn_lines(mut v: Vec<Variable>) -> Variable {
match v.remove(0) {
Variable::String(s) => s
.as_str()
.lines()
.map(|s| Variable::from(CompactString::new(s)))
.collect::<Vec<_>>()
.into(),
val => val,
}
}
pub(crate) fn fn_substring(v: Vec<Variable>) -> Variable {
v[0].to_string()
.as_str()
.chars()
.skip(v[1].to_usize().unwrap_or_default())
.take(v[2].to_usize().unwrap_or_default())
.collect::<CompactString>()
.into()
}
pub(crate) fn fn_strip_prefix(v: Vec<Variable>) -> Variable {
let mut v = v.into_iter();
let value = v.next().unwrap();
let prefix = v.next().unwrap().into_string();
value.transform(|s| match s {
StringCow::Borrowed(s) => s
.strip_prefix(prefix.as_str())
.map(Variable::from)
.unwrap_or_default(),
StringCow::Owned(s) => s
.strip_prefix(prefix.as_str())
.map(|s| Variable::from(CompactString::new(s)))
.unwrap_or_default(),
})
}
pub(crate) fn fn_strip_suffix(v: Vec<Variable>) -> Variable {
let mut v = v.into_iter();
let value = v.next().unwrap();
let suffix = v.next().unwrap().into_string();
value.transform(|s| match s {
StringCow::Borrowed(s) => s
.strip_suffix(suffix.as_str())
.map(Variable::from)
.unwrap_or_default(),
StringCow::Owned(s) => s
.strip_suffix(suffix.as_str())
.map(|s| Variable::from(CompactString::new(s)))
.unwrap_or_default(),
})
}
pub(crate) fn fn_split(v: Vec<Variable>) -> Variable {
let mut v = v.into_iter();
let value = v.next().unwrap().into_string();
let arg = v.next().unwrap().into_string();
match value {
StringCow::Borrowed(s) => s
.split(arg.as_str())
.map(Variable::from)
.collect::<Vec<_>>()
.into(),
StringCow::Owned(s) => s
.split(arg.as_str())
.map(|s| Variable::from(CompactString::new(s)))
.collect::<Vec<_>>()
.into(),
}
}
pub(crate) fn fn_rsplit(v: Vec<Variable>) -> Variable {
let mut v = v.into_iter();
let value = v.next().unwrap().into_string();
let arg = v.next().unwrap().into_string();
match value {
StringCow::Borrowed(s) => s
.rsplit(arg.as_str())
.map(Variable::from)
.collect::<Vec<_>>()
.into(),
StringCow::Owned(s) => s
.rsplit(arg.as_str())
.map(|s| Variable::from(CompactString::new(s)))
.collect::<Vec<_>>()
.into(),
}
}
pub(crate) fn fn_split_n(v: Vec<Variable>) -> Variable {
let mut v = v.into_iter();
let value = v.next().unwrap().into_string();
let arg = v.next().unwrap().into_string();
let num = v.next().unwrap().to_integer().unwrap_or_default() as usize;
fn split_n<'x, 'y>(s: &'x str, arg: &'y str, num: usize, mut f: impl FnMut(&'x str)) {
let mut s = s;
for _ in 0..num {
if let Some((a, b)) = s.split_once(arg) {
f(a);
s = b;
} else {
break;
}
}
f(s);
}
let mut result = Vec::new();
match value {
StringCow::Borrowed(s) => split_n(s, arg.as_str(), num, |s| result.push(Variable::from(s))),
StringCow::Owned(s) => split_n(&s, arg.as_str(), num, |s| {
result.push(Variable::from(CompactString::new(s)))
}),
}
result.into()
}
pub(crate) fn fn_split_once(v: Vec<Variable>) -> Variable {
let mut v = v.into_iter();
let value = v.next().unwrap().into_string();
let arg = v.next().unwrap().into_string();
match value {
StringCow::Borrowed(s) => s
.split_once(arg.as_str())
.map(|(a, b)| Variable::Array(vec![Variable::from(a), Variable::from(b)]))
.unwrap_or_default(),
StringCow::Owned(s) => s
.split_once(arg.as_str())
.map(|(a, b)| {
Variable::Array(vec![
Variable::from(CompactString::new(a)),
Variable::from(CompactString::new(b)),
])
})
.unwrap_or_default(),
}
}
pub(crate) fn fn_rsplit_once(v: Vec<Variable>) -> Variable {
let mut v = v.into_iter();
let value = v.next().unwrap().into_string();
let arg = v.next().unwrap().into_string();
match value {
StringCow::Borrowed(s) => s
.rsplit_once(arg.as_str())
.map(|(a, b)| Variable::Array(vec![Variable::from(a), Variable::from(b)]))
.unwrap_or_default(),
StringCow::Owned(s) => s
.rsplit_once(arg.as_str())
.map(|(a, b)| {
Variable::Array(vec![
Variable::from(CompactString::new(a)),
Variable::from(CompactString::new(b)),
])
})
.unwrap_or_default(),
}
}
pub(crate) fn fn_hash(v: Vec<Variable>) -> Variable {
use sha1::Digest;
let mut v = v.into_iter();
let value = v.next().unwrap().into_string();
let algo = v.next().unwrap().into_string();
match algo.as_str() {
"md5" => format_compact!("{:x}", md5::compute(value.as_bytes())).into(),
"sha1" => {
let mut hasher = Sha1::new();
hasher.update(value.as_bytes());
hasher.finalize().hex_encode().to_compact_string().into()
}
"sha256" => {
let mut hasher = Sha256::new();
hasher.update(value.as_bytes());
hasher.finalize().hex_encode().to_compact_string().into()
}
"sha512" => {
let mut hasher = Sha512::new();
hasher.update(value.as_bytes());
hasher.finalize().hex_encode().to_compact_string().into()
}
_ => Variable::default(),
}
}
crates/common/src/expr/if_block.rs
+250
View File
@@ -0,0 +1,250 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use super::{
ExpressionItem,
parser::ExpressionParser,
tokenizer::{TokenMap, Tokenizer},
};
use crate::expr::{Constant, Expression};
use compact_str::CompactString;
use registry::{
schema::{
prelude::{ExpressionContext, Property},
structs,
},
types::id::ObjectId,
};
use store::registry::bootstrap::Bootstrap;
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct IfThen {
pub expr: Expression,
pub then: Expression,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct IfBlock {
pub id: ObjectId,
pub property: Property,
pub if_then: Box<[IfThen]>,
pub default: Expression,
}
impl IfBlock {
pub fn new_default(id: ObjectId, expr_ctx: ExpressionContext<'_>) -> Self {
let token_map = TokenMap::default();
if let Some(default) = expr_ctx.default {
Self {
id,
property: expr_ctx.property,
if_then: default
.match_
.into_iter()
.map(|match_| IfThen {
expr: Expression::parse(&token_map, &match_.if_),
then: Expression::parse(&token_map, &match_.then),
})
.collect(),
default: Expression::parse(&token_map, &default.else_),
}
} else {
Self::empty(id, expr_ctx.property)
}
}
pub fn empty(id: ObjectId, property: Property) -> Self {
Self {
id,
property,
if_then: Default::default(),
default: Expression {
items: Default::default(),
},
}
}
pub fn is_empty(&self) -> bool {
self.default.is_empty() && self.if_then.is_empty()
}
}
impl Expression {
pub fn parse(token_map: &TokenMap, expr: &str) -> Self {
ExpressionParser::new(Tokenizer::new(expr, token_map))
.parse()
.unwrap()
}
}
pub trait BootstrapExprExt {
fn compile_expr(&mut self, id: ObjectId, expr_ctx: &ExpressionContext<'_>) -> IfBlock;
fn compile_default_expr(&mut self, id: ObjectId, expr_ctx: &ExpressionContext<'_>) -> IfBlock;
fn try_compile_expr(
&mut self,
id: ObjectId,
expr_ctx: &ExpressionContext<'_>,
expr: &structs::Expression,
) -> Option<IfBlock>;
}
impl BootstrapExprExt for Bootstrap {
fn compile_expr(&mut self, id: ObjectId, expr_ctx: &ExpressionContext<'_>) -> IfBlock {
if expr_ctx.expr.else_.is_empty() && expr_ctx.expr.match_.is_empty() {
return IfBlock::empty(id, expr_ctx.property);
}
if let Some(if_block) = self.try_compile_expr(id, expr_ctx, expr_ctx.expr) {
if_block
} else {
self.compile_default_expr(id, expr_ctx)
}
}
fn compile_default_expr(&mut self, id: ObjectId, expr_ctx: &ExpressionContext<'_>) -> IfBlock {
if let Some(default) = &expr_ctx.default {
self.try_compile_expr(id, expr_ctx, default)
.expect("Valid default expression")
} else {
IfBlock::empty(id, expr_ctx.property)
}
}
fn try_compile_expr(
&mut self,
id: ObjectId,
expr_ctx: &ExpressionContext<'_>,
expr: &structs::Expression,
) -> Option<IfBlock> {
// Parse conditions
let mut if_then = Vec::with_capacity(expr.match_.len());
if expr.else_.is_empty() {
if !expr.match_.is_empty() {
self.invalid_property(
id,
expr_ctx.property,
"Missing 'else' block in 'if' expression",
);
}
return None;
}
if expr
.match_
.iter()
.any(|m| m.if_.is_empty() || m.then.is_empty())
{
self.invalid_property(
id,
expr_ctx.property,
"All 'if' and 'then' blocks must be non-empty",
);
return None;
}
let token_map = TokenMap::default()
.with_variables(expr_ctx.allowed_variables)
.with_constants(expr_ctx.allowed_constants);
let default = match ExpressionParser::new(Tokenizer::new(&expr.else_, &token_map)).parse() {
Ok(expr) => expr,
Err(err) => {
self.invalid_property(
id,
expr_ctx.property,
format!("Error parsing 'else' expression: {}", err),
);
return None;
}
};
for (num, match_) in expr.match_.iter().enumerate() {
match ExpressionParser::new(Tokenizer::new(&match_.if_, &token_map)).parse() {
Ok(if_expr) => {
match ExpressionParser::new(Tokenizer::new(&match_.then, &token_map)).parse() {
Ok(then_expr) => {
if_then.push(IfThen {
expr: if_expr,
then: then_expr,
});
}
Err(err) => {
self.invalid_property(
id,
expr_ctx.property,
format!(
"Error parsing 'then' expression in condition #{}: {}",
num + 1,
err
),
);
return None;
}
}
}
Err(err) => {
self.invalid_property(
id,
expr_ctx.property,
format!(
"Error parsing 'if' expression in condition #{}: {}",
num + 1,
err
),
);
return None;
}
}
}
Some(IfBlock {
id,
property: expr_ctx.property,
if_then: if_then.into_boxed_slice(),
default,
})
}
}
impl IfBlock {
pub fn into_default(self, id: ObjectId, property: Property) -> IfBlock {
IfBlock {
id,
property,
if_then: Default::default(),
default: self.default,
}
}
pub fn all_items(&self) -> impl Iterator<Item = &ExpressionItem> {
self.if_then
.iter()
.flat_map(|if_then| if_then.expr.items().iter().chain(if_then.then.items()))
.chain(self.default.items())
}
pub fn default_string(&self) -> Option<&str> {
for expr_item in &self.default.items {
if let ExpressionItem::Constant(Constant::String(value)) = expr_item {
return Some(value.as_str());
}
}
None
}
pub fn into_default_string(self) -> Option<CompactString> {
for expr_item in self.default.items {
if let ExpressionItem::Constant(Constant::String(value)) = expr_item {
return Some(value);
}
}
None
}
}
crates/common/src/expr/mod.rs
+523
View File
@@ -0,0 +1,523 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use compact_str::CompactString;
use regex::Regex;
use registry::schema::{
enums::{ExpressionConstant, ExpressionVariable},
structs::Rate,
};
use std::{
borrow::Cow,
fmt::{Display, Formatter},
net::{IpAddr, Ipv4Addr, Ipv6Addr},
str::FromStr,
time::Duration,
};
use trc::MetricType;
use utils::cache::CacheItemWeight;
use crate::expr::if_block::IfBlock;
pub mod eval;
pub mod functions;
pub mod if_block;
pub mod parser;
pub mod tokenizer;
#[derive(Debug, PartialEq, Eq, Clone, Default)]
#[repr(transparent)]
pub struct Expression {
pub items: Box<[ExpressionItem]>,
}
#[derive(Debug, Clone)]
pub enum ExpressionItem {
Variable(ExpressionVariable),
Global(CompactString),
System(SystemVariable),
Capture(u32),
Constant(Constant),
BinaryOperator(BinaryOperator),
UnaryOperator(UnaryOperator),
Regex(Regex),
JmpIf { val: bool, pos: u32 },
Function { id: u32, num_args: u32 },
ArrayAccess,
ArrayBuild(u32),
}
#[derive(Debug, Clone)]
pub enum Variable<'x> {
String(StringCow<'x>),
Integer(i64),
Float(f64),
Array(Vec<Variable<'x>>),
Constant(ExpressionConstant),
}
#[derive(Debug, Clone)]
pub enum StringCow<'x> {
Owned(CompactString),
Borrowed(&'x str),
}
impl Default for Variable<'_> {
fn default() -> Self {
Variable::String(StringCow::Borrowed(""))
}
}
#[derive(Debug, PartialEq, Clone)]
pub enum Constant {
Static(ExpressionConstant),
Integer(i64),
Float(f64),
String(CompactString),
}
impl Eq for Constant {}
impl From<CompactString> for Constant {
fn from(value: CompactString) -> Self {
Constant::String(value)
}
}
impl From<bool> for Constant {
fn from(value: bool) -> Self {
Constant::Integer(value as i64)
}
}
impl From<i64> for Constant {
fn from(value: i64) -> Self {
Constant::Integer(value)
}
}
impl From<i32> for Constant {
fn from(value: i32) -> Self {
Constant::Integer(value as i64)
}
}
impl From<i16> for Constant {
fn from(value: i16) -> Self {
Constant::Integer(value as i64)
}
}
impl From<f64> for Constant {
fn from(value: f64) -> Self {
Constant::Float(value)
}
}
impl From<usize> for Constant {
fn from(value: usize) -> Self {
Constant::Integer(value as i64)
}
}
#[derive(Debug, PartialEq, Eq, Clone, Copy)]
pub enum BinaryOperator {
Add,
Subtract,
Multiply,
Divide,
And,
Or,
Xor,
Eq,
Ne,
Lt,
Le,
Gt,
Ge,
}
#[derive(Debug, PartialEq, Eq, Clone, Copy)]
pub enum UnaryOperator {
Not,
Minus,
}
#[derive(Debug, Clone)]
pub enum Token {
Variable(ExpressionVariable),
Global(CompactString),
Capture(u32),
Function {
name: Cow<'static, str>,
id: u32,
num_args: u32,
},
Constant(Constant),
System(SystemVariable),
Regex(Regex),
BinaryOperator(BinaryOperator),
UnaryOperator(UnaryOperator),
OpenParen,
CloseParen,
OpenBracket,
CloseBracket,
Comma,
}
#[derive(Debug, Clone)]
pub enum SystemVariable {
Hostname,
Domain,
NodeId,
NodeHostname,
NodeRole,
Metric(MetricType),
}
impl From<usize> for Variable<'_> {
fn from(value: usize) -> Self {
Variable::Integer(value as i64)
}
}
impl From<i64> for Variable<'_> {
fn from(value: i64) -> Self {
Variable::Integer(value)
}
}
impl From<u64> for Variable<'_> {
fn from(value: u64) -> Self {
Variable::Integer(value as i64)
}
}
impl From<i32> for Variable<'_> {
fn from(value: i32) -> Self {
Variable::Integer(value as i64)
}
}
impl From<u32> for Variable<'_> {
fn from(value: u32) -> Self {
Variable::Integer(value as i64)
}
}
impl From<u16> for Variable<'_> {
fn from(value: u16) -> Self {
Variable::Integer(value as i64)
}
}
impl From<i16> for Variable<'_> {
fn from(value: i16) -> Self {
Variable::Integer(value as i64)
}
}
impl From<f64> for Variable<'_> {
fn from(value: f64) -> Self {
Variable::Float(value)
}
}
impl<'x> From<&'x str> for Variable<'x> {
fn from(value: &'x str) -> Self {
Variable::String(StringCow::Borrowed(value))
}
}
impl From<CompactString> for Variable<'_> {
fn from(value: CompactString) -> Self {
Variable::String(StringCow::Owned(value))
}
}
impl<'x> From<Vec<Variable<'x>>> for Variable<'x> {
fn from(value: Vec<Variable<'x>>) -> Self {
Variable::Array(value)
}
}
impl From<bool> for Variable<'_> {
fn from(value: bool) -> Self {
Variable::Integer(value as i64)
}
}
impl<T: Into<Constant>> From<T> for Expression {
fn from(value: T) -> Self {
Expression {
items: Box::new([ExpressionItem::Constant(value.into())]),
}
}
}
impl PartialEq for ExpressionItem {
fn eq(&self, other: &Self) -> bool {
match (self, other) {
(Self::Variable(l0), Self::Variable(r0)) => l0 == r0,
(Self::Constant(l0), Self::Constant(r0)) => l0 == r0,
(Self::BinaryOperator(l0), Self::BinaryOperator(r0)) => l0 == r0,
(Self::UnaryOperator(l0), Self::UnaryOperator(r0)) => l0 == r0,
(Self::Regex(_), Self::Regex(_)) => true,
(
Self::JmpIf {
val: l_val,
pos: l_pos,
},
Self::JmpIf {
val: r_val,
pos: r_pos,
},
) => l_val == r_val && l_pos == r_pos,
(
Self::Function {
id: l_id,
num_args: l_num_args,
},
Self::Function {
id: r_id,
num_args: r_num_args,
},
) => l_id == r_id && l_num_args == r_num_args,
(Self::ArrayBuild(l0), Self::ArrayBuild(r0)) => l0 == r0,
_ => core::mem::discriminant(self) == core::mem::discriminant(other),
}
}
}
impl Eq for ExpressionItem {}
impl PartialEq for Token {
fn eq(&self, other: &Self) -> bool {
match (self, other) {
(Self::Variable(l0), Self::Variable(r0)) => l0 == r0,
(
Self::Function {
name: l_name,
id: l_id,
num_args: l_num_args,
},
Self::Function {
name: r_name,
id: r_id,
num_args: r_num_args,
},
) => l_name == r_name && l_id == r_id && l_num_args == r_num_args,
(Self::Constant(l0), Self::Constant(r0)) => l0 == r0,
(Self::Regex(_), Self::Regex(_)) => true,
(Self::BinaryOperator(l0), Self::BinaryOperator(r0)) => l0 == r0,
(Self::UnaryOperator(l0), Self::UnaryOperator(r0)) => l0 == r0,
_ => core::mem::discriminant(self) == core::mem::discriminant(other),
}
}
}
impl Eq for Token {}
impl From<()> for Constant {
fn from(_: ()) -> Self {
Constant::Integer(0)
}
}
impl<'x> TryFrom<Variable<'x>> for () {
type Error = ();
fn try_from(_: Variable<'x>) -> Result<Self, Self::Error> {
Ok(())
}
}
impl<'x> TryFrom<Variable<'x>> for Duration {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
match value {
Variable::Integer(value) if value > 0 => Ok(Duration::from_millis(value as u64)),
Variable::Float(value) if value > 0.0 => Ok(Duration::from_millis(value as u64)),
Variable::String(value) if !value.is_empty() => {
registry::types::duration::Duration::from_str(value.as_str())
.map(|v| v.into_inner())
.map_err(|_| ())
}
_ => Err(()),
}
}
}
impl StringCow<'_> {
pub fn as_str(&self) -> &str {
match self {
StringCow::Owned(s) => s.as_str(),
StringCow::Borrowed(s) => s,
}
}
pub fn as_bytes(&self) -> &[u8] {
match self {
StringCow::Owned(s) => s.as_bytes(),
StringCow::Borrowed(s) => s.as_bytes(),
}
}
pub fn is_empty(&self) -> bool {
match self {
StringCow::Owned(s) => s.is_empty(),
StringCow::Borrowed(s) => s.is_empty(),
}
}
pub fn len(&self) -> usize {
match self {
StringCow::Owned(s) => s.len(),
StringCow::Borrowed(s) => s.len(),
}
}
pub fn into_owned(self) -> CompactString {
match self {
StringCow::Owned(s) => s,
StringCow::Borrowed(s) => s.into(),
}
}
}
impl<'x> From<Cow<'x, str>> for StringCow<'x> {
fn from(value: Cow<'x, str>) -> Self {
match value {
Cow::Borrowed(s) => StringCow::Borrowed(s),
Cow::Owned(s) => StringCow::Owned(s.into()),
}
}
}
impl From<CompactString> for StringCow<'_> {
fn from(value: CompactString) -> Self {
StringCow::Owned(value)
}
}
impl AsRef<str> for StringCow<'_> {
fn as_ref(&self) -> &str {
self.as_str()
}
}
impl AsRef<[u8]> for StringCow<'_> {
fn as_ref(&self) -> &[u8] {
self.as_str().as_bytes()
}
}
impl Display for StringCow<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
match self {
StringCow::Owned(s) => write!(f, "{}", s),
StringCow::Borrowed(s) => write!(f, "{}", s),
}
}
}
impl From<Duration> for Constant {
fn from(value: Duration) -> Self {
Constant::Integer(value.as_millis() as i64)
}
}
impl<'x> TryFrom<Variable<'x>> for Rate {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
match value {
Variable::Array(items) if items.len() == 2 => {
let requests = items[0].to_integer().ok_or(())?;
let period = items[1].to_integer().ok_or(())?;
if requests > 0 && period > 0 {
Ok(Rate {
count: requests as u64,
period: registry::types::duration::Duration::from_millis(period as u64),
})
} else {
Err(())
}
}
_ => Err(()),
}
}
}
impl<'x> TryFrom<Variable<'x>> for Ipv4Addr {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
match value {
Variable::String(value) => value.as_str().parse().map_err(|_| ()),
_ => Err(()),
}
}
}
impl<'x> TryFrom<Variable<'x>> for Ipv6Addr {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
match value {
Variable::String(value) => value.as_str().parse().map_err(|_| ()),
_ => Err(()),
}
}
}
impl<'x> TryFrom<Variable<'x>> for IpAddr {
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
match value {
Variable::String(value) => value.as_str().parse().map_err(|_| ()),
_ => Err(()),
}
}
}
impl<'x, T: TryFrom<Variable<'x>>> TryFrom<Variable<'x>> for Vec<T>
where
Result<Vec<T>, ()>: FromIterator<Result<T, <T as TryFrom<Variable<'x>>>::Error>>,
{
type Error = ();
fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {
value
.into_array()
.into_iter()
.map(|v| T::try_from(v))
.collect()
}
}
impl CacheItemWeight for Expression {
fn weight(&self) -> u64 {
self.items.len() as u64 * std::mem::size_of::<ExpressionItem>() as u64
}
}
impl CacheItemWeight for IfBlock {
fn weight(&self) -> u64 {
std::mem::size_of::<IfBlock>() as u64
+ self
.if_then
.iter()
.map(|if_then| if_then.expr.weight() + if_then.then.weight())
.sum::<u64>()
+ self.default.weight()
}
}
crates/common/src/expr/parser.rs
+277
View File
@@ -0,0 +1,277 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use super::{BinaryOperator, Expression, ExpressionItem, Token, tokenizer::Tokenizer};
pub struct ExpressionParser<'x> {
pub(crate) tokenizer: Tokenizer<'x>,
pub(crate) output: Vec<ExpressionItem>,
operator_stack: Vec<(Token, Option<usize>)>,
arg_count: Vec<i32>,
}
pub(crate) const ID_ARRAY_ACCESS: u32 = u32::MAX;
pub(crate) const ID_ARRAY_BUILD: u32 = u32::MAX - 1;
impl<'x> ExpressionParser<'x> {
pub fn new(tokenizer: Tokenizer<'x>) -> Self {
Self {
tokenizer,
output: Vec::new(),
operator_stack: Vec::new(),
arg_count: Vec::new(),
}
}
pub fn parse(mut self) -> Result<Expression, String> {
let mut last_is_var_or_fnc = false;
while let Some(token) = self.tokenizer.next()? {
let mut is_var_or_fnc = false;
match token {
Token::Variable(v) => {
self.inc_arg_count();
is_var_or_fnc = true;
self.output.push(ExpressionItem::Variable(v))
}
Token::Constant(c) => {
self.inc_arg_count();
self.output.push(ExpressionItem::Constant(c))
}
Token::Global(g) => {
self.inc_arg_count();
self.output.push(ExpressionItem::Global(g))
}
Token::Capture(c) => {
self.inc_arg_count();
self.output.push(ExpressionItem::Capture(c))
}
Token::UnaryOperator(uop) => {
self.operator_stack.push((Token::UnaryOperator(uop), None))
}
Token::OpenParen => self.operator_stack.push((token, None)),
Token::CloseParen | Token::CloseBracket => {
let expect_token = if matches!(token, Token::CloseParen) {
Token::OpenParen
} else {
Token::OpenBracket
};
loop {
match self.operator_stack.pop() {
Some((t, _)) if t == expect_token => {
break;
}
Some((Token::BinaryOperator(bop), jmp_pos)) => {
self.update_jmp_pos(jmp_pos);
self.output.push(ExpressionItem::BinaryOperator(bop))
}
Some((Token::UnaryOperator(uop), _)) => {
self.output.push(ExpressionItem::UnaryOperator(uop))
}
_ => return Err("Mismatched parentheses".to_string()),
}
}
match self.operator_stack.last() {
Some((Token::Function { id, num_args, name }, _)) => {
let got_args = self.arg_count.pop().unwrap();
if got_args != *num_args as i32 {
return Err(if *id != u32::MAX {
format!(
"Expression function {:?} expected {} arguments, got {}",
name, num_args, got_args
)
} else {
"Missing array index".to_string()
});
}
let expr = match *id {
ID_ARRAY_ACCESS => ExpressionItem::ArrayAccess,
ID_ARRAY_BUILD => ExpressionItem::ArrayBuild(*num_args),
id => ExpressionItem::Function {
id,
num_args: *num_args,
},
};
self.operator_stack.pop();
self.output.push(expr);
}
Some((Token::Regex(regex), _)) => {
if self.arg_count.pop().unwrap() != 1 {
return Err("Expression function \"matches\" expected 2 arguments"
.to_string());
}
self.output.push(ExpressionItem::Regex(regex.clone()));
self.operator_stack.pop();
}
Some((Token::System(setting), _)) => {
if self.arg_count.pop().unwrap() != 0 {
return Err("Expression function expected 1 argument".to_string());
}
self.output.push(ExpressionItem::System(setting.clone()));
self.operator_stack.pop();
}
_ => {}
}
is_var_or_fnc = true;
}
Token::BinaryOperator(bop) => {
self.dec_arg_count();
while let Some((top_token, prev_jmp_pos)) = self.operator_stack.last() {
match top_token {
Token::BinaryOperator(top_bop) => {
if bop.precedence() <= top_bop.precedence() {
let top_bop = *top_bop;
let jmp_pos = *prev_jmp_pos;
self.update_jmp_pos(jmp_pos);
self.operator_stack.pop();
self.output.push(ExpressionItem::BinaryOperator(top_bop));
} else {
break;
}
}
Token::UnaryOperator(top_uop) => {
let top_uop = *top_uop;
self.operator_stack.pop();
self.output.push(ExpressionItem::UnaryOperator(top_uop));
}
_ => break,
}
}
// Add jump instruction for short-circuiting
let jmp_pos = match bop {
BinaryOperator::And => {
self.output
.push(ExpressionItem::JmpIf { val: false, pos: 0 });
Some(self.output.len() - 1)
}
BinaryOperator::Or => {
self.output
.push(ExpressionItem::JmpIf { val: true, pos: 0 });
Some(self.output.len() - 1)
}
_ => None,
};
self.operator_stack
.push((Token::BinaryOperator(bop), jmp_pos));
}
token @ (Token::Function { .. } | Token::Regex(_) | Token::System(_)) => {
self.inc_arg_count();
self.arg_count.push(0);
self.operator_stack.push((token, None))
}
Token::OpenBracket => {
// Array functions
let (id, num_args, arg_count) = if last_is_var_or_fnc {
(ID_ARRAY_ACCESS, 2, 1)
} else {
self.inc_arg_count();
(ID_ARRAY_BUILD, 0, 0)
};
self.arg_count.push(arg_count);
self.operator_stack.push((
Token::Function {
id,
name: "array".into(),
num_args,
},
None,
));
self.operator_stack.push((token, None));
}
Token::Comma => {
while let Some((token, jmp_pos)) = self.operator_stack.last() {
match token {
Token::OpenParen => break,
Token::BinaryOperator(bop) => {
let bop = *bop;
let jmp_pos = *jmp_pos;
self.update_jmp_pos(jmp_pos);
self.output.push(ExpressionItem::BinaryOperator(bop));
self.operator_stack.pop();
}
Token::UnaryOperator(uop) => {
self.output.push(ExpressionItem::UnaryOperator(*uop));
self.operator_stack.pop();
}
_ => break,
}
}
}
}
last_is_var_or_fnc = is_var_or_fnc;
}
while let Some((token, jmp_pos)) = self.operator_stack.pop() {
match token {
Token::BinaryOperator(bop) => {
self.update_jmp_pos(jmp_pos);
self.output.push(ExpressionItem::BinaryOperator(bop))
}
Token::UnaryOperator(uop) => self.output.push(ExpressionItem::UnaryOperator(uop)),
_ => return Err("Invalid token on the operator stack".to_string()),
}
}
if self.operator_stack.is_empty() {
Ok(Expression {
items: self.output.into_boxed_slice(),
})
} else {
Err("Invalid expression".to_string())
}
}
fn inc_arg_count(&mut self) {
if let Some(x) = self.arg_count.last_mut() {
*x = x.saturating_add(1);
let op_pos = self.operator_stack.len().saturating_sub(2);
match self.operator_stack.get_mut(op_pos) {
Some((Token::Function { num_args, id, .. }, _)) if *id == ID_ARRAY_BUILD => {
*num_args += 1;
}
_ => {}
}
}
}
fn dec_arg_count(&mut self) {
if let Some(x) = self.arg_count.last_mut() {
*x = x.saturating_sub(1);
}
}
fn update_jmp_pos(&mut self, jmp_pos: Option<usize>) {
if let Some(jmp_pos) = jmp_pos {
let cur_pos = self.output.len();
if let ExpressionItem::JmpIf { pos, .. } = &mut self.output[jmp_pos] {
*pos = (cur_pos - jmp_pos) as u32;
} else {
#[cfg(test)]
panic!("Invalid jump position");
}
}
}
}
impl BinaryOperator {
fn precedence(&self) -> i32 {
match self {
BinaryOperator::Multiply | BinaryOperator::Divide => 7,
BinaryOperator::Add | BinaryOperator::Subtract => 6,
BinaryOperator::Gt | BinaryOperator::Ge | BinaryOperator::Lt | BinaryOperator::Le => 5,
BinaryOperator::Eq | BinaryOperator::Ne => 4,
BinaryOperator::Xor => 3,
BinaryOperator::And => 2,
BinaryOperator::Or => 1,
}
}
}
crates/common/src/expr/tokenizer.rs
+401
View File
@@ -0,0 +1,401 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use super::{
functions::{ASYNC_FUNCTIONS, FUNCTIONS},
*,
};
use ahash::AHashSet;
use regex::Regex;
use registry::{schema::enums::ExpressionConstant, types::EnumImpl};
use std::{borrow::Cow, iter::Peekable, slice::Iter};
use trc::MetricType;
pub struct Tokenizer<'x> {
pub(crate) iter: Peekable<Iter<'x, u8>>,
token_map: &'x TokenMap,
buf: Vec<u8>,
depth: u32,
next_token: Vec<Token>,
has_number: bool,
has_dot: bool,
has_alpha: bool,
is_start: bool,
is_eof: bool,
}
#[derive(Debug, Default, Clone)]
pub struct TokenMap {
pub variables: AHashSet<ExpressionVariable>,
pub constants: AHashSet<ExpressionConstant>,
}
impl<'x> Tokenizer<'x> {
#[allow(clippy::should_implement_trait)]
pub fn new(expr: &'x str, token_map: &'x TokenMap) -> Self {
Self {
iter: expr.as_bytes().iter().peekable(),
buf: Vec::new(),
depth: 0,
next_token: Vec::with_capacity(2),
has_number: false,
has_dot: false,
has_alpha: false,
is_start: true,
is_eof: false,
token_map,
}
}
#[allow(clippy::should_implement_trait)]
pub fn next(&mut self) -> Result<Option<Token>, String> {
if let Some(token) = self.next_token.pop() {
return Ok(Some(token));
} else if self.is_eof {
return Ok(None);
}
while let Some(&ch) = self.iter.next() {
match ch {
b'A'..=b'Z' | b'a'..=b'z' | b'_' | b'$' => {
self.buf.push(ch);
self.has_alpha = true;
}
b'0'..=b'9' => {
self.buf.push(ch);
self.has_number = true;
}
b'.' => {
self.buf.push(ch);
self.has_dot = true;
}
b'}' => {
self.is_eof = true;
break;
}
b'-' if self.buf.last().is_some_and(|c| *c == b'[') => {
self.buf.push(ch);
}
b':' if self.buf.contains(&b'.') => {
self.buf.push(ch);
}
b']' if self.buf.contains(&b'[') => {
self.buf.push(b']');
}
b'*' if self.buf.last().is_some_and(|&c| c == b'[' || c == b'.') => {
self.buf.push(ch);
}
_ => {
let (prev_token, ch) = if ch == b'(' && !self.buf.is_empty() {
match self.buf.as_slice() {
b"matches" => {
// Parse regular expressions
let stop_ch = self.find_char(b"\"'")?;
let regex_str = self.parse_string(stop_ch)?;
let regex = Regex::new(&regex_str).map_err(|e| {
format!("Invalid regular expression {:?}: {}", regex_str, e)
})?;
self.has_alpha = false;
self.buf.clear();
self.find_char(b",")?;
(Token::Regex(regex).into(), b'(')
}
b"metric" => {
let stop_ch = self.find_char(b"\"'")?;
let metric_str = self.parse_string(stop_ch)?;
let metric = MetricType::parse(&metric_str).ok_or_else(|| {
format!("Invalid metric name {:?}", metric_str)
})?;
self.has_alpha = false;
self.buf.clear();
(Token::System(SystemVariable::Metric(metric)).into(), b'(')
}
b"system" => {
let stop_ch = self.find_char(b"\"'")?;
let var = match self.parse_string(stop_ch)?.as_str() {
"domain" => SystemVariable::Domain,
"hostname" => SystemVariable::Hostname,
"node_id" => SystemVariable::NodeId,
"node_hostname" => SystemVariable::NodeHostname,
"node_role" => SystemVariable::NodeRole,
other => {
return Err(format!(
"Invalid system variable name {:?}",
other
));
}
};
self.has_alpha = false;
self.buf.clear();
(Token::System(var).into(), b'(')
}
_ => {
self.is_start = false;
(self.parse_buf()?.into(), ch)
}
}
} else if !self.buf.is_empty() {
self.is_start = false;
(self.parse_buf()?.into(), ch)
} else {
(None, ch)
};
let token = match ch {
b'&' => {
if matches!(self.iter.peek(), Some(b'&')) {
self.iter.next();
}
Token::BinaryOperator(BinaryOperator::And)
}
b'|' => {
if matches!(self.iter.peek(), Some(b'|')) {
self.iter.next();
}
Token::BinaryOperator(BinaryOperator::Or)
}
b'!' => {
if matches!(self.iter.peek(), Some(b'=')) {
self.iter.next();
Token::BinaryOperator(BinaryOperator::Ne)
} else {
Token::UnaryOperator(UnaryOperator::Not)
}
}
b'^' => Token::BinaryOperator(BinaryOperator::Xor),
b'(' => {
self.depth += 1;
Token::OpenParen
}
b')' => {
if self.depth == 0 {
return Err("Unmatched close parenthesis".to_string());
}
self.depth -= 1;
Token::CloseParen
}
b'+' => Token::BinaryOperator(BinaryOperator::Add),
b'*' => Token::BinaryOperator(BinaryOperator::Multiply),
b'/' => Token::BinaryOperator(BinaryOperator::Divide),
b'-' => {
if self.is_start {
Token::UnaryOperator(UnaryOperator::Minus)
} else {
Token::BinaryOperator(BinaryOperator::Subtract)
}
}
b'=' => match self.iter.next() {
Some(b'=') => Token::BinaryOperator(BinaryOperator::Eq),
Some(b'>') => Token::BinaryOperator(BinaryOperator::Ge),
Some(b'<') => Token::BinaryOperator(BinaryOperator::Le),
_ => Token::BinaryOperator(BinaryOperator::Eq),
},
b'>' => match self.iter.peek() {
Some(b'=') => {
self.iter.next();
Token::BinaryOperator(BinaryOperator::Ge)
}
_ => Token::BinaryOperator(BinaryOperator::Gt),
},
b'<' => match self.iter.peek() {
Some(b'=') => {
self.iter.next();
Token::BinaryOperator(BinaryOperator::Le)
}
_ => Token::BinaryOperator(BinaryOperator::Lt),
},
b',' => Token::Comma,
b'[' => Token::OpenBracket,
b']' => Token::CloseBracket,
b' ' | b'\r' | b'\n' => {
if prev_token.is_some() {
return Ok(prev_token);
} else {
continue;
}
}
b'\"' | b'\'' => Token::Constant(Constant::String(self.parse_string(ch)?)),
_ => {
return Err(format!("Invalid character {:?}", char::from(ch),));
}
};
self.is_start = matches!(
token,
Token::OpenParen | Token::Comma | Token::BinaryOperator(_)
);
return if prev_token.is_some() {
self.next_token.push(token);
Ok(prev_token)
} else {
Ok(Some(token))
};
}
}
}
if self.depth > 0 {
Err("Unmatched open parenthesis".to_string())
} else if !self.buf.is_empty() {
self.parse_buf().map(Some)
} else {
Ok(None)
}
}
fn find_char(&mut self, chars: &[u8]) -> Result<u8, String> {
for &ch in self.iter.by_ref() {
if !ch.is_ascii_whitespace() {
return if chars.contains(&ch) {
Ok(ch)
} else {
Err(format!(
"Expected {:?}, found invalid character {:?}",
char::from(chars[0]),
char::from(ch),
))
};
}
}
Err("Unexpected end of expression".to_string())
}
fn parse_string(&mut self, stop_ch: u8) -> Result<CompactString, String> {
let mut buf = Vec::with_capacity(16);
let mut last_ch = 0;
let mut found_end = false;
for &ch in self.iter.by_ref() {
if last_ch != b'\\' {
if ch != stop_ch {
buf.push(ch);
} else {
found_end = true;
break;
}
} else {
match ch {
b'n' => {
buf.push(b'\n');
}
b'r' => {
buf.push(b'\r');
}
b't' => {
buf.push(b'\t');
}
_ => {
buf.push(ch);
}
}
}
last_ch = ch;
}
if found_end {
CompactString::from_utf8(buf).map_err(|_| "Invalid UTF-8".into())
} else {
Err("Unterminated string".to_string())
}
}
fn parse_buf(&mut self) -> Result<Token, String> {
let buf = String::from_utf8(std::mem::take(&mut self.buf)).unwrap_or_default();
if self.has_number && !self.has_alpha {
self.has_number = false;
if self.has_dot {
self.has_dot = false;
buf.parse::<f64>()
.map(|f| Token::Constant(Constant::Float(f)))
.map_err(|_| format!("Invalid float value {}", buf,))
} else {
buf.parse::<i64>()
.map(|i| Token::Constant(Constant::Integer(i)))
.map_err(|_| format!("Invalid integer value {}", buf,))
}
} else {
let has_dot = self.has_dot;
let has_number = self.has_number;
self.has_alpha = false;
self.has_number = false;
self.has_dot = false;
if !has_number && !has_dot && [4, 5].contains(&buf.len()) {
if buf == "true" {
return Ok(Token::Constant(Constant::Integer(1)));
} else if buf == "false" {
return Ok(Token::Constant(Constant::Integer(0)));
}
}
if let Some(variable) = buf.strip_prefix('$').filter(|s| !s.is_empty()) {
if variable.chars().all(|c| c.is_ascii_digit()) {
Ok(variable
.parse::<u32>()
.map(Token::Capture)
.unwrap_or_else(|_| Token::Global(variable.into())))
} else {
Ok(Token::Global(variable.into()))
}
} else if let Some((idx, (name, _, num_args))) = FUNCTIONS
.iter()
.enumerate()
.find(|(_, (name, _, _))| name == &buf)
{
Ok(Token::Function {
name: Cow::Borrowed(*name),
id: idx as u32,
num_args: *num_args,
})
} else if let Some((name, idx, num_args)) =
ASYNC_FUNCTIONS.iter().find(|(name, _, _)| name == &buf)
{
Ok(Token::Function {
name: Cow::Borrowed(*name),
id: *idx + FUNCTIONS.len() as u32,
num_args: *num_args,
})
} else if let Some(variable) = ExpressionVariable::parse(buf.as_str()) {
if self.token_map.variables.is_empty()
|| self.token_map.variables.contains(&variable)
{
Ok(Token::Variable(variable))
} else {
Err(format!("Variable {:?} not allowed in this context", buf))
}
} else if let Some(constant) = ExpressionConstant::parse(buf.as_str()) {
if self.token_map.constants.is_empty()
|| self.token_map.constants.contains(&constant)
{
Ok(Token::Constant(Constant::Static(constant)))
} else {
Err(format!("Constant {:?} not allowed in this context", buf))
}
} else if let Ok(duration) = registry::types::duration::Duration::from_str(&buf) {
Ok(Token::Constant(Constant::Integer(
duration.as_millis() as i64
)))
} else {
Err(format!("Invalid variable or constant {buf:?}"))
}
}
}
}
impl TokenMap {
pub fn with_variables(mut self, variables: &[ExpressionVariable]) -> Self {
self.variables.extend(variables.iter().copied());
self
}
pub fn with_constants(mut self, constants: &[ExpressionConstant]) -> Self {
self.constants.extend(constants.iter().copied());
self
}
}
crates/common/src/i18n.rs
+99
View File
@@ -0,0 +1,99 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
include!(concat!(env!("OUT_DIR"), "/locales.rs"));
const TRADITIONAL_CHINESE: [&str; 4] = ["hant", "tw", "hk", "mo"];
pub fn locale_or_default(name: &str) -> &'static Locale {
if let Some(locale) = locale(name) {
return locale;
}
let mut subtags = name.split(['_', '-']);
let language = subtags.next().unwrap_or(name);
if language.eq_ignore_ascii_case("zh")
&& subtags.any(|subtag| {
TRADITIONAL_CHINESE
.iter()
.any(|variant| subtag.eq_ignore_ascii_case(variant))
})
{
return &ZH_TW_LOCALES;
}
locale_by_language(language).unwrap_or(&EN_US_LOCALES)
}
#[cfg(test)]
mod tests {
use super::{ALL_LOCALES, locale, locale_or_default};
const LOCALES: [&str; 37] = [
"en-US", "es-ES", "fr-FR", "de-DE", "it-IT", "pt-PT", "pt-BR", "nl-NL", "da-DK", "ca-ES",
"el-GR", "sv-SE", "pl-PL", "ru-RU", "uk-UA", "bg-BG", "cs-CZ", "sk-SK", "sl-SI", "hr-HR",
"lt-LT", "hu-HU", "ro-RO", "fi-FI", "nb-NO", "tr-TR", "zh-CN", "zh-TW", "ja-JP", "ko-KR",
"th-TH", "vi-VN", "id-ID", "hi-IN", "ar-SA", "he-IL", "fa-IR",
];
#[test]
fn locales_are_named_after_themselves() {
for lang in LOCALES {
assert_eq!(locale(lang).expect("locale must exist").name, lang);
}
assert_eq!(ALL_LOCALES.len(), LOCALES.len());
}
#[test]
fn bare_and_hyphenated_language_tags_resolve() {
for (input, expected) in [
("es-ES", "es-ES"),
("es", "es-ES"),
("es-MX", "es-ES"),
("pt-BR", "pt-BR"),
("pt-PT", "pt-PT"),
("pt", "pt-BR"),
("zh-Hans", "zh-CN"),
("zh-Hant", "zh-TW"),
("zh-HK", "zh-TW"),
("zh-Hant-HK", "zh-TW"),
("zh", "zh-CN"),
("zz", "en-US"),
("", "en-US"),
// BCP 47 tags are case-insensitive
("ES", "es-ES"),
("es-es", "es-ES"),
("PT-br", "pt-BR"),
("EL-GR", "el-GR"),
("ZH-HANT", "zh-TW"),
] {
assert_eq!(
locale_or_default(input).name,
expected,
"failed for {input}"
);
}
}
#[test]
fn right_to_left_locales_are_flagged() {
for lang in ["ar-SA", "he-IL", "fa-IR"] {
assert_eq!(
locale_or_default(lang).direction,
"rtl",
"failed for {lang}"
);
}
for lang in ["en-US", "de-DE", "ja-JP", "ru-RU"] {
assert_eq!(
locale_or_default(lang).direction,
"ltr",
"failed for {lang}"
);
}
}
}
crates/common/src/ipc.rs
+330
View File
@@ -0,0 +1,330 @@
/*
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*/
use crate::config::smtp::{
queue::QueueName,
report::AggregateFrequency,
resolver::{Policy, Tlsa},
};
use ahash::RandomState;
use mail_auth::{
dmarc::Dmarc,
mta_sts::TlsRpt,
report::{Record, tlsrpt::FailureDetails},
};
use registry::{schema::prelude::ObjectType, types::id::ObjectId};
use std::sync::{
Arc,
atomic::{AtomicBool, Ordering},
};
use tokio::sync::{Semaphore, SemaphorePermit, mpsc};
use types::type_state::{DataType, StateChange};
use utils::map::bitmap::Bitmap;
#[derive(Debug)]
pub enum PushEvent {
Subscribe {
account_ids: Vec<u32>,
types: Bitmap<DataType>,
tx: mpsc::Sender<PushNotification>,
},
Publish {
notification: PushNotification,
broadcast: bool,
},
PushServerRegister {
activate: Vec<u32>,
expired: Vec<u32>,
},
PushServerUpdate {
account_id: u32,
broadcast: bool,
},
Stop,
}
#[derive(Debug, Clone)]
pub enum PushNotification {
StateChange(StateChange),
CalendarAlert(CalendarAlert),
EmailPush(EmailPush),
}
#[derive(Debug, Clone)]
pub struct EmailPush {
pub account_id: u32,
pub email_id: u32,
pub change_id: u64,
}
#[derive(Debug, Clone)]
pub struct CalendarAlert {
pub account_id: u32,
pub event_id: u32,
pub recurrence_id: Option<i64>,
pub uid: String,
pub alert_id: String,
}
#[derive(Debug)]
pub enum BroadcastEvent {
PushNotification(PushNotification),
PushServerUpdate(u32),
RegistryChange(RegistryChange),
CacheInvalidate(Vec<CacheInvalidation>),
CacheInvalidateAll,
CacheInvalidateNegative,
MtaQueueStatus { is_running: bool },
QueueRefresh,
}
#[derive(Debug, Clone, Copy)]
pub enum RegistryChange {
Insert(ObjectId),
Delete(ObjectId),
Reload(ObjectType),
}
#[derive(Debug, Clone, Copy, Hash, PartialEq, Eq)]
pub enum CacheInvalidation {
AccessToken(u32),
DavResources(u32),
Domain(u32),
Account(u32),
DkimSignature(u32),
Tenant(u32),
Role(u32),
List(u32),
DomainLogo(u32),
TenantLogo(u32),
EmailNegative {
domain_id: u32,
local_part_hash: u32,
},
DomainNegative,
}
#[derive(Debug)]
pub enum QueueEvent {
Refresh,
WorkerDone {
queue_id: u64,
queue_name: QueueName,
status: QueueEventStatus,
},
Paused(bool),
ReloadSettings,
Stop,
}
#[derive(Debug)]
pub enum QueueEventStatus {
Completed,
Locked,
Deferred,
}
#[derive(Debug)]
pub enum ReportingEvent {
Dmarc(Box<DmarcEvent>),
Tls(Box<TlsEvent>),
Stop,
}
#[derive(Debug)]
pub struct DmarcEvent {
pub domain: String,
pub report_record: Record,
pub dmarc_record: Arc<Dmarc>,
pub interval: AggregateFrequency,
pub span_id: u64,
}
#[derive(Debug)]
pub struct TlsEvent {
pub domain: String,
pub policy: PolicyType,
pub failure: Option<FailureDetails>,
pub tls_record: Arc<TlsRpt>,
pub interval: AggregateFrequency,
pub span_id: u64,
}
#[derive(Debug, Hash, PartialEq, Eq)]
pub enum PolicyType {
Tlsa(Option<Arc<Tlsa>>),
Sts(Option<Arc<Policy>>),
None,
}
pub struct TrainTaskController {
semaphore: Semaphore,
stop_flag: AtomicBool,
}
impl Default for TrainTaskController {
fn default() -> Self {
Self {
semaphore: Semaphore::new(1),
stop_flag: AtomicBool::new(false),
}
}
}
impl TrainTaskController {
pub fn try_run(&self) -> Option<SemaphorePermit<'_>> {
let permit = self.semaphore.try_acquire().ok()?;
self.stop_flag.store(false, Ordering::SeqCst);
Some(permit)
}
pub fn is_running(&self) -> bool {
self.semaphore.available_permits() == 0
}
pub fn stop(&self) {
self.stop_flag.store(true, Ordering::SeqCst);
}
pub fn should_stop(&self) -> bool {
self.stop_flag.load(Ordering::SeqCst)
}
}
impl BroadcastEvent {
pub fn reload(object: ObjectType) -> Self {
BroadcastEvent::RegistryChange(RegistryChange::Reload(object))
}
}
pub trait ToHash {
fn to_hash(&self) -> u64;
}
impl ToHash for Dmarc {
fn to_hash(&self) -> u64 {
RandomState::with_seeds(1, 9, 7, 9).hash_one(self)
}
}
impl ToHash for PolicyType {
fn to_hash(&self) -> u64 {
RandomState::with_seeds(1, 9, 7, 9).hash_one(self)
}
}
impl From<DmarcEvent> for ReportingEvent {
fn from(value: DmarcEvent) -> Self {
ReportingEvent::Dmarc(Box::new(value))
}
}
impl From<TlsEvent> for ReportingEvent {
fn from(value: TlsEvent) -> Self {
ReportingEvent::Tls(Box::new(value))
}
}
impl From<Arc<Tlsa>> for PolicyType {
fn from(value: Arc<Tlsa>) -> Self {
PolicyType::Tlsa(Some(value))
}
}
impl From<Arc<Policy>> for PolicyType {
fn from(value: Arc<Policy>) -> Self {
PolicyType::Sts(Some(value))
}
}
impl From<&Arc<Tlsa>> for PolicyType {
fn from(value: &Arc<Tlsa>) -> Self {
PolicyType::Tlsa(Some(value.clone()))
}
}
impl From<&Arc<Policy>> for PolicyType {
fn from(value: &Arc<Policy>) -> Self {
PolicyType::Sts(Some(value.clone()))
}
}
impl From<(&Option<Arc<Policy>>, &Option<Arc<Tlsa>>)> for PolicyType {
fn from(value: (&Option<Arc<Policy>>, &Option<Arc<Tlsa>>)) -> Self {
match value {
(Some(value), _) => PolicyType::Sts(Some(value.clone())),
(_, Some(value)) => PolicyType::Tlsa(Some(value.clone())),
_ => PolicyType::None,
}
}
}
impl PushNotification {
pub fn account_id(&self) -> u32 {
match self {
PushNotification::StateChange(state_change) => state_change.account_id,
PushNotification::CalendarAlert(calendar_alert) => calendar_alert.account_id,
PushNotification::EmailPush(email_push) => email_push.account_id,
}
}
pub fn filter_types(&self, types: &Bitmap<DataType>) -> Option<PushNotification> {
match self {
PushNotification::StateChange(state_change) => {
let mut filtered_types = state_change.types;
filtered_types.intersection(types);
if !filtered_types.is_empty() {
Some(PushNotification::StateChange(StateChange {
account_id: state_change.account_id,
change_id: state_change.change_id,
types: filtered_types,
}))
} else {
None
}
}
PushNotification::CalendarAlert(_) => {
if types.contains(DataType::CalendarAlert) {
Some(self.clone())
} else {
None
}
}
PushNotification::EmailPush(_) => {
if types.contains_any(
[
DataType::EmailDelivery,
DataType::Email,
DataType::Mailbox,
DataType::Thread,
]
.into_iter(),
) {
Some(self.clone())
} else {
None
}
}
}
}
}
impl EmailPush {
pub fn to_state_change(&self) -> StateChange {
StateChange {
account_id: self.account_id,
change_id: self.change_id,
types: Bitmap::from_iter([
DataType::EmailDelivery,
DataType::Email,
DataType::Mailbox,
DataType::Thread,
]),
}
}
}

Some files were not shown because too many files have changed in this diff Show More