Compare commits

..
Author SHA1 Message Date
jcoffey-dev d9754c46a6 Release 2026.9.30
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
publish / version (push) Skipped
publish / publish-amd64 (push) Skipped
publish / publish-arm64 (push) Skipped
publish / release (push) Skipped
publish / binaries (push) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 11m5s
github/ci (tag) GitHub Actions
publish / github (push) Failing after 49m30s
publish / announce (push) Skipped
2026-09-30 12:04:16 -07:00
jcoffey-dev 4481279f1c Merge pull request 'x:Metric: say which node wrote each sample' (#133) from fix/metric-node-id into main
ci / fork-checks (push) Skipped
ci / build (push) Skipped
github/ci (branch) GitHub Actions
ci / github (push) Successful in 45m8s
Reviewed-on: #133
2026-09-30 18:56:20 +00:00
jcoffey-dev 20abf69d31 x:Metric: say which node wrote each sample
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 7m5s
Each node stores histograms as running totals since it started. A sample
didn't say which node wrote it (the node was only in the id's low bits),
so a reader couldn't diff totals per node, and the console diffed across
nodes: on the three-node production cluster the delivery attempt time
read 14.7 s over the last hour against 0.7 s from the nodes' own figures.

x:Metric/get now returns nodeId alongside timestamp, both from the id.
The telemetry suite checks every sample carries it.
2026-09-30 11:43:13 -07:00
jcoffey-dev 69ef48239a Merge pull request 'ci: copy each release image to GHCR as a replica' (#132) from ci/ghcr-replica into main
ci / fork-checks (push) Skipped
ci / build (push) Skipped
github/ci (branch) GitHub Actions
ci / github (push) Successful in 44m28s
2026-09-30 16:34:53 +00:00
jcoffey-dev 00f00d6d75 ci: copy each release image to GHCR as a replica
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 6m27s
The Gitea registry stays authoritative; GHCR becomes a copy of it, the way
the GitHub repository is a copy of the Gitea one. After the tag build has
pushed the release image to the registry, a new ghcr job copies it to
ghcr.io under the same version tag and :latest with `imagetools create` --
a copy, not a rebuild, so the digest on GHCR is the digest on the registry.

Anything still pulling the old ghcr.io name, including the TrueNAS app
submission, keeps receiving releases. The job uses the run's own token and is
left out of the status reported to Gitea, so a GHCR problem cannot fail a
release.
2026-09-30 09:27:55 -07:00
jcoffey-dev 68d3ad795e Merge pull request 'ci: copy each release to GitHub after the tag build' (#131) from ci/github-release-copy into main
ci / fork-checks (push) Skipped
ci / build (push) Skipped
github/ci (branch) GitHub Actions
ci / github (push) Successful in 50m7s
2026-09-30 13:59:22 +00:00
jcoffey-dev d486747c11 Merge pull request 'ci: drop the build cache from tag image builds' (#130) from ci/tag-path-hardening into main
ci / fork-checks (push) Skipped
ci / build (push) Skipped
github/ci (branch) GitHub Actions
ci / github (push) Canceled after 6m56s
2026-09-30 13:52:24 +00:00
jcoffey-dev e69df1ae8d ci: copy each release to GitHub after the tag build
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 6m47s
The mirror carries tags to GitHub but not releases, so the replica's
Releases page -- and anyone watching the repository there -- stopped at the
last release made on GitHub. After the tag build has published, a new
github-release job copies the tag's Gitea release to a GitHub release: the
same notes, with PR and issue numbers rewritten to Gitea links, the same
files, and a line pointing back to the Gitea release.

It uses the run's own token and is left out of the status reported to
Gitea, so it cannot fail a release. With no Gitea release for the tag it
does nothing.
2026-09-30 06:52:24 -07:00
jcoffey-dev 031d028ba4 ci: drop the build cache from tag image builds
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 7m28s
GitHub scopes a run's Actions cache to its ref, so the cache a tag build
wrote could only ever be read by that same tag: the next release built cold
anyway. Each release also parked several GB of Rust layers in the
repository's 10 GB cache, enough to evict main's cargo cache and slow
everyday builds too. The image builds now run without a cache.
2026-09-30 06:44:41 -07:00
jcoffey-dev 6d7afc3c06 Merge pull request 'ci: run the github wait job on its own runner label' (#129) from ci/wait-runner into main
ci / fork-checks (push) Skipped
ci / build (push) Skipped
github/ci (branch) GitHub Actions
ci / github (push) Successful in 47m28s
2026-09-30 07:44:00 +00:00
jcoffey-dev 3d5a1692ab Merge pull request 'docs: point issues and discussions at Gitea and the forum' (#127) from docs/mirror-note into main
ci / build (push) Skipped
ci / fork-checks (push) Skipped
github/ci (branch) GitHub Actions
ci / github (push) Canceled after 33s
2026-09-30 07:43:27 +00:00
jcoffey-dev 1de77316f0 ci: run the github wait job on its own runner label
ci / build (pull_request) Skipped
ci / fork-checks (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 7m30s
The github job only polls Gitea for GitHub's commit status, but it holds a
runner slot for as long as the GitHub build takes -- the better part of an
hour for a cold build. On the shared build runners a handful of those
could take every slot and stall real work, so it now runs on the `wait`
label: a runner of its own, with many slots, no docker socket and a small
CPU and memory cap.
2026-09-30 00:36:20 -07:00
jcoffey-dev 26c7c6a897 Merge pull request 'ci: a cancelled GitHub run no longer reports failure to Gitea' (#128) from fix/ci-report-cancelled into main
ci / fork-checks (push) Skipped
ci / build (push) Skipped
github/ci (branch) GitHub Actions
ci / github (push) Canceled after 9m46s
2026-09-30 07:33:39 +00:00
jcoffey-dev 4ba1896eb1 ci: a cancelled GitHub run no longer reports failure to Gitea
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 6m5s
The mirror can push one commit twice in quick succession. GitHub then
starts two runs and cancels the older, and that run's report job posted
"failure" for the commit. Gitea's github job, seeing the newest status,
failed the check while the surviving run was still building and later
passed.

A cancelled run now posts nothing and leaves the result to the run that
superseded it. A real failure still reports failure.
2026-09-30 00:26:48 -07:00
jcoffey-dev b0e53ef966 docs: point issues and discussions at Gitea and the forum
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 26m44s
This repository is now push-mirrored to GitHub, where issues and pull
requests would never reach the maintainers. A note under the title says
where development happens, and sends issues to git.coffeylabs.org and
discussions to community.coffeylabs.org.
2026-09-30 00:16:15 -07:00
jcoffey-dev dd57709522 Merge pull request 'ci: build on GitHub via the mirror, switchable with BUILD_ON' (#126) from ci/build-on-github into main
ci / fork-checks (push) Successful in 1m36s
ci / github (push) Skipped
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
ci / github (pull_request) Canceled after 5m7s
ci / build (push) Canceled after 39m39s
github/ci (branch) GitHub Actions
Reviewed-on: #126
2026-09-30 06:52:16 +00:00
jcoffey-dev 3450c31345 Build on the GitHub mirror when BUILD_ON=github
ci / build (pull_request) Successful in 7m46s
ci / github (pull_request) Skipped
ci / fork-checks (pull_request) Successful in 2m4s
Gitea stays where the project lives and push-mirrors every branch and tag
to GitHub. With the Actions variable BUILD_ON set to 'github' on both
forges, the GitHub copy does the building and reports back to Gitea as a
commit status; unset, nothing changes and Gitea builds as before.

.github/workflows/ci.yml replaces the GitHub-era files. Branch pushes run
what Gitea's ci.yml checks (fork checks, dev build, test targets, the
release profile on main). v* tags run what publish.yml does, with the same
two guards: the image per architecture on native runners side by side,
the multi-arch index and :latest, the Gitea Release if the tag has none,
and the host-install binaries taken out of the image. A final job posts
"github/ci (branch)" or "github/ci (tag)" to the commit on Gitea.

On Gitea, the heavy jobs skip under BUILD_ON=github and a `github` job
waits for that status and passes or fails with it, so pull requests and
merges still look at a Gitea run. The weekly release, the upstream watch
and the announcement stay on Gitea.

Removed: cleanup.yml and publish.yml (GHCR), release.yml (a second weekly
schedule), and dependabot.yml, whose pull request branches every mirror
sync would delete.
2026-09-29 23:06:52 -07:00
13 changed files with 584 additions and 597 deletions
+44 -1
View File
@@ -7,7 +7,12 @@
# instance resolves short `uses:` against itself, never GitHub, so nothing # instance resolves short `uses:` against itself, never GitHub, so nothing
# unreviewed can be pulled in. # unreviewed can be pulled in.
# #
# Not ported, as on GitLab: publish.yml and release.yml still need doing. # BUILD_ON: when the Actions variable BUILD_ON is 'github' (org or repo),
# fork-checks and build skip here and the `github` job below waits for the
# same work done by .github/workflows/ci.yml on the GitHub mirror, passing or
# failing with it -- so this run still carries the answer pull requests and
# merges look at. Unset, everything builds here as before. If GitHub is
# unavailable, unset BUILD_ON and nothing else has to change.
name: ci name: ci
on: on:
@@ -25,6 +30,7 @@ jobs:
# without the AGPL 5(a) notice. Seconds, and needs no toolchain. The notice # without the AGPL 5(a) notice. Seconds, and needs no toolchain. The notice
# check diffs against the upstream snapshot branch, hence the full fetch. # check diffs against the upstream snapshot branch, hence the full fetch.
fork-checks: fork-checks:
if: ${{ vars.BUILD_ON != 'github' }}
runs-on: light runs-on: light
container: container:
image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim
@@ -52,6 +58,7 @@ jobs:
run: python3 -m unittest discover -s tools/fork/tests run: python3 -m unittest discover -s tools/fork/tests
build: build:
if: ${{ vars.BUILD_ON != 'github' }}
# Either runner (host1 or host2): the build needs no docker socket. # Either runner (host1 or host2): the build needs no docker socket.
runs-on: light runs-on: light
container: container:
@@ -101,3 +108,39 @@ jobs:
used=$(du -s --block-size=1G /cache/target 2>/dev/null | cut -f1) used=$(du -s --block-size=1G /cache/target 2>/dev/null | cut -f1)
echo "target dir: ${used:-0} GB" echo "target dir: ${used:-0} GB"
if [ "${used:-0}" -gt 60 ]; then rm -rf /cache/target && echo "over 60 GB: target dir cleared"; fi if [ "${used:-0}" -gt 60 ]; then rm -rf /cache/target && echo "over 60 GB: target dir cleared"; fi
# BUILD_ON=github: the GitHub mirror builds this commit and posts the result
# back as the commit status "github/ci (branch)". This waits for that status
# and takes its answer. The mirror pushes on every commit, so a missing
# status means GitHub has not got the push or is not running: after the
# timeout this fails, which is the cue to unset BUILD_ON.
github:
if: ${{ vars.BUILD_ON == 'github' }}
# Its own runner label with plenty of slots: this job only polls, but holds a slot
# for as long as the GitHub build takes, and must not starve the build runners.
runs-on: wait
timeout-minutes: 150
container:
image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim
steps:
- env:
TOKEN: ${{ secrets.GITHUB_TOKEN }}
SHA: ${{ github.event.pull_request.head.sha || github.sha }}
CONTEXT: github/ci (branch)
run: |
python3 - <<'EOF'
import json, os, time, urllib.request
url = (f"{os.environ['CI_SERVER_INTERNAL']}/api/v1/repos/{os.environ['GITHUB_REPOSITORY']}"
f"/commits/{os.environ['SHA']}/statuses?limit=50")
req = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['TOKEN']}"})
ctx, last = os.environ["CONTEXT"], None
print(f"waiting for '{ctx}' on {os.environ['SHA']}", flush=True)
while True:
mine = [s for s in json.load(urllib.request.urlopen(req)) if s["context"] == ctx]
state = max(mine, key=lambda s: s["id"]) if mine else None
if state and state["status"] != last:
last = state["status"]; print(f"{ctx}: {last} {state.get('target_url', '')}", flush=True)
if last == "success": raise SystemExit(0)
if last in ("failure", "error"): raise SystemExit(1)
time.sleep(20)
EOF
+54 -1
View File
@@ -42,6 +42,14 @@
# #
# The push logs in with PACKAGE_TOKEN (jcoffey-dev, write:package): the job's # The push logs in with PACKAGE_TOKEN (jcoffey-dev, write:package): the job's
# own token is refused by the container registry. # own token is refused by the container registry.
#
# BUILD_ON: when the Actions variable BUILD_ON is 'github' (org or repo), every
# job here but the announcement skips, and the tag is published by
# .github/workflows/ci.yml on the GitHub mirror instead -- same guards, same
# tags, the same Release and binaries, created here through the API. The
# `github` job waits for that run's commit status, "github/ci (tag)", and the
# announcement follows it as it follows the binaries here. Unset, everything
# runs here as before.
name: publish name: publish
on: on:
@@ -50,6 +58,7 @@ on:
jobs: jobs:
version: version:
if: ${{ vars.BUILD_ON != 'github' }}
runs-on: light runs-on: light
container: container:
image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim
@@ -88,6 +97,7 @@ jobs:
echo "version $V" echo "version $V"
publish-amd64: publish-amd64:
if: ${{ vars.BUILD_ON != 'github' }}
needs: [version] needs: [version]
runs-on: docker runs-on: docker
container: container:
@@ -128,6 +138,7 @@ jobs:
run: docker logout "$REGISTRY" || true run: docker logout "$REGISTRY" || true
publish-arm64: publish-arm64:
if: ${{ vars.BUILD_ON != 'github' }}
needs: [version, publish-amd64] needs: [version, publish-amd64]
runs-on: docker runs-on: docker
container: container:
@@ -162,11 +173,46 @@ jobs:
- if: always() - if: always()
run: docker logout "$REGISTRY" || true run: docker logout "$REGISTRY" || true
# BUILD_ON=github: waits for the GitHub mirror's run for this tag, which
# posts its result back as the commit status "github/ci (tag)", and takes
# its answer. Fails after the timeout if no answer comes.
github:
if: ${{ vars.BUILD_ON == 'github' }}
# Its own runner label with plenty of slots: this job only polls, but holds a slot
# for as long as the GitHub build takes, and must not starve the build runners.
runs-on: wait
timeout-minutes: 240
container:
image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim
steps:
- env:
TOKEN: ${{ secrets.GITHUB_TOKEN }}
SHA: ${{ github.sha }}
CONTEXT: github/ci (tag)
run: |
python3 - <<'EOF'
import json, os, time, urllib.request
url = (f"{os.environ['CI_SERVER_INTERNAL']}/api/v1/repos/{os.environ['GITHUB_REPOSITORY']}"
f"/commits/{os.environ['SHA']}/statuses?limit=50")
req = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['TOKEN']}"})
ctx, last = os.environ["CONTEXT"], None
print(f"waiting for '{ctx}' on {os.environ['SHA']}", flush=True)
while True:
mine = [s for s in json.load(urllib.request.urlopen(req)) if s["context"] == ctx]
state = max(mine, key=lambda s: s["id"]) if mine else None
if state and state["status"] != last:
last = state["status"]; print(f"{ctx}: {last} {state.get('target_url', '')}", flush=True)
if last == "success": raise SystemExit(0)
if last in ("failure", "error"): raise SystemExit(1)
time.sleep(20)
EOF
# The weekly release creates its Release (and so the tag) first; a tag # The weekly release creates its Release (and so the tag) first; a tag
# pushed by hand has none. Either way the tag ends up with exactly one # pushed by hand has none. Either way the tag ends up with exactly one
# Release, created once the amd64 image exists so its pull instructions # Release, created once the amd64 image exists so its pull instructions
# work; arm64 and the binaries follow. # work; arm64 and the binaries follow.
release: release:
if: ${{ vars.BUILD_ON != 'github' }}
needs: [version, publish-amd64] needs: [version, publish-amd64]
runs-on: light runs-on: light
container: container:
@@ -216,6 +262,7 @@ jobs:
# `docker create` does not start anything, so pulling an arm64 image on an # `docker create` does not start anything, so pulling an arm64 image on an
# amd64 runner and copying a file out of it needs no emulation. # amd64 runner and copying a file out of it needs no emulation.
binaries: binaries:
if: ${{ vars.BUILD_ON != 'github' }}
needs: [version, publish-arm64, release] needs: [version, publish-arm64, release]
runs-on: docker runs-on: docker
container: container:
@@ -289,8 +336,14 @@ jobs:
# The release above is made with the job's own token, and Gitea starts no # The release above is made with the job's own token, and Gitea starts no
# workflow for events the Actions bot causes -- announce.yml's # workflow for events the Actions bot causes -- announce.yml's
# 'on: release' never fires for it -- so announce it from here. # 'on: release' never fires for it -- so announce it from here.
#
# With BUILD_ON=github the release and binaries come from the GitHub run,
# so the announcement waits for the `github` job instead. The Release that
# run creates for a hand-pushed tag is made with a user token, so
# announce.yml fires for it too; discourse-release keeps one topic per tag.
announce: announce:
needs: [release, binaries] needs: [release, binaries, github]
if: ${{ always() && ((needs.release.result == 'success' && needs.binaries.result == 'success') || needs.github.result == 'success') }}
runs-on: light runs-on: light
steps: steps:
- uses: coffey-labs/actions/discourse-release@e9293996e2efa770839121fa8f8da93083f216be - uses: coffey-labs/actions/discourse-release@e9293996e2efa770839121fa8f8da93083f216be
-42
View File
@@ -1,42 +0,0 @@
version: 2
updates:
# Cargo. One entry: the workspace has a single lockfile at the root, and
# ~30 manifests that upstream bumps on every release -- pointing entries at
# individual crates would find manifests with no lockfile beside them.
#
# Minor and patch arrive as one pull request a week. Majors are left out of
# the group on purpose: they are migrations rather than bumps, and each one
# deserves its own pull request and its own CI run.
- package-ecosystem: cargo
directory: "/"
schedule:
interval: weekly
day: tuesday
time: "09:00"
timezone: Etc/UTC
open-pull-requests-limit: 5
groups:
minor-and-patch:
update-types:
- minor
- patch
- package-ecosystem: github-actions
directory: "/"
schedule:
interval: weekly
day: tuesday
time: "09:00"
timezone: Etc/UTC
groups:
actions:
patterns:
- "*"
# The Dockerfiles pin their base images, so this is what keeps a published
# image off a stale base between releases.
- package-ecosystem: docker
directory: "/"
schedule:
interval: weekly
day: tuesday
time: "09:00"
timezone: Etc/UTC
+453 -38
View File
@@ -1,51 +1,466 @@
# What CI can check without a mail server's worth of infrastructure. # CI and publishing on GitHub, for the repository Gitea mirrors here.
# #
# The build, and that every test target compiles. It deliberately does not # Gitea (git.coffeylabs.org) is where this project lives: pull requests,
# *run* the test suites: the unit tests only build with the integration crate # issues, releases and the container registry are all there, and it pushes
# in the graph, because that is what switches on the `test_mode` features they # every branch and tag to this GitHub copy as it changes. GitHub's hosted
# rely on (docs/spec/SPEC.md 2.2b), and the integration suites need a `STORE`, # runners are faster than the self-hosted ones -- and have native arm64 -- so
# fixed ports, and in most cases a container apiece (docs/spec/ # the building happens here, and the answer goes back to Gitea as a commit
# container-tests.md). Running them here would mean either a green tick that # status that Gitea's own ci.yml / publish.yml wait on.
# skipped everything, or a red one that means "the runner has no Redis".
# #
# So this catches what it can honestly catch -- code that does not compile, # One switch decides which side builds: the Actions variable BUILD_ON, set on
# including test code -- and the suites are run by hand, one at a time, as # both forges. BUILD_ON=github runs every job below and turns Gitea's heavy
# that page describes. If that changes, it changes because someone made the # jobs into a wait for this one; anything else leaves Gitea building exactly
# suites runnable unattended, not because CI started ignoring failures. # as before and every job here skips. If GitHub is ever unavailable, unset it
name: CI # on Gitea and nothing else has to change.
#
# Needs, as organization settings rather than anything in this file:
# variables BUILD_ON=github, REGISTRY (the Gitea container registry),
# GITEA_URL (the Gitea base URL)
# secret GITEA_TOKEN -- jcoffey-dev, write:repository + write:package:
# commit statuses, the release and its assets, the registry push
#
# There is no pull_request trigger: pull requests happen on Gitea, and their
# branch arrives here as an ordinary push. Branch pushes get what Gitea's
# ci.yml checks; v* tags get what its publish.yml does. Schedules (the weekly
# release, the upstream watch) and the release announcement stay on Gitea.
#
# Every `uses:` is pinned to a full commit SHA with the release in the
# trailing comment. A tag is a mutable pointer; do not "simplify" a pin back
# to one. Only GitHub's own actions and the three docker/* ones are used.
name: ci
on: on:
push: push:
branches: [main] branches: ['**']
pull_request: tags: ['**']
# Lets CI be run by hand against any ref, including one that predates a CI
# change, without pushing an empty commit to move it.
workflow_dispatch: workflow_dispatch:
# A second push to a branch cancels the run still going for the first: the # A newer push to a branch cancels the run for the older one, whose answer is
# older run's answer is about code nobody is looking at any more. # about code nobody is looking at any more. A tag run is never cancelled: it
# publishes.
concurrency: concurrency:
group: ci-${{ github.ref }} group: ci-${{ github.ref }}
cancel-in-progress: true cancel-in-progress: ${{ github.ref_type == 'branch' }}
permissions:
contents: read
env:
GITEA_URL: ${{ vars.GITEA_URL }}
# The Gitea status this run answers for. Gitea waits on the one matching
# its own event: "(branch)" from ci.yml, "(tag)" from publish.yml.
STATUS_CONTEXT: github/ci (${{ github.ref_type }})
jobs: jobs:
build: # Tells Gitea a run has started, so a pull request shows it as pending
# rather than missing while the build is still going.
start:
if: ${{ vars.BUILD_ON == 'github' }}
runs-on: ubuntu-latest
steps:
- env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
run: |
jq -n --arg c "$STATUS_CONTEXT" \
--arg u "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" \
'{state:"pending", context:$c, target_url:$u, description:"GitHub Actions"}' |
curl -fsS -o /dev/null -X POST -H "Authorization: token $GITEA_TOKEN" \
-H 'Content-Type: application/json' --data @- \
"$GITEA_URL/api/v1/repos/$GITHUB_REPOSITORY/statuses/$GITHUB_SHA"
# ----------------------------------------------------------- branches ------
# What an upstream merge can bring in or leave behind without a conflict:
# the upstream name in a new string literal, and a changed upstream file
# without the AGPL 5(a) notice. Seconds, and needs no toolchain. The notice
# check diffs against the upstream snapshot in the history, hence the full
# fetch.
fork-checks:
if: ${{ vars.BUILD_ON == 'github' && github.ref_type == 'branch' }}
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
# Every `uses:` here is pinned to a full commit SHA, with the release it
# belongs to in the trailing comment. A tag is a mutable pointer, so
# trusting `@v7` is trusting every future version of that action,
# including one pushed by whoever compromises the account. Dependabot
# updates both halves together -- do not "simplify" a pin back to a tag.
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2 with:
- name: System dependencies fetch-depth: 0
# foundationdb and the search backends are off by default, but the - run: python3 tools/fork/name-check.py
# default feature set still links against the system's C libraries. - if: always()
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends clang run: python3 tools/fork/notice-check.py
- name: Build the server # Cargo can patch a dependency to a directory in this repository, and
run: cargo build -p inbuxa --locked # the image builds from a context .dockerignore prunes to almost
- name: Compile every test target # nothing. CI never sees the difference; a release does.
# `--no-run` is the point: it builds the unit tests and the integration - if: always()
# crate together, which is the combination that resolves the test run: python3 tools/fork/context-check.py
# features, and stops short of running anything that wants a store. # The personal-data catalog must classify every object and field the
run: cargo test --workspace --locked --no-run # schema has, and name nothing that is gone.
- if: always()
run: python3 tools/fork/privacy-check.py
# The admin reads each expression field's allowed values and variables
# from the schema; they're generated from the registry and must match it.
- if: always()
run: python3 tools/fork/expr-schema.py --check
- if: always()
run: python3 -m unittest discover -s tools/fork/tests
# The build, and that every test target compiles. The suites are not run:
# they need a store, fixed ports and containers (docs/spec/
# container-tests.md), and are run by hand.
build:
if: ${{ vars.BUILD_ON == 'github' && github.ref_type == 'branch' }}
runs-on: ubuntu-latest
env:
CARGO_INCREMENTAL: "0"
# Debug info is most of a dev target dir, and nothing here runs a
# debugger. Without it the dev and test builds fit the runner's disk and
# the cache below stays small enough to be worth restoring.
CARGO_PROFILE_DEV_DEBUG: "0"
CARGO_PROFILE_TEST_DEBUG: "0"
steps:
# The hosted image carries toolchains this build never touches; a dev,
# test and release build of RocksDB and the workspace needs the room.
- run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
df -h /
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
# Current stable, as Gitea's rust:1 image is.
- id: rust
run: |
rustup toolchain install stable --profile minimal
rustup default stable
echo "version=$(rustc -V | cut -d' ' -f2)" >> "$GITHUB_OUTPUT"
- run: sudo apt-get update -qq && sudo apt-get install -y -qq --no-install-recommends clang >/dev/null
# Cargo's download cache and the dev/test target dir, keyed on the
# lockfile and the compiler. Saved from main only, so the one cache
# every branch restores is main's, and branches cannot evict it.
- uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
~/.cargo/registry/index
~/.cargo/registry/cache
~/.cargo/git/db
target/debug
key: cargo-${{ steps.rust.outputs.version }}-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-${{ steps.rust.outputs.version }}-
- run: cargo build -p inbuxa --locked
# --no-run: compiles every test target without running them, which
# catches a test that no longer builds without needing a store.
- run: cargo test --workspace --locked --no-run
- if: github.ref == 'refs/heads/main'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
~/.cargo/registry/index
~/.cargo/registry/cache
~/.cargo/git/db
target/debug
key: cargo-${{ steps.rust.outputs.version }}-${{ hashFiles('Cargo.lock') }}
# The release profile, on main only. It is the profile the image is
# built with, and it fails in ways the dev profile does not: v2026.9.24
# was tagged on a commit whose CI was green and whose release build
# could not compile the scim crate at all.
- if: github.ref == 'refs/heads/main'
run: cargo build -p inbuxa --locked --release
# --------------------------------------------------------------- tags ------
# Two guards before anything is pushed, the same as Gitea's publish.yml:
# * the tag must be v<brand_version!>. The version is a string in
# crates/types/src/branding.rs, not Cargo.toml, and the image is tagged
# with it, so a tag beside an unbumped macro would publish an image that
# reports a different version from its tag.
# * the tag must be on main or on a release/* branch, so an image never
# describes code that was never reviewed onto one of them. A release/*
# branch carries a hotfix cut from an earlier release tag.
version:
if: ${{ vars.BUILD_ON == 'github' && github.ref_type == 'tag' && startsWith(github.ref_name, 'v') }}
runs-on: ubuntu-latest
outputs:
version: ${{ steps.v.outputs.version }}
steps:
# Full history, and every branch as origin/*: the ancestry check cannot
# be answered from a shallow clone.
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
- id: v
env:
TAG: ${{ github.ref_name }}
run: |
set -euo pipefail
# Scoped to the macro body: branding.rs holds other string literals,
# and tagging an image from one of those would be worse than failing.
V="$(awk '/macro_rules! brand_version /,/^}/' crates/types/src/branding.rs \
| grep -om1 '"[0-9][^"]*"' | tr -d '"')"
[ -n "$V" ] || { echo "could not read brand_version! from branding.rs" >&2; exit 1; }
if [ "$TAG" != "v$V" ]; then
echo "Tag $TAG names a commit whose brand_version! says $V." >&2
echo "Refusing to publish an image that would report the wrong version." >&2
exit 1
fi
commit="$(git rev-parse "${TAG}^{commit}")"
on=""
for ref in origin/main $(git for-each-ref --format='%(refname:short)' 'refs/remotes/origin/release/*'); do
if git merge-base --is-ancestor "$commit" "$ref"; then on="$ref"; break; fi
done
[ -n "$on" ] || { echo "$TAG is not on main or a release/* branch" >&2; exit 1; }
echo "$TAG is on $on"
echo "version=$V" >> "$GITHUB_OUTPUT"
# Each architecture on its own native runner, side by side. The Dockerfile
# cross-compiles from the build platform, and on the self-hosted runners one
# machine built both one after the other; here two machines build at once,
# each natively (the builder stage picks the matching target, and the
# aarch64 toolchain it installs exists on arm64 too), and the small final
# stage needs no QEMU. amd64 also moves :<version> as soon as it is done, so
# a production deploy can start from it; :latest waits for the index below,
# so it never names an image without arm64.
publish:
needs: [version]
runs-on: ${{ matrix.runner }}
strategy:
fail-fast: false
matrix:
include:
- arch: amd64
runner: ubuntu-latest
- arch: arm64
runner: ubuntu-24.04-arm
env:
VERSION: ${{ needs.version.outputs.version }}
steps:
- run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
echo "IMAGE=${{ vars.REGISTRY }}/${GITHUB_REPOSITORY,,}" >> "$GITHUB_ENV"
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ${{ vars.REGISTRY }}
username: jcoffey-dev
password: ${{ secrets.GITEA_TOKEN }}
# Attestations off: they add manifests of their own, and the index
# should hold the two images and nothing else. No build cache: GitHub
# scopes a tag run's cache to that tag, so the next release could never
# read it, and each one would park several GB in the repository's 10 GB
# cache and evict main's cargo cache.
- uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
with:
context: .
platforms: linux/${{ matrix.arch }}
provenance: false
sbom: false
push: true
tags: |
${{ env.IMAGE }}:${{ env.VERSION }}-${{ matrix.arch }}
${{ matrix.arch == 'amd64' && format('{0}:{1}', env.IMAGE, env.VERSION) || '' }}
# Joins the two per-architecture tags into :<version> and :latest. Built
# from the per-architecture tags rather than :<version>, which by now is
# the amd64 image and would be read as such.
index:
needs: [version, publish]
runs-on: ubuntu-latest
env:
VERSION: ${{ needs.version.outputs.version }}
steps:
- run: echo "IMAGE=${{ vars.REGISTRY }}/${GITHUB_REPOSITORY,,}" >> "$GITHUB_ENV"
- uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ${{ vars.REGISTRY }}
username: jcoffey-dev
password: ${{ secrets.GITEA_TOKEN }}
- run: |
docker buildx imagetools create \
--tag "$IMAGE:$VERSION" \
--tag "$IMAGE:latest" \
"$IMAGE:$VERSION-amd64" "$IMAGE:$VERSION-arm64"
docker buildx imagetools inspect "$IMAGE:$VERSION"
# Gitea keeps a container package on its owner; linking it shows it on
# the repository's Packages tab. Idempotent.
- env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
run: |
owner="${GITHUB_REPOSITORY%%/*}"; name="${GITHUB_REPOSITORY#*/}"
curl -fsS -o /dev/null -X POST -H "Authorization: token $GITEA_TOKEN" \
"$GITEA_URL/api/v1/packages/${owner,,}/container/$name/-/link/$name" \
|| echo "package already linked (or link refused); not fatal"
# The weekly release creates its Release (and so the tag) on Gitea first; a
# tag pushed by hand has none. Either way the tag ends up with exactly one
# Release there, created once the image exists so its pull instructions
# work.
release:
needs: [version, index]
runs-on: ubuntu-latest
steps:
- env:
TAG: ${{ github.ref_name }}
VERSION: ${{ needs.version.outputs.version }}
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
REGISTRY: ${{ vars.REGISTRY }}
run: |
set -euo pipefail
api="$GITEA_URL/api/v1/repos/$GITHUB_REPOSITORY"
code="$(curl -sS -o /dev/null -w '%{http_code}' -H "Authorization: token $GITEA_TOKEN" "$api/releases/tags/$TAG")"
if [ "$code" = 200 ]; then echo "$TAG already has a release"; exit 0; fi
[ "$code" = 404 ] || { echo "looking up the release for $TAG answered $code" >&2; exit 1; }
image="$REGISTRY/${GITHUB_REPOSITORY,,}:$VERSION"
body="Container image: \`$image\` (linux/amd64, linux/arm64); also \`:latest\`.
Binaries for a host install are attached: \`inbuxa-linux-amd64.tar.gz\` and \`inbuxa-linux-arm64.tar.gz\`, with \`SHA256SUMS\`. Each is the binary out of this release's image for that architecture, so it is the same build. The image grants it \`cap_net_bind_service\`; a host install has to grant that itself (\`setcap\`, or \`AmbientCapabilities\` in the unit) to bind port 25."
jq -n --arg tag "$TAG" --arg name "INBUXA $VERSION" --arg body "$body" \
'{tag_name:$tag, name:$name, body:$body}' |
curl -fsS -X POST -H "Authorization: token $GITEA_TOKEN" -H 'Content-Type: application/json' \
--data @- "$api/releases" | jq -r '"created release " + .tag_name'
# The binaries for a host install, taken out of the image that was just
# pushed rather than compiled again: the binary in the tarball is the file
# the image runs. `docker create` starts nothing, so copying a file out of
# the arm64 image on an amd64 runner needs no emulation.
binaries:
needs: [version, index, release]
runs-on: ubuntu-latest
env:
VERSION: ${{ needs.version.outputs.version }}
TAG: ${{ github.ref_name }}
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
steps:
- run: echo "IMAGE=${{ vars.REGISTRY }}/${GITHUB_REPOSITORY,,}" >> "$GITHUB_ENV"
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ${{ vars.REGISTRY }}
username: jcoffey-dev
password: ${{ secrets.GITEA_TOKEN }}
- name: take the binaries out of the image
run: |
set -euo pipefail
mkdir -p out && cd out
for arch in amd64 arm64; do
docker pull -q --platform "linux/$arch" "$IMAGE:$VERSION"
id="$(docker create --platform "linux/$arch" "$IMAGE:$VERSION")"
docker cp "$id:/usr/local/bin/inbuxa" inbuxa
docker rm -f "$id" >/dev/null
chmod 0755 inbuxa
tar -czf "inbuxa-linux-$arch.tar.gz" inbuxa
rm inbuxa
done
sha256sum inbuxa-linux-*.tar.gz > SHA256SUMS
cat SHA256SUMS
# A re-run of a tag replaces its assets rather than leaving two files
# with the same name and different contents.
- name: attach them to the release
run: |
set -euo pipefail
api="$GITEA_URL/api/v1/repos/$GITHUB_REPOSITORY"
auth="Authorization: token $GITEA_TOKEN"
rel="$(curl -fsS -H "$auth" "$api/releases/tags/$TAG" | jq -r .id)"
assets="$(curl -fsS -H "$auth" "$api/releases/$rel/assets")"
for f in out/inbuxa-linux-amd64.tar.gz out/inbuxa-linux-arm64.tar.gz out/SHA256SUMS; do
name="$(basename "$f")"
old="$(jq -r --arg n "$name" '.[] | select(.name == $n) | .id' <<<"$assets")"
for id in $old; do curl -fsS -o /dev/null -X DELETE -H "$auth" "$api/releases/$rel/assets/$id"; done
curl -fsS -o /dev/null -X POST -H "$auth" -F "attachment=@$f" "$api/releases/$rel/assets?name=$name"
echo "attached $name"
done
# ------------------------------------------------------ ghcr replica ------
# Copies the release image from the Gitea registry, which stays the
# authoritative one, to ghcr.io under the same version tag and :latest. It is
# a copy, not a second build: the digest on GHCR is the digest on the
# registry, so `docker pull ghcr.io/...` gets exactly the same image. Left
# out of the report to Gitea, like the release copy, so a GHCR problem
# cannot fail a release.
ghcr:
if: ${{ vars.BUILD_ON == 'github' && github.ref_type == 'tag' }}
needs: [version, index]
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ needs.version.outputs.version }}
run: |
set -euo pipefail
src="${{ vars.REGISTRY }}/${GITHUB_REPOSITORY,,}"
dst="ghcr.io/${GITHUB_REPOSITORY,,}"
tag="$TAG"
echo "$GH_TOKEN" | docker login ghcr.io -u "$GITHUB_ACTOR" --password-stdin
docker buildx imagetools create -t "$dst:$tag" -t "$dst:latest" "$src:$tag"
want="$(docker buildx imagetools inspect "$src:$tag" --format '{{json .Manifest.Digest}}')"
got="$(docker buildx imagetools inspect "$dst:$tag" --format '{{json .Manifest.Digest}}')"
echo "registry $src:$tag = $want"
echo "ghcr $dst:$tag = $got"
[ "$want" = "$got" ] || echo "::warning::GHCR digest differs from the registry's"
docker logout ghcr.io
# ---------------------------------------------------- github release ------
# Copies this tag's Gitea release -- notes and files -- to a GitHub release,
# so the replica's Releases page, and anyone watching it, keeps up. Gitea's
# release is the real one; this is left out of the report to Gitea, so a
# failure here cannot fail a release. PR and issue numbers in the notes are
# rewritten to Gitea links: on GitHub a bare #16 is some other PR.
github-release:
if: ${{ vars.BUILD_ON == 'github' && github.ref_type == 'tag' }}
needs: [binaries]
runs-on: ubuntu-latest
permissions:
contents: write
env:
GITEA_URL: ${{ vars.GITEA_URL }}
GH_TOKEN: ${{ github.token }}
TAG: ${{ github.ref_name }}
steps:
- run: |
set -euo pipefail
if gh release view "$TAG" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then
echo "GitHub already has a release for $TAG"; exit 0
fi
# The Gitea release exists by now if this run made it; if the weekly
# release job made it, it came before the tag. Allow a few minutes.
code=0
for _ in $(seq 1 15); do
code="$(curl -sS -o rel.json -w '%{http_code}' "$GITEA_URL/api/v1/repos/$GITHUB_REPOSITORY/releases/tags/$TAG")"
[ "$code" = 200 ] && break
sleep 20
done
if [ "$code" != 200 ]; then echo "No Gitea release for $TAG; nothing to copy"; exit 0; fi
if [ "$(jq -r .draft rel.json)" = true ]; then echo "The Gitea release is a draft; not copying"; exit 0; fi
export BASE="$(jq -r '.html_url | sub("/releases/tag/.*$"; "")' rel.json)"
jq -r '.body // ""' rel.json | perl -pe 's{(?<![\w/&\[])#(\d+)\b}{[#$1]($ENV{BASE}/pulls/$1)}g' > notes.md
printf '\n\n_Mirrored from [the Gitea release](%s); report issues on [Gitea](%s/issues)._\n' \
"$(jq -r .html_url rel.json)" "$BASE" >> notes.md
files=()
mkdir -p files
while IFS=$'\t' read -r name url; do
curl -fsSL -o "files/$name" "$url"; files+=("files/$name")
done < <(jq -r '.assets[]? | [.name, .browser_download_url] | @tsv' rel.json)
title="$(jq -r '.name // ""' rel.json)"; [ -n "$title" ] || title="$TAG"
if [ "$(jq -r .prerelease rel.json)" = true ]; then kind=--prerelease; else kind=--latest; fi
gh release create "$TAG" --repo "$GITHUB_REPOSITORY" --verify-tag --title "$title" \
--notes-file notes.md "$kind" "${files[@]}"
echo "created the GitHub release for $TAG with ${#files[@]} file(s)"
# ------------------------------------------------------------- report ------
# One commit status on Gitea for the whole run: what Gitea's ci.yml and
# publish.yml wait on. Skipped jobs (the tag jobs on a branch, and the other
# way round) count as passing; a failed or cancelled one does not.
report:
if: ${{ always() && vars.BUILD_ON == 'github' }}
needs: [start, fork-checks, build, version, publish, index, release, binaries]
runs-on: ubuntu-latest
steps:
- env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
STATE: ${{ contains(needs.*.result, 'failure') && 'failure' || (contains(needs.*.result, 'cancelled') && 'cancelled' || 'success') }}
run: |
# A cancelled run was superseded by a newer run for the same commit (the
# mirror can push one commit twice); that run reports. Posting "failure"
# here would fail the Gitea check while the real build is still going.
if [ "$STATE" = cancelled ]; then echo "cancelled: leaving the result to the newer run"; exit 0; fi
jq -n --arg s "$STATE" --arg c "$STATUS_CONTEXT" \
--arg u "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" \
'{state:$s, context:$c, target_url:$u, description:"GitHub Actions"}' |
curl -fsS -o /dev/null -X POST -H "Authorization: token $GITEA_TOKEN" \
-H 'Content-Type: application/json' --data @- \
"$GITEA_URL/api/v1/repos/$GITHUB_REPOSITORY/statuses/$GITHUB_SHA"
echo "$STATUS_CONTEXT: $STATE"
-69
View File
@@ -1,69 +0,0 @@
# Prune old image versions from GHCR.
#
# Releases are kept forever -- they carry no assets and their generated notes
# are this project's only changelog, so deleting one destroys history that
# cannot be reconstructed for nothing saved. Images are the opposite: a
# multi-arch build a week, and the by-digest push in publish.yml leaves two
# untagged per-architecture manifests behind each time on top of the tagged
# index. Those accumulate and nobody wants fifty of them.
#
# THE FOOTGUN: the obvious tool for this -- delete-package-versions with
# `delete-only-untagged-versions` -- will happily delete the per-architecture
# manifests that a multi-arch tag points *at*, because they are untagged by
# design. Nothing appears to break: the tag still exists, and pulls simply
# start failing for one architecture. This action understands manifest lists
# and will not orphan a retained index, and `validate` re-checks every
# multi-arch manifest against the registry afterwards.
#
# Separate from publish.yml, and dispatchable on its own, so `dry_run` can show
# exactly what would be deleted without rebuilding and re-pushing an image to
# find out.
name: Prune images
on:
workflow_call:
inputs:
dry_run:
type: boolean
default: false
workflow_dispatch:
inputs:
dry_run:
description: "List what would be deleted, delete nothing"
type: boolean
default: true
jobs:
prune:
runs-on: ubuntu-latest
permissions:
packages: write
steps:
# The only third-party action here that is not published by GitHub or
# Docker, and the one with the most to lose: it is handed
# `packages: write` and its whole job is deletion, so a ref repointed at
# something else -- by a compromise or a mistake upstream -- is a bad
# day. It was pinned to a commit long before the rest of them were.
- uses: dataaxiom/ghcr-cleanup-action@d52806a0dc70b430571a37da1fde39733ffd640f # v1.2.2
with:
owner: inbuxa
package: inbuxa-server
token: ${{ secrets.GITHUB_TOKEN }}
# Ten weekly releases is roughly a quarter of history, which is more
# than enough to roll back to and far less than the year's worth that
# would otherwise pile up. Older *releases* stay either way; this
# only removes the images.
keep-n-tagged: 10
# Belt and braces on top of the action's own manifest awareness:
# `latest` is never a candidate for deletion under any counting.
exclude-tags: latest
delete-untagged: true
# Sweeps the wreckage of a half-failed run: an index whose platform
# images did not all land, and referrers whose parent is gone.
delete-partial-images: true
delete-orphaned-images: true
# Checks every remaining multi-architecture manifest still resolves
# in the registry. This is the step that would catch the footgun
# above rather than leaving a reader to discover it on `docker pull`.
validate: true
dry-run: ${{ inputs.dry_run }}
-198
View File
@@ -1,198 +0,0 @@
# Publish the container image to GHCR.
#
# The README and the docs site have told people to run
# `ghcr.io/inbuxa/inbuxa-server:latest` for a long time, and nothing ever
# pushed it: `docker pull` answered `denied`, because the package did not
# exist. This is the workflow that makes those instructions true. It is also
# the prerequisite for the self-hosted app catalogs -- TrueNAS and Unraid
# both install by pulling an image and neither builds from source.
#
# FIRST RUN: a package GHCR creates for the first time is **private**, even in
# a public repository, and an anonymous `docker pull` will still answer
# `denied`. Nothing in a workflow can change that -- the visibility is set once
# by hand under the package's settings, and until it is, this looks like it
# worked while the docs stay just as wrong as before. Check with a logged-out
# pull, not with one from a machine that has credentials.
#
# Two architectures, each built on its own native runner rather than under
# QEMU. Emulated arm64 has to run `npm ci` and the Vite build through
# instruction translation, which takes tens of minutes and occasionally runs
# out of memory; `ubuntu-24.04-arm` is free for public repositories and does
# the same work at native speed. The cost is the by-digest dance below: each
# runner pushes an untagged image, and a final job joins the two digests into
# one multi-arch tag.
name: Publish image
on:
release:
types: [published]
# Callable, so release.yml can build the release it just cut. This is not a
# stylistic choice: a release created with GITHUB_TOKEN does **not** raise a
# `release` event -- GitHub refuses to let a token trigger another workflow,
# to stop a workflow looping on its own output. A scheduled job that cut a
# release and expected this file to notice would silently never publish. The
# alternatives are a personal access token kept as a secret, or calling the
# workflow directly. This is the one that needs no credential.
workflow_call:
inputs:
ref:
description: "Tag, branch or SHA to build"
required: true
type: string
tag_latest:
description: "Also move :latest to this build"
type: boolean
default: false
# Same reasoning as ci.yml's dispatch trigger: a run GitHub queues and then
# orphans can be neither rerun nor canceled, and this workflow otherwise
# only fires on a release -- which is not something to cut twice because a
# runner died. `ref` also allows publishing an image for a tag that predates
# this workflow, which is how the first one gets built.
workflow_dispatch:
inputs:
ref:
description: "Tag, branch or SHA to build"
required: true
default: main
tag_latest:
description: "Also move :latest to this build"
type: boolean
default: false
env:
# Hardcoded rather than derived from github.repository, which would have to
# be lowercased to be a legal registry path. This is the string the docs name.
IMAGE: ghcr.io/inbuxa/inbuxa-server
jobs:
# The version is read once and handed to both builds, so the two
# architectures cannot disagree about what they are. It is read from the
# macro the binary itself compiles in, which the weekly release commits
# before this runs -- so the image is tagged with the version it reports.
version:
runs-on: ubuntu-latest
outputs:
version: ${{ steps.v.outputs.version }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ inputs.ref || github.ref }}
- id: v
run: |
set -euo pipefail
# Scoped to the macro body: branding.rs holds other string literals,
# and tagging an image from one of those would be worse than failing.
V="$(awk '/macro_rules! brand_version/,/^}/' crates/types/src/branding.rs \
| grep -om1 '"[0-9][^"]*"' | tr -d '"')"
[ -n "$V" ] || { echo "could not read brand_version! from branding.rs" >&2; exit 1; }
# A date version carries nothing a Docker tag objects to, so there is
# no second, sanitized form of it here.
echo "version=$V" >> "$GITHUB_OUTPUT"
echo "version $V"
build:
needs: version
runs-on: ${{ matrix.runner }}
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
include:
- platform: linux/amd64
runner: ubuntu-latest
- platform: linux/arm64
runner: ubuntu-24.04-arm
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ inputs.ref || github.ref }}
- uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push by digest
id: push
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
with:
context: .
platforms: ${{ matrix.platform }}
# Attestations are off deliberately: they add manifests of their own
# to the index, and `imagetools create` below expects the two entries
# it pushed rather than four.
provenance: false
sbom: false
cache-from: type=gha,scope=${{ matrix.platform }}
cache-to: type=gha,mode=max,scope=${{ matrix.platform }}
outputs: type=image,name=${{ env.IMAGE }},push-by-digest=true,name-canonical=true,push=true
- name: Save the digest
run: |
mkdir -p /tmp/digests
# The prefix is stripped here and put back in the merge job, so the
# filename is the bare hash. Leaving it on produces
# `image@sha256:sha256:...` when the reference is rebuilt.
digest="${{ steps.push.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
# One artifact per platform; the merge job globs them back together.
name: digest-${{ strategy.job-index }}
path: /tmp/digests/*
retention-days: 1
if-no-files-found: error
# Joins the per-architecture digests into a single tagged manifest, so
# `docker pull ghcr.io/inbuxa/inbuxa-server:<tag>` resolves on both.
publish:
needs: [version, build]
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
path: /tmp/digests
pattern: digest-*
merge-multiple: true
- uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Create the manifest
run: |
# Arrays rather than a string: the tags and the digest references
# have to reach docker as separate arguments, and building them by
# word-splitting an unquoted variable is the version of this that
# breaks the day a value contains a space.
tags=(-t "${IMAGE}:${{ needs.version.outputs.version }}")
# :latest follows real releases only. A prerelease that moved it
# would hand every `:latest` deployment an unfinished build, and a
# dispatch run has to ask for it on purpose.
if [ "${{ github.event_name }}" = "release" ] && [ "${{ github.event.release.prerelease }}" = "false" ]; then
tags+=(-t "${IMAGE}:latest")
elif [ "${{ inputs.tag_latest }}" = "true" ]; then
tags+=(-t "${IMAGE}:latest")
fi
refs=()
for f in /tmp/digests/*; do
refs+=("${IMAGE}@sha256:$(basename "$f")")
done
echo "tags: ${tags[*]}"
echo "refs: ${refs[*]}"
docker buildx imagetools create "${tags[@]}" "${refs[@]}"
- name: Show what landed
run: docker buildx imagetools inspect "${IMAGE}:${{ needs.version.outputs.version }}"
# Runs only after a successful publish, because that is the only moment the
# package grows. See cleanup.yml for why this is not the obvious one-liner.
prune:
needs: publish
permissions:
packages: write
uses: ./.github/workflows/cleanup.yml
-246
View File
@@ -1,246 +0,0 @@
# Cut a release once a week, but only if there is something in it.
#
# It does nothing on a quiet week. A release with no commits in it is worse
# than no release: it moves `:latest` to an identical build, spends a version
# number, and mails everybody watching the repository about nothing.
#
# INBUXA's version is a string in crates/types/src/branding.rs, deliberately
# not in Cargo.toml so that upstream's version bumps merge without conflicts.
# So this writes it: the bump is committed to main, and the tag names that
# commit. The tree a tag points at therefore reports the version the tag
# claims, which a tag placed beside an unbumped macro cannot promise.
name: Weekly release
on:
schedule:
# Mondays, 10:07 UTC, and last of the three: INBUXA Admin and the webmail
# release ahead of the server they talk to. Staggered rather than
# simultaneous so three releases do not compete for runners, and so a bad
# Monday names one repository instead of three. GitHub runs scheduled jobs
# best-effort and can delay a run considerably, so the exact minute is not
# a promise; the odd minute keeps it off the crowded top of the hour.
#
# Note also that GitHub disables scheduled workflows in a repository with
# no activity for 60 days, which is worth checking for before assuming
# this file is broken.
- cron: "7 10 * * 1"
workflow_dispatch:
inputs:
dry_run:
description: "Work out what would be released, then stop"
type: boolean
default: false
# One at a time. Two overlapping runs would race to write the same version and
# create the same tag, and the loser fails noisily for a reason that has
# nothing to do with the code.
concurrency:
group: weekly-release
cancel-in-progress: false
jobs:
check:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
should_release: ${{ steps.decide.outputs.should_release }}
version: ${{ steps.decide.outputs.version }}
tag: ${{ steps.decide.outputs.tag }}
previous: ${{ steps.decide.outputs.previous }}
count: ${{ steps.decide.outputs.count }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: main
fetch-depth: 0
- id: decide
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
# The newest published release, or empty on a repository that has
# never had one -- in which case everything counts as new. Drafts are
# excluded: an unpublished draft is not a release anybody has, so
# counting from it would hide commits that have never shipped.
previous="$(gh release list --limit 1 --exclude-drafts --json tagName --jq '.[0].tagName // ""')"
# A tag named by a release is normally present after a full checkout,
# but a release can outlive its tag. Falling back to the whole
# history is the safe direction to be wrong in: it over-counts, which
# cuts a release that was due anyway, where under-counting would skip
# one that was.
if [ -n "$previous" ] && git rev-parse -q --verify "refs/tags/${previous}" >/dev/null; then
count="$(git rev-list --count "${previous}..HEAD")"
else
count="$(git rev-list --count HEAD)"
fi
# INBUXA's version is the date: YYYY.M.D, unpadded, as branding.rs
# documents. A second release on one day takes a `.N` suffix,
# counting from 2, which is why this asks the tags rather than
# assuming today is free.
today="$(date -u +%Y.%-m.%-d)"
version="$today"
n=2
while git rev-parse -q --verify "refs/tags/v${version}" >/dev/null; do
version="${today}.${n}"
n=$((n + 1))
done
should_release=true
reason=""
if [ "$count" -eq 0 ]; then
should_release=false
reason="no commits since ${previous}"
fi
{
echo "should_release=$should_release"
echo "version=$version"
echo "tag=v${version}"
echo "previous=$previous"
echo "count=$count"
} >> "$GITHUB_OUTPUT"
# Written to the run summary so a skipped week reads as a decision
# rather than as a workflow that quietly did nothing.
{
echo "### Weekly release"
echo
if [ "$should_release" = "true" ]; then
echo "Releasing **v${version}** — ${count} commit(s) since ${previous:-the beginning}."
else
echo "Nothing to release: ${reason}."
fi
} >> "$GITHUB_STEP_SUMMARY"
cut:
needs: check
if: needs.check.outputs.should_release == 'true' && !inputs.dry_run
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
outputs:
sha: ${{ steps.land.outputs.sha }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: main
fetch-depth: 0
- id: bump
env:
VERSION: ${{ needs.check.outputs.version }}
BRANCH: release/v${{ needs.check.outputs.version }}
run: |
set -euo pipefail
# Scoped to the macro body rather than replacing the first quoted
# string in the file, and asserted to have matched exactly once.
# branding.rs holds other string literals, and a bump that silently
# edited one of those -- or none -- would ship a build whose version
# disagrees with its tag.
python3 - <<'PY'
import os, re
path = "crates/types/src/branding.rs"
src = open(path, encoding="utf-8").read()
pattern = re.compile(r'(macro_rules! brand_version \{\s*\(\) => \{\s*")[^"]+(")')
out, n = pattern.subn(lambda m: m.group(1) + os.environ["VERSION"] + m.group(2), src, count=1)
assert n == 1, f"brand_version! not found in {path}"
open(path, "w", encoding="utf-8").write(out)
PY
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add crates/types/src/branding.rs
git commit -m "Version ${VERSION}"
git push origin "HEAD:refs/heads/${BRANCH}"
# main is protected: it takes a pull request with a green build, and
# GITHUB_TOKEN is not among the bypass actors. So the bump lands the way
# every other change does. The alternative was to hand the release a
# credential that outranks the rule, which is a worse thing to own than
# a slower Monday.
- id: land
env:
VERSION: ${{ needs.check.outputs.version }}
BRANCH: release/v${{ needs.check.outputs.version }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
url="$(gh pr create --base main --head "${BRANCH}" \
--title "Version ${VERSION}" \
--body "Weekly release. Bumps \`brand_version!\` to ${VERSION} so the tag names a tree that reports the version the tag claims.")"
# The number, not the branch: the branch is deleted on merge, and a
# deleted branch no longer resolves to its pull request.
pr="${url##*/}"
echo "Opened #${pr}"
# The build is what the rule actually requires, and it is also the
# thing worth waiting for: a release cut from a tree that does not
# compile is the failure this whole arrangement exists to prevent.
# A full build of this tree is long, so the deadline is generous.
deadline=$(( SECONDS + 3600 ))
while :; do
state="$(gh pr view "${pr}" --json statusCheckRollup \
--jq '[.statusCheckRollup[]? | .conclusion // "PENDING"] | join(",")')"
case "${state}" in
*FAILURE*|*CANCELLED*|*TIMED_OUT*)
echo "::error::CI failed on ${BRANCH} (${state}); no release cut. PR #${pr} is left open."
exit 1 ;;
*SUCCESS*) break ;;
esac
if [ "${SECONDS}" -ge "${deadline}" ]; then
echo "::error::timed out waiting for CI on ${BRANCH}. PR #${pr} is left open."
exit 1
fi
sleep 30
done
gh pr merge "${pr}" --rebase --delete-branch
# A rebase merge rewrites the commit, so the sha to tag is the one
# GitHub recorded for the merge, not the tip that was pushed. It can
# take a moment to appear.
sha=""
for _ in $(seq 1 30); do
sha="$(gh pr view "${pr}" --json mergeCommit --jq '.mergeCommit.oid // ""')"
[ -n "${sha}" ] && break
sleep 5
done
if [ -z "${sha}" ]; then
echo "::error::#${pr} merged but GitHub reported no merge commit; nothing safe to tag."
exit 1
fi
echo "sha=${sha}" >> "$GITHUB_OUTPUT"
- env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
args=(--target "${{ steps.land.outputs.sha }}"
--title "INBUXA ${{ needs.check.outputs.version }}"
--generate-notes)
# Bound the notes to what is actually new. Without a start tag the
# generator reaches back to whatever it decides is previous, which on
# a repository carrying upstream's tag shapes is not always the last
# release.
if [ -n "${{ needs.check.outputs.previous }}" ]; then
args+=(--notes-start-tag "${{ needs.check.outputs.previous }}")
fi
gh release create "${{ needs.check.outputs.tag }}" "${args[@]}"
# Called rather than left to the `release` trigger on purpose: see the note
# at the top of publish.yml. A release created with GITHUB_TOKEN raises no
# event, so without this the tag would exist and no image would follow it.
publish:
needs: [check, cut]
permissions:
contents: read
packages: write
uses: ./.github/workflows/publish.yml
with:
ref: ${{ needs.cut.outputs.sha }}
tag_latest: true
+4
View File
@@ -8,6 +8,10 @@
--- ---
> [!NOTE]
> Development happens on [git.coffeylabs.org/inbuxa/inbuxa-server](https://git.coffeylabs.org/inbuxa/inbuxa-server); the copy on GitHub is a read-only mirror.
> Report issues at **[git.coffeylabs.org/inbuxa/inbuxa-server/issues](https://git.coffeylabs.org/inbuxa/inbuxa-server/issues)**, and join discussions at **[community.coffeylabs.org](https://community.coffeylabs.org)**.
**inbuxa** is a mail and collaboration server: JMAP, IMAP, POP3, SMTP, **inbuxa** is a mail and collaboration server: JMAP, IMAP, POP3, SMTP,
CalDAV, CardDAV and WebDAV, in one Rust binary, with ihasmail as its web front CalDAV, CardDAV and WebDAV, in one Rust binary, with ihasmail as its web front
end. It is a fork of [Stalwart](https://github.com/stalwartlabs/stalwart). end. It is a fork of [Stalwart](https://github.com/stalwartlabs/stalwart).
@@ -104,6 +104,12 @@ impl StoredMetric {
pub fn timestamp(&self) -> u64 { pub fn timestamp(&self) -> u64 {
SnowflakeIdGenerator::to_timestamp(self.id) SnowflakeIdGenerator::to_timestamp(self.id)
} }
/// The node that wrote the sample. Histogram totals are per node, so a
/// reader diffs them per node.
pub fn node_id(&self) -> u64 {
SnowflakeIdGenerator::to_node_id(self.id)
}
} }
/// What the node wrote last, so counters and histograms are written as /// What the node wrote last, so counters and histograms are written as
+5 -1
View File
@@ -6,7 +6,7 @@
//! `x:Metric/get` and `/query` over the stored history (monitoring spec, //! `x:Metric/get` and `/query` over the stored history (monitoring spec,
//! "Interfaces"). Samples are server-level (MON-31) and read-only (MON-32). //! "Interfaces"). Samples are server-level (MON-31) and read-only (MON-32).
//! A sample's `timestamp` comes from its id. //! A sample's `timestamp` and `nodeId` come from its id.
use crate::{ use crate::{
api::query::QueryResponseBuilder, api::query::QueryResponseBuilder,
@@ -54,12 +54,16 @@ fn metric_type(metric: &Metric) -> MetricType {
fn to_value(sample: StoredMetric) -> JmapValue<'static> { fn to_value(sample: StoredMetric) -> JmapValue<'static> {
let timestamp = sample.timestamp(); let timestamp = sample.timestamp();
let node_id = sample.node_id();
let mut value = sample.metric.into_value(); let mut value = sample.metric.into_value();
if let JmapValue::Object(obj) = &mut value { if let JmapValue::Object(obj) = &mut value {
obj.insert_unchecked( obj.insert_unchecked(
Property::Timestamp, Property::Timestamp,
JmapValue::Str(UTCDateTime::from_timestamp(timestamp as i64).to_string().into()), JmapValue::Str(UTCDateTime::from_timestamp(timestamp as i64).to_string().into()),
); );
// Histograms are running totals per node; without this a reader
// diffs one node's total against another's
obj.insert_unchecked(Property::NodeId, JmapValue::Number(node_id.into()));
} }
value value
} }
+1 -1
View File
@@ -81,7 +81,7 @@ fn legacy_setting(name: &str, is_set: impl Fn(&str) -> bool) -> Option<String> {
#[macro_export] #[macro_export]
macro_rules! brand_version { macro_rules! brand_version {
() => { () => {
"2026.9.29.2" "2026.9.30"
}; };
} }
+6
View File
@@ -108,6 +108,12 @@ impl SnowflakeIdGenerator {
(id >> (SEQUENCE_LEN + NODE_ID_LEN)) / 1000 + DEFAULT_EPOCH (id >> (SEQUENCE_LEN + NODE_ID_LEN)) / 1000 + DEFAULT_EPOCH
} }
// inbuxa: the node that made the id, so per-node history (metric
// totals) can be told apart
pub fn to_node_id(id: u64) -> u64 {
id & NODE_ID_MASK
}
#[inline(always)] #[inline(always)]
pub fn past_id(&self, period: Duration) -> Option<u64> { pub fn past_id(&self, period: Duration) -> Option<u64> {
self.epoch.elapsed().ok().map(|elapsed| { self.epoch.elapsed().ok().map(|elapsed| {
+11
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use crate::utils::server::TestServer; use crate::utils::server::TestServer;
@@ -53,6 +55,15 @@ pub async fn test(test: &TestServer) {
); );
assert_eq!(metrics.len(), metric_ids.len()); assert_eq!(metrics.len(), metric_ids.len());
// Every sample says which node wrote it, so histogram totals can be
// diffed per node
for metric in metrics {
assert!(
metric.get("nodeId").is_some_and(|v| v.is_u64()),
"Missing nodeId in {metric}"
);
}
// Fetch the last 48 hours of metrics // Fetch the last 48 hours of metrics
let metric_ids = admin let metric_ids = admin
.registry_query( .registry_query(