For the chart's scheduled backup and store checks (inbuxa-charts#11), and three bugs the lab's restore test found on the way. Added - `--export s3://bucket/prefix` streams the export into multipart uploads, in a new folder named for the time (20261119T204500Z), and writes export.json last, naming the files: an export without it didn't finish. Nothing the size of the store lands on local disk. Parts start at 16 MiB and double every thousand up to 512 MiB, so one file can be nearly 3 TiB; one part is held at a time. - `--import s3://bucket/prefix/<name>` reads one back, streamed; from `s3://bucket/prefix`, the newest finished one. - The bucket from INBUXA_EXPORT_S3_ENDPOINT (S3-compatible, path-style; AWS without it), INBUXA_EXPORT_S3_REGION, and the keys from INBUXA_EXPORT_S3_ACCESS_KEY_FILE and INBUXA_EXPORT_S3_SECRET_KEY_FILE (the AWS defaults without them). - `--store-check quick|full` runs a store check to its end and exits: 0 nothing wrong, 3 problems found, 2 another check running, 1 failed. It's saved with the console's other checks, under a node id no server holds. Fixed - A live export could name blobs it didn't have: the store's areas went out in a set's order, so the blobs could go first and a message stored after them pointed at a blob the export never got. Blobs go last now. - An import into a fresh store refused to run: every start writes the fork's subspace (where the blob store is, permission grants, the log files' setting), the import's own start too, and the import took that for data. Since the blob store change (#333): 27.1-rc.16 and 2026.11.9.1. The fork's subspace is no longer counted; its records are keyed values an import sets over. The store test now imports over a start-up record, and fails without this. - An import wrote the blobs into the data store, the empty store's blob store, even when the imported settings name another (S3, say): the restored server then found none of them. Blobs are imported last, into the blob store the imported x:BlobStore names. - The S3 export and import could stop for good on a pod limited to one CPU (one runtime worker): a reader or sender waiting on the transfer held the worker the transfer needed. A wait hands its worker over first. Tested: inbuxa-k8s-lab backup-test.sh, on the kind lab (Postgres, Garage, NATS, five server pods): mail for an account, the chart's backup CronJob, the database dropped and the blob bucket emptied, the import, and every mailbox's count and message hashes the same after; quick and full store checks clean. store tests pass on SQLite and RocksDB; unit tests pass.
519 lines
26 KiB
YAML
519 lines
26 KiB
YAML
# Publish the container image, ported from .github/workflows/publish.yml when
|
|
# the project moved to the self-hosted Gitea (2026-09-22). Starts on a v* tag,
|
|
# whether a person pushed it or weekly-release.yml created it through the
|
|
# releases API.
|
|
#
|
|
# The image is multi-arch (linux/amd64, linux/arm64), built by two jobs on
|
|
# the image-build runner rather than one buildx run for both. The Dockerfile's
|
|
# builder stage runs on the build platform and cross-compiles with an aarch64
|
|
# linker, so only the small final stage (apt, setcap) goes through QEMU for
|
|
# arm64 -- but two release builds (LTO, one codegen unit) side by side on one
|
|
# machine each take twice as long. Production runs amd64, so amd64 goes first
|
|
# and on its own:
|
|
# * publish-amd64 pushes :<version>-amd64 and :<version>, a plain amd64
|
|
# image, as soon as its build is done. A deploy can start from it.
|
|
# * publish-arm64 then builds arm64, pushes :<version>-arm64, and replaces
|
|
# :<version> with the two-platform index. The series tags (and :latest, for a
|
|
# release) move only here, so they never name an image without arm64.
|
|
#
|
|
# Both jobs use one BuildKit builder, `gitea-builder`, whose container
|
|
# (buildx_buildkit_gitea-builder0) and state volume stay on the runner's host
|
|
# between jobs: a job container's `buildx create` finds the existing container
|
|
# and reuses it and its cache. The dependency build (`cargo chef cook`) is
|
|
# keyed on the recipe, which only a dependency change alters, so a release
|
|
# normally compiles just the workspace. Removing that container or its volume
|
|
# costs the next release a cold build, nothing more. The planner and dependency
|
|
# layers for the build platform are shared, so arm64 also reuses what amd64
|
|
# just did where it can.
|
|
#
|
|
# Two guards before anything is pushed:
|
|
# * the tag must be v<version>, the workspace version in Cargo.toml as tags
|
|
# name it (tools/release/version.py: 27.1.0 is v27.1, 27.1.0-rc.1 is
|
|
# v27.1-rc.1). The server reports that same version, so a tag beside an
|
|
# unbumped Cargo.toml would publish an image that says otherwise.
|
|
# * the tag must be on main or on a release/* branch, so an image never
|
|
# describes code that was never reviewed onto one of them. A release/*
|
|
# branch carries a hotfix: it starts at an earlier release tag, takes
|
|
# fixes through pull requests into it, and is tagged there, so production
|
|
# can get a fix without everything that has landed on main since.
|
|
#
|
|
# Each version also moves its series' tags (tools/release/version.py
|
|
# --image-tags): a release 27.1.N moves :27.1 and :latest; a release candidate
|
|
# 27.1-rc.N moves :27.1-rc and never :latest, and its Release is a
|
|
# pre-release.
|
|
#
|
|
# The push logs in with PACKAGE_TOKEN (jcoffey-dev, write:package): the job's
|
|
# own token is refused by the container registry.
|
|
#
|
|
# inbuxa Relay (relay spec, RL-31) ships from the same tag: publish-relay
|
|
# builds its image (Dockerfile.relay) for both platforms, binaries takes its
|
|
# static binaries out too, and relay-sign signs them with the inbuxa release
|
|
# key, the secret RELAY_RELEASE_KEY, which only this forge holds
|
|
# (tools/release/new-relay-keys.sh makes it). relay-sign runs whichever
|
|
# forge built the release, once the binaries are on it. Without the secret
|
|
# it warns and signs nothing, and no relay takes those binaries as an
|
|
# upgrade (RL-29).
|
|
#
|
|
# BUILD_ON: when the Actions variable BUILD_ON is 'github' (org or repo), every
|
|
# job here but the announcement and relay-sign skips, and the tag is published by
|
|
# .github/workflows/ci.yml on the GitHub mirror instead -- same guards, same
|
|
# tags, the same Release and binaries, created here through the API. The
|
|
# `github` job waits for that run's commit status, "github/ci (tag)", and the
|
|
# announcement follows it as it follows the binaries here. Unset, everything
|
|
# runs here as before.
|
|
name: publish
|
|
|
|
on:
|
|
push:
|
|
tags: ['v*']
|
|
|
|
jobs:
|
|
version:
|
|
if: ${{ vars.BUILD_ON != 'github' }}
|
|
runs-on: light
|
|
container:
|
|
image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim
|
|
outputs:
|
|
version: ${{ steps.v.outputs.version }}
|
|
tags: ${{ steps.v.outputs.tags }}
|
|
prerelease: ${{ steps.v.outputs.prerelease }}
|
|
cargo: ${{ steps.v.outputs.cargo }}
|
|
steps:
|
|
# Full history: the ancestry check cannot be answered from a shallow
|
|
# clone. The checkout also fetches every branch as origin/*.
|
|
- uses: coffey-labs/actions/checkout@fab0c4d45e0162963965f1555df27b7bed5e20ec
|
|
with:
|
|
fetch-depth: 0
|
|
- id: v
|
|
shell: bash
|
|
env:
|
|
TAG: ${{ github.ref_name }}
|
|
run: |
|
|
set -euo pipefail
|
|
V="$(python3 tools/release/version.py --tag "$TAG")"
|
|
tags="$(python3 tools/release/version.py --image-tags | tr '\n' ' ')"
|
|
if python3 tools/release/version.py --prerelease; then pre=true; else pre=false; fi
|
|
commit="$(git rev-parse "${TAG}^{commit}")"
|
|
on=""
|
|
for ref in origin/main $(git for-each-ref --format='%(refname:short)' 'refs/remotes/origin/release/*'); do
|
|
if git merge-base --is-ancestor "$commit" "$ref"; then on="$ref"; break; fi
|
|
done
|
|
[ -n "$on" ] || { echo "$TAG is not on main or a release/* branch" >&2; exit 1; }
|
|
echo "$TAG is on $on"
|
|
echo "version=$V" >> "$GITHUB_OUTPUT"
|
|
echo "tags=${tags% }" >> "$GITHUB_OUTPUT"
|
|
echo "prerelease=$pre" >> "$GITHUB_OUTPUT"
|
|
echo "cargo=$(python3 tools/release/version.py --cargo)" >> "$GITHUB_OUTPUT"
|
|
echo "version $V; image tags: ${tags% }; prerelease: $pre"
|
|
|
|
publish-amd64:
|
|
if: ${{ vars.BUILD_ON != 'github' }}
|
|
needs: [version]
|
|
runs-on: docker
|
|
container:
|
|
image: docker:28-cli@sha256:625d9431a9f54c5a2bc90f24f0e1c3d55b1349fd857dd85035f98c2c9acbdd4d # 28-cli
|
|
volumes:
|
|
- /var/run/docker.sock:/var/run/docker.sock
|
|
env:
|
|
DOCKER_BUILDKIT: "1"
|
|
REGISTRY: ${{ vars.REGISTRY }}
|
|
IMAGE: ${{ vars.REGISTRY }}/${{ github.repository }}
|
|
VERSION: ${{ needs.version.outputs.version }}
|
|
PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
|
|
steps:
|
|
- uses: coffey-labs/actions/checkout@fab0c4d45e0162963965f1555df27b7bed5e20ec
|
|
- run: |
|
|
test -n "$REGISTRY" && test -n "$VERSION"
|
|
test -n "$PACKAGE_TOKEN" || { echo "PACKAGE_TOKEN secret is not set on this repository" >&2; exit 1; }
|
|
echo "$PACKAGE_TOKEN" | docker login -u jcoffey-dev --password-stdin "$REGISTRY"
|
|
docker buildx create --use --name gitea-builder --driver docker-container || docker buildx use gitea-builder
|
|
# Attestations off, as before: they add manifests of their own, and the
|
|
# index should hold the two images and nothing else.
|
|
- run: |
|
|
docker buildx build \
|
|
--platform linux/amd64 \
|
|
--provenance=false --sbom=false \
|
|
--build-arg INBUXA_REVISION="${{ github.sha }}" \
|
|
--tag "$IMAGE:$VERSION-amd64" \
|
|
--tag "$IMAGE:$VERSION" \
|
|
--push .
|
|
docker buildx imagetools inspect "$IMAGE:$VERSION"
|
|
# Gitea keeps a container package on its owner; linking it shows it on
|
|
# the repository's Packages tab. Idempotent.
|
|
- run: |
|
|
apk add --no-cache -q curl
|
|
curl -fsS -o /dev/null -X POST -H "Authorization: token $PACKAGE_TOKEN" \
|
|
"$CI_SERVER_INTERNAL/api/v1/packages/${GITHUB_REPOSITORY%%/*}/container/${GITHUB_REPOSITORY#*/}/-/link/${GITHUB_REPOSITORY#*/}" \
|
|
|| echo "package already linked (or link refused); not fatal"
|
|
- if: always()
|
|
run: docker logout "$REGISTRY" || true
|
|
|
|
publish-arm64:
|
|
if: ${{ vars.BUILD_ON != 'github' }}
|
|
needs: [version, publish-amd64]
|
|
runs-on: docker
|
|
container:
|
|
image: docker:28-cli@sha256:625d9431a9f54c5a2bc90f24f0e1c3d55b1349fd857dd85035f98c2c9acbdd4d # 28-cli
|
|
volumes:
|
|
- /var/run/docker.sock:/var/run/docker.sock
|
|
env:
|
|
DOCKER_BUILDKIT: "1"
|
|
REGISTRY: ${{ vars.REGISTRY }}
|
|
IMAGE: ${{ vars.REGISTRY }}/${{ github.repository }}
|
|
VERSION: ${{ needs.version.outputs.version }}
|
|
TAGS: ${{ needs.version.outputs.tags }}
|
|
PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
|
|
steps:
|
|
- uses: coffey-labs/actions/checkout@fab0c4d45e0162963965f1555df27b7bed5e20ec
|
|
- run: |
|
|
echo "$PACKAGE_TOKEN" | docker login -u jcoffey-dev --password-stdin "$REGISTRY"
|
|
docker run --privileged --rm tonistiigi/binfmt --install arm64
|
|
docker buildx create --use --name gitea-builder --driver docker-container || docker buildx use gitea-builder
|
|
# The index is built from the two per-architecture tags rather than from
|
|
# :<version>, which by now is the amd64 image and would be read as such.
|
|
- run: |
|
|
docker buildx build \
|
|
--platform linux/arm64 \
|
|
--provenance=false --sbom=false \
|
|
--build-arg INBUXA_REVISION="${{ github.sha }}" \
|
|
--tag "$IMAGE:$VERSION-arm64" \
|
|
--push .
|
|
args=""
|
|
for t in $TAGS; do args="$args --tag $IMAGE:$t"; done
|
|
docker buildx imagetools create $args \
|
|
"$IMAGE:$VERSION-amd64" "$IMAGE:$VERSION-arm64"
|
|
docker buildx imagetools inspect "$IMAGE:$VERSION"
|
|
- if: always()
|
|
run: docker logout "$REGISTRY" || true
|
|
|
|
# The relay's image, both platforms in one build: its builder isn't a
|
|
# cross build, so arm64 compiles under QEMU here, slowly; the relay is
|
|
# small. It runs after the server's arm64 build, not beside it. Before any
|
|
# tag moves, the amd64 image must say it is this version and hold no
|
|
# server binary (RL-T24).
|
|
publish-relay:
|
|
if: ${{ vars.BUILD_ON != 'github' }}
|
|
needs: [version, publish-arm64]
|
|
runs-on: docker
|
|
container:
|
|
image: docker:28-cli@sha256:625d9431a9f54c5a2bc90f24f0e1c3d55b1349fd857dd85035f98c2c9acbdd4d # 28-cli
|
|
volumes:
|
|
- /var/run/docker.sock:/var/run/docker.sock
|
|
env:
|
|
DOCKER_BUILDKIT: "1"
|
|
REGISTRY: ${{ vars.REGISTRY }}
|
|
RELAY_IMAGE: ${{ vars.REGISTRY }}/${{ github.repository_owner }}/inbuxa-relay
|
|
VERSION: ${{ needs.version.outputs.version }}
|
|
CARGO_VERSION: ${{ needs.version.outputs.cargo }}
|
|
TAGS: ${{ needs.version.outputs.tags }}
|
|
PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
|
|
steps:
|
|
- uses: coffey-labs/actions/checkout@fab0c4d45e0162963965f1555df27b7bed5e20ec
|
|
- run: |
|
|
echo "$PACKAGE_TOKEN" | docker login -u jcoffey-dev --password-stdin "$REGISTRY"
|
|
docker run --privileged --rm tonistiigi/binfmt --install arm64
|
|
docker buildx create --use --name gitea-builder --driver docker-container || docker buildx use gitea-builder
|
|
- run: |
|
|
set -eu
|
|
for arch in amd64 arm64; do
|
|
docker buildx build -f Dockerfile.relay \
|
|
--platform "linux/$arch" \
|
|
--provenance=false --sbom=false \
|
|
--tag "$RELAY_IMAGE:$VERSION-$arch" \
|
|
--push .
|
|
done
|
|
said="$(docker run --rm --platform linux/amd64 "$RELAY_IMAGE:$VERSION-amd64" --version)"
|
|
[ "$said" = "inbuxa-relay $CARGO_VERSION" ] || { echo "the relay image says '$said'" >&2; exit 1; }
|
|
id="$(docker create --platform linux/amd64 "$RELAY_IMAGE:$VERSION-amd64")"
|
|
if docker export "$id" | tar -t | grep -qE '(^|/)inbuxa$'; then
|
|
docker rm -f "$id" >/dev/null
|
|
echo "the relay image holds a server binary" >&2; exit 1
|
|
fi
|
|
docker rm -f "$id" >/dev/null
|
|
args=""
|
|
for t in $TAGS; do args="$args --tag $RELAY_IMAGE:$t"; done
|
|
docker buildx imagetools create $args \
|
|
"$RELAY_IMAGE:$VERSION-amd64" "$RELAY_IMAGE:$VERSION-arm64"
|
|
docker buildx imagetools inspect "$RELAY_IMAGE:$VERSION"
|
|
- run: |
|
|
apk add --no-cache -q curl
|
|
curl -fsS -o /dev/null -X POST -H "Authorization: token $PACKAGE_TOKEN" \
|
|
"$CI_SERVER_INTERNAL/api/v1/packages/${GITHUB_REPOSITORY%%/*}/container/inbuxa-relay/-/link/${GITHUB_REPOSITORY#*/}" \
|
|
|| echo "package already linked (or link refused); not fatal"
|
|
- if: always()
|
|
run: docker logout "$REGISTRY" || true
|
|
|
|
# BUILD_ON=github: waits for the GitHub mirror's run for this tag, which
|
|
# posts its result back as the commit status "github/ci (tag)", and takes
|
|
# its answer. Fails after the timeout if no answer comes.
|
|
github:
|
|
if: ${{ vars.BUILD_ON == 'github' }}
|
|
# Its own runner label with plenty of slots: this job only polls, but holds a slot
|
|
# for as long as the GitHub build takes, and must not starve the build runners.
|
|
runs-on: wait
|
|
timeout-minutes: 240
|
|
container:
|
|
image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim
|
|
steps:
|
|
- env:
|
|
TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
SHA: ${{ github.sha }}
|
|
CONTEXT: github/ci (tag)
|
|
run: |
|
|
python3 - <<'EOF'
|
|
import json, os, time, urllib.request
|
|
url = (f"{os.environ['CI_SERVER_INTERNAL']}/api/v1/repos/{os.environ['GITHUB_REPOSITORY']}"
|
|
f"/commits/{os.environ['SHA']}/statuses?limit=50")
|
|
req = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['TOKEN']}"})
|
|
ctx, last = os.environ["CONTEXT"], None
|
|
print(f"waiting for '{ctx}' on {os.environ['SHA']}", flush=True)
|
|
while True:
|
|
mine = [s for s in json.load(urllib.request.urlopen(req)) if s["context"] == ctx]
|
|
state = max(mine, key=lambda s: s["id"]) if mine else None
|
|
if state and state["status"] != last:
|
|
last = state["status"]; print(f"{ctx}: {last} {state.get('target_url', '')}", flush=True)
|
|
if last == "success": raise SystemExit(0)
|
|
if last in ("failure", "error"): raise SystemExit(1)
|
|
time.sleep(20)
|
|
EOF
|
|
|
|
# The weekly release creates its Release (and so the tag) first; a tag
|
|
# pushed by hand has none. Either way the tag ends up with exactly one
|
|
# Release, created once the amd64 image exists so its pull instructions
|
|
# work; arm64 and the binaries follow.
|
|
release:
|
|
if: ${{ vars.BUILD_ON != 'github' }}
|
|
needs: [version, publish-amd64]
|
|
runs-on: light
|
|
container:
|
|
image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim
|
|
steps:
|
|
- shell: bash
|
|
env:
|
|
TAG: ${{ github.ref_name }}
|
|
VERSION: ${{ needs.version.outputs.version }}
|
|
TAGS: ${{ needs.version.outputs.tags }}
|
|
PRERELEASE: ${{ needs.version.outputs.prerelease }}
|
|
TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
REPO: ${{ github.repository }}
|
|
REGISTRY: ${{ vars.REGISTRY }}
|
|
run: |
|
|
python3 - <<'PY'
|
|
import json, os, urllib.request, urllib.error
|
|
api = f"{os.environ['CI_SERVER_INTERNAL']}/api/v1/repos/{os.environ['REPO']}"
|
|
h = {"Authorization": f"token {os.environ['TOKEN']}", "Content-Type": "application/json"}
|
|
tag, version = os.environ["TAG"], os.environ["VERSION"]
|
|
try:
|
|
urllib.request.urlopen(urllib.request.Request(f"{api}/releases/tags/{tag}", headers=h))
|
|
print(f"{tag} already has a release"); raise SystemExit
|
|
except urllib.error.HTTPError as e:
|
|
if e.code != 404: raise
|
|
image = f"{os.environ['REGISTRY']}/{os.environ['REPO']}:{version}"
|
|
relay = f"{os.environ['REGISTRY']}/{os.environ['REPO'].split('/')[0]}/inbuxa-relay:{version}"
|
|
also = " ".join(f"`:{t}`" for t in os.environ["TAGS"].split()[1:])
|
|
pre = os.environ["PRERELEASE"] == "true"
|
|
body = (("Release candidate for the next series, to try before it ships. "
|
|
"`:latest` stays on the current release.\n\n" if pre else "")
|
|
+ f"Container image: `{image}` (linux/amd64, linux/arm64); also {also}. "
|
|
"amd64 is published first; arm64 is added to the same tag when its build "
|
|
"finishes, and the series tags move then.\n\n"
|
|
"Binaries for a host install are attached: `inbuxa-linux-amd64.tar.gz` and "
|
|
"`inbuxa-linux-arm64.tar.gz`, with `SHA256SUMS`. Each is the binary out of this "
|
|
"release's image for that architecture, so it is the same build. The image "
|
|
"grants it `cap_net_bind_service`; a host install has to grant that itself "
|
|
"(`setcap`, or `AmbientCapabilities` in the unit) to bind port 25.\n\n"
|
|
f"inbuxa Relay, the edge relay, ships from the same tag: image `{relay}` "
|
|
"(linux/amd64, linux/arm64), and static binaries `inbuxa-relay-x86_64-linux-musl` "
|
|
"and `inbuxa-relay-aarch64-linux-musl`, each with a `.sig` file by the inbuxa "
|
|
"release key. inbuxa-admin's Relays page takes a binary and its `.sig` to "
|
|
"upgrade a relay.")
|
|
data = json.dumps({"tag_name": tag, "name": f"INBUXA {version}", "body": body,
|
|
"prerelease": pre}).encode()
|
|
r = json.load(urllib.request.urlopen(urllib.request.Request(f"{api}/releases", data=data, headers=h)))
|
|
print(f"created release {r['tag_name']}")
|
|
PY
|
|
|
|
# The binaries for a host install, taken out of the image that was just
|
|
# pushed rather than compiled again.
|
|
#
|
|
# Building them separately would mean a second Rust build per architecture
|
|
# -- the slowest thing this pipeline does -- and would leave two artifacts
|
|
# that are supposed to be the same build but only probably are. Extracting
|
|
# them makes that identity a fact: the binary in the tarball is the file
|
|
# the image runs.
|
|
#
|
|
# `docker create` does not start anything, so pulling an arm64 image on an
|
|
# amd64 runner and copying a file out of it needs no emulation.
|
|
binaries:
|
|
if: ${{ vars.BUILD_ON != 'github' }}
|
|
needs: [version, publish-arm64, publish-relay, release]
|
|
runs-on: docker
|
|
container:
|
|
image: docker:28-cli@sha256:625d9431a9f54c5a2bc90f24f0e1c3d55b1349fd857dd85035f98c2c9acbdd4d # 28-cli
|
|
volumes:
|
|
- /var/run/docker.sock:/var/run/docker.sock
|
|
env:
|
|
REGISTRY: ${{ vars.REGISTRY }}
|
|
IMAGE: ${{ vars.REGISTRY }}/${{ github.repository }}
|
|
VERSION: ${{ needs.version.outputs.version }}
|
|
TAG: ${{ github.ref_name }}
|
|
REPO: ${{ github.repository }}
|
|
PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
|
|
TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
MINISIGN_SECRET_KEY: ${{ secrets.MINISIGN_SECRET_KEY }}
|
|
steps:
|
|
- name: take the binaries out of the image
|
|
run: |
|
|
set -euo pipefail
|
|
echo "$PACKAGE_TOKEN" | docker login -u jcoffey-dev --password-stdin "$REGISTRY"
|
|
mkdir -p /out && cd /out
|
|
for arch in amd64 arm64; do
|
|
docker pull -q --platform "linux/$arch" "$IMAGE:$VERSION"
|
|
id="$(docker create --platform "linux/$arch" "$IMAGE:$VERSION")"
|
|
docker cp "$id:/usr/local/bin/inbuxa" "inbuxa"
|
|
docker rm -f "$id" >/dev/null
|
|
chmod 0755 inbuxa
|
|
tar -czf "inbuxa-linux-$arch.tar.gz" inbuxa
|
|
rm inbuxa
|
|
done
|
|
# The relay's, as bare binaries: an upgrade (RL-29) takes the file itself
|
|
relay="$REGISTRY/${REPO%%/*}/inbuxa-relay:$VERSION"
|
|
for pair in amd64:x86_64 arm64:aarch64; do
|
|
arch="${pair%%:*}"; name="inbuxa-relay-${pair#*:}-linux-musl"
|
|
docker pull -q --platform "linux/$arch" "$relay"
|
|
id="$(docker create --platform "linux/$arch" "$relay")"
|
|
docker cp "$id:/usr/local/bin/inbuxa-relay" "$name"
|
|
docker rm -f "$id" >/dev/null
|
|
chmod 0755 "$name"
|
|
done
|
|
sha256sum inbuxa-linux-*.tar.gz inbuxa-relay-* > SHA256SUMS
|
|
cat SHA256SUMS
|
|
# Sign SHA256SUMS with the suite's minisign key, so a download can be
|
|
# verified against the public key the docs publish. This is separate
|
|
# from the relay .sig (relay-sign), which the server uses only to let a
|
|
# relay take a binary as an upgrade. No key (as on a fork, or before the
|
|
# key is added) leaves the release unsigned.
|
|
- name: sign SHA256SUMS with minisign
|
|
run: |
|
|
set -euo pipefail
|
|
if [ -z "${MINISIGN_SECRET_KEY:-}" ]; then
|
|
echo "::warning::MINISIGN_SECRET_KEY is not set; this release will be unsigned"
|
|
exit 0
|
|
fi
|
|
apk add --no-cache -q minisign
|
|
umask 077
|
|
key="$(mktemp)"; printf '%s\n' "$MINISIGN_SECRET_KEY" > "$key"
|
|
minisign -S -s "$key" -m /out/SHA256SUMS -t "inbuxa $TAG"
|
|
rm -f "$key"
|
|
echo "signed /out/SHA256SUMS -> /out/SHA256SUMS.minisig"
|
|
- name: attach them to the release
|
|
run: |
|
|
set -euo pipefail
|
|
apk add --no-cache -q python3
|
|
python3 - <<'PY'
|
|
import json, os, urllib.request, urllib.error, uuid, pathlib
|
|
api = f"{os.environ['CI_SERVER_INTERNAL']}/api/v1/repos/{os.environ['REPO']}"
|
|
tok = {"Authorization": f"token {os.environ['TOKEN']}"}
|
|
tag = os.environ["TAG"]
|
|
|
|
def get(path):
|
|
return json.load(urllib.request.urlopen(urllib.request.Request(api + path, headers=tok)))
|
|
|
|
rel = get(f"/releases/tags/{tag}")
|
|
assets = {a["name"]: a["id"] for a in get(f"/releases/{rel['id']}/assets")}
|
|
|
|
for path in ["/out/inbuxa-linux-amd64.tar.gz", "/out/inbuxa-linux-arm64.tar.gz",
|
|
"/out/inbuxa-relay-x86_64-linux-musl", "/out/inbuxa-relay-aarch64-linux-musl",
|
|
"/out/SHA256SUMS", "/out/SHA256SUMS.minisig"]:
|
|
if not pathlib.Path(path).exists():
|
|
continue # SHA256SUMS.minisig is absent only on an unsigned release
|
|
name = os.path.basename(path)
|
|
# A re-run of a tag replaces its assets rather than leaving two
|
|
# files with the same name and different contents.
|
|
if name in assets:
|
|
urllib.request.urlopen(urllib.request.Request(
|
|
f"{api}/releases/{rel['id']}/assets/{assets[name]}", headers=tok, method="DELETE"))
|
|
boundary = uuid.uuid4().hex
|
|
body = b"".join([
|
|
f"--{boundary}\r\nContent-Disposition: form-data; name=\"attachment\"; filename=\"{name}\"\r\n".encode(),
|
|
b"Content-Type: application/octet-stream\r\n\r\n",
|
|
pathlib.Path(path).read_bytes(),
|
|
f"\r\n--{boundary}--\r\n".encode(),
|
|
])
|
|
req = urllib.request.Request(
|
|
f"{api}/releases/{rel['id']}/assets?name={name}", data=body, method="POST",
|
|
headers={**tok, "Content-Type": f"multipart/form-data; boundary={boundary}"})
|
|
urllib.request.urlopen(req)
|
|
print("attached", name)
|
|
PY
|
|
- if: always()
|
|
run: docker logout "$REGISTRY" || true
|
|
|
|
# Signs the relay binaries on the release with the inbuxa release key
|
|
# (RL-29), whichever forge put them there: this forge's binaries job, or
|
|
# the GitHub run the `github` job waited for. Each .sig names the Cargo
|
|
# version the binary's --version prints and the target in its file name.
|
|
# The key is written to a file only for the signing, and removed.
|
|
relay-sign:
|
|
needs: [binaries, github]
|
|
if: ${{ always() && (needs.binaries.result == 'success' || needs.github.result == 'success') }}
|
|
runs-on: docker
|
|
container:
|
|
image: docker:28-cli@sha256:625d9431a9f54c5a2bc90f24f0e1c3d55b1349fd857dd85035f98c2c9acbdd4d # 28-cli
|
|
env:
|
|
TAG: ${{ github.ref_name }}
|
|
REPO: ${{ github.repository }}
|
|
TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
RELAY_RELEASE_KEY: ${{ secrets.RELAY_RELEASE_KEY }}
|
|
steps:
|
|
- uses: coffey-labs/actions/checkout@fab0c4d45e0162963965f1555df27b7bed5e20ec
|
|
- shell: sh
|
|
run: |
|
|
set -eu
|
|
if [ -z "$RELAY_RELEASE_KEY" ]; then
|
|
echo "::warning::RELAY_RELEASE_KEY is not set: the relay binaries stay unsigned, and no relay takes them as an upgrade"
|
|
exit 0
|
|
fi
|
|
apk add --no-cache -q bash coreutils curl jq openssl python3
|
|
version="$(python3 tools/release/version.py --cargo)"
|
|
api="$CI_SERVER_INTERNAL/api/v1/repos/$REPO"
|
|
auth="Authorization: token $TOKEN"
|
|
rel="$(curl -fsS -H "$auth" "$api/releases/tags/$TAG" | jq -r .id)"
|
|
assets="$(curl -fsS -H "$auth" "$api/releases/$rel/assets")"
|
|
umask 077
|
|
key="$(mktemp)"
|
|
trap 'rm -f "$key"' EXIT
|
|
printf '%s\n' "$RELAY_RELEASE_KEY" > "$key"
|
|
for target in x86_64-linux-musl aarch64-linux-musl; do
|
|
name="inbuxa-relay-$target"
|
|
id="$(echo "$assets" | jq -r --arg n "$name" '.[] | select(.name == $n) | .id' | head -n1)"
|
|
[ -n "$id" ] || { echo "the release has no $name" >&2; exit 1; }
|
|
curl -fsSL -H "$auth" -o "$name" "$CI_SERVER_INTERNAL/$REPO/releases/download/$TAG/$name"
|
|
bash tools/release/sign-relay.sh "$key" "$name" "$version" "$target"
|
|
for old in $(echo "$assets" | jq -r --arg n "$name.sig" '.[] | select(.name == $n) | .id'); do
|
|
curl -fsS -o /dev/null -X DELETE -H "$auth" "$api/releases/$rel/assets/$old"
|
|
done
|
|
curl -fsS -o /dev/null -X POST -H "$auth" -F "attachment=@$name.sig" "$api/releases/$rel/assets?name=$name.sig"
|
|
echo "signed $name ($version, $target)"
|
|
cat "$name.sig"
|
|
done
|
|
|
|
# The release above is made with the job's own token, and Gitea starts no
|
|
# workflow for events the Actions bot causes -- announce.yml's
|
|
# 'on: release' never fires for it -- so announce it from here.
|
|
#
|
|
# With BUILD_ON=github the release and binaries come from the GitHub run,
|
|
# so the announcement waits for the `github` job instead. The Release that
|
|
# run creates for a hand-pushed tag is made with a user token, so
|
|
# announce.yml fires for it too; discourse-release keeps one topic per tag.
|
|
announce:
|
|
needs: [release, binaries, github]
|
|
if: ${{ always() && ((needs.release.result == 'success' && needs.binaries.result == 'success') || needs.github.result == 'success') }}
|
|
runs-on: light
|
|
steps:
|
|
- uses: coffey-labs/actions/discourse-release@958b3a35a345744abd236ca5abf679cbedaca7a0
|
|
with:
|
|
api-key: ${{ secrets.DISCOURSE_RELEASE_KEY }}
|
|
discord-webhook: ${{ secrets.DISCORD_RELEASE_WEBHOOK }}
|
|
tag: ${{ github.ref_name }}
|