Files
inbuxa-server/.gitea/workflows/publish.yml
jcoffey-dev 4f2187b805
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 14m26s
Backups for Kubernetes: export to and import from S3, store checks as a command (K-C11)
For the chart's scheduled backup and store checks (inbuxa-charts#11), and
three bugs the lab's restore test found on the way.

Added
- `--export s3://bucket/prefix` streams the export into multipart
  uploads, in a new folder named for the time (20261119T204500Z), and
  writes export.json last, naming the files: an export without it didn't
  finish. Nothing the size of the store lands on local disk. Parts start
  at 16 MiB and double every thousand up to 512 MiB, so one file can be
  nearly 3 TiB; one part is held at a time.
- `--import s3://bucket/prefix/<name>` reads one back, streamed; from
  `s3://bucket/prefix`, the newest finished one.
- The bucket from INBUXA_EXPORT_S3_ENDPOINT (S3-compatible, path-style;
  AWS without it), INBUXA_EXPORT_S3_REGION, and the keys from
  INBUXA_EXPORT_S3_ACCESS_KEY_FILE and INBUXA_EXPORT_S3_SECRET_KEY_FILE
  (the AWS defaults without them).
- `--store-check quick|full` runs a store check to its end and exits: 0
  nothing wrong, 3 problems found, 2 another check running, 1 failed. It's
  saved with the console's other checks, under a node id no server holds.

Fixed
- A live export could name blobs it didn't have: the store's areas went
  out in a set's order, so the blobs could go first and a message stored
  after them pointed at a blob the export never got. Blobs go last now.
- An import into a fresh store refused to run: every start writes the
  fork's subspace (where the blob store is, permission grants, the log
  files' setting), the import's own start too, and the import took that
  for data. Since the blob store change (#333): 27.1-rc.16 and
  2026.11.9.1. The fork's subspace is no longer counted; its records are
  keyed values an import sets over. The store test now imports over a
  start-up record, and fails without this.
- An import wrote the blobs into the data store, the empty store's blob
  store, even when the imported settings name another (S3, say): the
  restored server then found none of them. Blobs are imported last, into
  the blob store the imported x:BlobStore names.
- The S3 export and import could stop for good on a pod limited to one CPU
  (one runtime worker): a reader or sender waiting on the transfer held
  the worker the transfer needed. A wait hands its worker over first.

Tested: inbuxa-k8s-lab backup-test.sh, on the kind lab (Postgres, Garage,
NATS, five server pods): mail for an account, the chart's backup CronJob,
the database dropped and the blob bucket emptied, the import, and every
mailbox's count and message hashes the same after; quick and full store
checks clean. store tests pass on SQLite and RocksDB; unit tests pass.
2026-11-19 13:47:43 -07:00

519 lines
26 KiB
YAML

# Publish the container image, ported from .github/workflows/publish.yml when
# the project moved to the self-hosted Gitea (2026-09-22). Starts on a v* tag,
# whether a person pushed it or weekly-release.yml created it through the
# releases API.
#
# The image is multi-arch (linux/amd64, linux/arm64), built by two jobs on
# the image-build runner rather than one buildx run for both. The Dockerfile's
# builder stage runs on the build platform and cross-compiles with an aarch64
# linker, so only the small final stage (apt, setcap) goes through QEMU for
# arm64 -- but two release builds (LTO, one codegen unit) side by side on one
# machine each take twice as long. Production runs amd64, so amd64 goes first
# and on its own:
# * publish-amd64 pushes :<version>-amd64 and :<version>, a plain amd64
# image, as soon as its build is done. A deploy can start from it.
# * publish-arm64 then builds arm64, pushes :<version>-arm64, and replaces
# :<version> with the two-platform index. The series tags (and :latest, for a
# release) move only here, so they never name an image without arm64.
#
# Both jobs use one BuildKit builder, `gitea-builder`, whose container
# (buildx_buildkit_gitea-builder0) and state volume stay on the runner's host
# between jobs: a job container's `buildx create` finds the existing container
# and reuses it and its cache. The dependency build (`cargo chef cook`) is
# keyed on the recipe, which only a dependency change alters, so a release
# normally compiles just the workspace. Removing that container or its volume
# costs the next release a cold build, nothing more. The planner and dependency
# layers for the build platform are shared, so arm64 also reuses what amd64
# just did where it can.
#
# Two guards before anything is pushed:
# * the tag must be v<version>, the workspace version in Cargo.toml as tags
# name it (tools/release/version.py: 27.1.0 is v27.1, 27.1.0-rc.1 is
# v27.1-rc.1). The server reports that same version, so a tag beside an
# unbumped Cargo.toml would publish an image that says otherwise.
# * the tag must be on main or on a release/* branch, so an image never
# describes code that was never reviewed onto one of them. A release/*
# branch carries a hotfix: it starts at an earlier release tag, takes
# fixes through pull requests into it, and is tagged there, so production
# can get a fix without everything that has landed on main since.
#
# Each version also moves its series' tags (tools/release/version.py
# --image-tags): a release 27.1.N moves :27.1 and :latest; a release candidate
# 27.1-rc.N moves :27.1-rc and never :latest, and its Release is a
# pre-release.
#
# The push logs in with PACKAGE_TOKEN (jcoffey-dev, write:package): the job's
# own token is refused by the container registry.
#
# inbuxa Relay (relay spec, RL-31) ships from the same tag: publish-relay
# builds its image (Dockerfile.relay) for both platforms, binaries takes its
# static binaries out too, and relay-sign signs them with the inbuxa release
# key, the secret RELAY_RELEASE_KEY, which only this forge holds
# (tools/release/new-relay-keys.sh makes it). relay-sign runs whichever
# forge built the release, once the binaries are on it. Without the secret
# it warns and signs nothing, and no relay takes those binaries as an
# upgrade (RL-29).
#
# BUILD_ON: when the Actions variable BUILD_ON is 'github' (org or repo), every
# job here but the announcement and relay-sign skips, and the tag is published by
# .github/workflows/ci.yml on the GitHub mirror instead -- same guards, same
# tags, the same Release and binaries, created here through the API. The
# `github` job waits for that run's commit status, "github/ci (tag)", and the
# announcement follows it as it follows the binaries here. Unset, everything
# runs here as before.
name: publish
on:
push:
tags: ['v*']
jobs:
version:
if: ${{ vars.BUILD_ON != 'github' }}
runs-on: light
container:
image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim
outputs:
version: ${{ steps.v.outputs.version }}
tags: ${{ steps.v.outputs.tags }}
prerelease: ${{ steps.v.outputs.prerelease }}
cargo: ${{ steps.v.outputs.cargo }}
steps:
# Full history: the ancestry check cannot be answered from a shallow
# clone. The checkout also fetches every branch as origin/*.
- uses: coffey-labs/actions/checkout@fab0c4d45e0162963965f1555df27b7bed5e20ec
with:
fetch-depth: 0
- id: v
shell: bash
env:
TAG: ${{ github.ref_name }}
run: |
set -euo pipefail
V="$(python3 tools/release/version.py --tag "$TAG")"
tags="$(python3 tools/release/version.py --image-tags | tr '\n' ' ')"
if python3 tools/release/version.py --prerelease; then pre=true; else pre=false; fi
commit="$(git rev-parse "${TAG}^{commit}")"
on=""
for ref in origin/main $(git for-each-ref --format='%(refname:short)' 'refs/remotes/origin/release/*'); do
if git merge-base --is-ancestor "$commit" "$ref"; then on="$ref"; break; fi
done
[ -n "$on" ] || { echo "$TAG is not on main or a release/* branch" >&2; exit 1; }
echo "$TAG is on $on"
echo "version=$V" >> "$GITHUB_OUTPUT"
echo "tags=${tags% }" >> "$GITHUB_OUTPUT"
echo "prerelease=$pre" >> "$GITHUB_OUTPUT"
echo "cargo=$(python3 tools/release/version.py --cargo)" >> "$GITHUB_OUTPUT"
echo "version $V; image tags: ${tags% }; prerelease: $pre"
publish-amd64:
if: ${{ vars.BUILD_ON != 'github' }}
needs: [version]
runs-on: docker
container:
image: docker:28-cli@sha256:625d9431a9f54c5a2bc90f24f0e1c3d55b1349fd857dd85035f98c2c9acbdd4d # 28-cli
volumes:
- /var/run/docker.sock:/var/run/docker.sock
env:
DOCKER_BUILDKIT: "1"
REGISTRY: ${{ vars.REGISTRY }}
IMAGE: ${{ vars.REGISTRY }}/${{ github.repository }}
VERSION: ${{ needs.version.outputs.version }}
PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
steps:
- uses: coffey-labs/actions/checkout@fab0c4d45e0162963965f1555df27b7bed5e20ec
- run: |
test -n "$REGISTRY" && test -n "$VERSION"
test -n "$PACKAGE_TOKEN" || { echo "PACKAGE_TOKEN secret is not set on this repository" >&2; exit 1; }
echo "$PACKAGE_TOKEN" | docker login -u jcoffey-dev --password-stdin "$REGISTRY"
docker buildx create --use --name gitea-builder --driver docker-container || docker buildx use gitea-builder
# Attestations off, as before: they add manifests of their own, and the
# index should hold the two images and nothing else.
- run: |
docker buildx build \
--platform linux/amd64 \
--provenance=false --sbom=false \
--build-arg INBUXA_REVISION="${{ github.sha }}" \
--tag "$IMAGE:$VERSION-amd64" \
--tag "$IMAGE:$VERSION" \
--push .
docker buildx imagetools inspect "$IMAGE:$VERSION"
# Gitea keeps a container package on its owner; linking it shows it on
# the repository's Packages tab. Idempotent.
- run: |
apk add --no-cache -q curl
curl -fsS -o /dev/null -X POST -H "Authorization: token $PACKAGE_TOKEN" \
"$CI_SERVER_INTERNAL/api/v1/packages/${GITHUB_REPOSITORY%%/*}/container/${GITHUB_REPOSITORY#*/}/-/link/${GITHUB_REPOSITORY#*/}" \
|| echo "package already linked (or link refused); not fatal"
- if: always()
run: docker logout "$REGISTRY" || true
publish-arm64:
if: ${{ vars.BUILD_ON != 'github' }}
needs: [version, publish-amd64]
runs-on: docker
container:
image: docker:28-cli@sha256:625d9431a9f54c5a2bc90f24f0e1c3d55b1349fd857dd85035f98c2c9acbdd4d # 28-cli
volumes:
- /var/run/docker.sock:/var/run/docker.sock
env:
DOCKER_BUILDKIT: "1"
REGISTRY: ${{ vars.REGISTRY }}
IMAGE: ${{ vars.REGISTRY }}/${{ github.repository }}
VERSION: ${{ needs.version.outputs.version }}
TAGS: ${{ needs.version.outputs.tags }}
PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
steps:
- uses: coffey-labs/actions/checkout@fab0c4d45e0162963965f1555df27b7bed5e20ec
- run: |
echo "$PACKAGE_TOKEN" | docker login -u jcoffey-dev --password-stdin "$REGISTRY"
docker run --privileged --rm tonistiigi/binfmt --install arm64
docker buildx create --use --name gitea-builder --driver docker-container || docker buildx use gitea-builder
# The index is built from the two per-architecture tags rather than from
# :<version>, which by now is the amd64 image and would be read as such.
- run: |
docker buildx build \
--platform linux/arm64 \
--provenance=false --sbom=false \
--build-arg INBUXA_REVISION="${{ github.sha }}" \
--tag "$IMAGE:$VERSION-arm64" \
--push .
args=""
for t in $TAGS; do args="$args --tag $IMAGE:$t"; done
docker buildx imagetools create $args \
"$IMAGE:$VERSION-amd64" "$IMAGE:$VERSION-arm64"
docker buildx imagetools inspect "$IMAGE:$VERSION"
- if: always()
run: docker logout "$REGISTRY" || true
# The relay's image, both platforms in one build: its builder isn't a
# cross build, so arm64 compiles under QEMU here, slowly; the relay is
# small. It runs after the server's arm64 build, not beside it. Before any
# tag moves, the amd64 image must say it is this version and hold no
# server binary (RL-T24).
publish-relay:
if: ${{ vars.BUILD_ON != 'github' }}
needs: [version, publish-arm64]
runs-on: docker
container:
image: docker:28-cli@sha256:625d9431a9f54c5a2bc90f24f0e1c3d55b1349fd857dd85035f98c2c9acbdd4d # 28-cli
volumes:
- /var/run/docker.sock:/var/run/docker.sock
env:
DOCKER_BUILDKIT: "1"
REGISTRY: ${{ vars.REGISTRY }}
RELAY_IMAGE: ${{ vars.REGISTRY }}/${{ github.repository_owner }}/inbuxa-relay
VERSION: ${{ needs.version.outputs.version }}
CARGO_VERSION: ${{ needs.version.outputs.cargo }}
TAGS: ${{ needs.version.outputs.tags }}
PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
steps:
- uses: coffey-labs/actions/checkout@fab0c4d45e0162963965f1555df27b7bed5e20ec
- run: |
echo "$PACKAGE_TOKEN" | docker login -u jcoffey-dev --password-stdin "$REGISTRY"
docker run --privileged --rm tonistiigi/binfmt --install arm64
docker buildx create --use --name gitea-builder --driver docker-container || docker buildx use gitea-builder
- run: |
set -eu
for arch in amd64 arm64; do
docker buildx build -f Dockerfile.relay \
--platform "linux/$arch" \
--provenance=false --sbom=false \
--tag "$RELAY_IMAGE:$VERSION-$arch" \
--push .
done
said="$(docker run --rm --platform linux/amd64 "$RELAY_IMAGE:$VERSION-amd64" --version)"
[ "$said" = "inbuxa-relay $CARGO_VERSION" ] || { echo "the relay image says '$said'" >&2; exit 1; }
id="$(docker create --platform linux/amd64 "$RELAY_IMAGE:$VERSION-amd64")"
if docker export "$id" | tar -t | grep -qE '(^|/)inbuxa$'; then
docker rm -f "$id" >/dev/null
echo "the relay image holds a server binary" >&2; exit 1
fi
docker rm -f "$id" >/dev/null
args=""
for t in $TAGS; do args="$args --tag $RELAY_IMAGE:$t"; done
docker buildx imagetools create $args \
"$RELAY_IMAGE:$VERSION-amd64" "$RELAY_IMAGE:$VERSION-arm64"
docker buildx imagetools inspect "$RELAY_IMAGE:$VERSION"
- run: |
apk add --no-cache -q curl
curl -fsS -o /dev/null -X POST -H "Authorization: token $PACKAGE_TOKEN" \
"$CI_SERVER_INTERNAL/api/v1/packages/${GITHUB_REPOSITORY%%/*}/container/inbuxa-relay/-/link/${GITHUB_REPOSITORY#*/}" \
|| echo "package already linked (or link refused); not fatal"
- if: always()
run: docker logout "$REGISTRY" || true
# BUILD_ON=github: waits for the GitHub mirror's run for this tag, which
# posts its result back as the commit status "github/ci (tag)", and takes
# its answer. Fails after the timeout if no answer comes.
github:
if: ${{ vars.BUILD_ON == 'github' }}
# Its own runner label with plenty of slots: this job only polls, but holds a slot
# for as long as the GitHub build takes, and must not starve the build runners.
runs-on: wait
timeout-minutes: 240
container:
image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim
steps:
- env:
TOKEN: ${{ secrets.GITHUB_TOKEN }}
SHA: ${{ github.sha }}
CONTEXT: github/ci (tag)
run: |
python3 - <<'EOF'
import json, os, time, urllib.request
url = (f"{os.environ['CI_SERVER_INTERNAL']}/api/v1/repos/{os.environ['GITHUB_REPOSITORY']}"
f"/commits/{os.environ['SHA']}/statuses?limit=50")
req = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['TOKEN']}"})
ctx, last = os.environ["CONTEXT"], None
print(f"waiting for '{ctx}' on {os.environ['SHA']}", flush=True)
while True:
mine = [s for s in json.load(urllib.request.urlopen(req)) if s["context"] == ctx]
state = max(mine, key=lambda s: s["id"]) if mine else None
if state and state["status"] != last:
last = state["status"]; print(f"{ctx}: {last} {state.get('target_url', '')}", flush=True)
if last == "success": raise SystemExit(0)
if last in ("failure", "error"): raise SystemExit(1)
time.sleep(20)
EOF
# The weekly release creates its Release (and so the tag) first; a tag
# pushed by hand has none. Either way the tag ends up with exactly one
# Release, created once the amd64 image exists so its pull instructions
# work; arm64 and the binaries follow.
release:
if: ${{ vars.BUILD_ON != 'github' }}
needs: [version, publish-amd64]
runs-on: light
container:
image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim
steps:
- shell: bash
env:
TAG: ${{ github.ref_name }}
VERSION: ${{ needs.version.outputs.version }}
TAGS: ${{ needs.version.outputs.tags }}
PRERELEASE: ${{ needs.version.outputs.prerelease }}
TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
REGISTRY: ${{ vars.REGISTRY }}
run: |
python3 - <<'PY'
import json, os, urllib.request, urllib.error
api = f"{os.environ['CI_SERVER_INTERNAL']}/api/v1/repos/{os.environ['REPO']}"
h = {"Authorization": f"token {os.environ['TOKEN']}", "Content-Type": "application/json"}
tag, version = os.environ["TAG"], os.environ["VERSION"]
try:
urllib.request.urlopen(urllib.request.Request(f"{api}/releases/tags/{tag}", headers=h))
print(f"{tag} already has a release"); raise SystemExit
except urllib.error.HTTPError as e:
if e.code != 404: raise
image = f"{os.environ['REGISTRY']}/{os.environ['REPO']}:{version}"
relay = f"{os.environ['REGISTRY']}/{os.environ['REPO'].split('/')[0]}/inbuxa-relay:{version}"
also = " ".join(f"`:{t}`" for t in os.environ["TAGS"].split()[1:])
pre = os.environ["PRERELEASE"] == "true"
body = (("Release candidate for the next series, to try before it ships. "
"`:latest` stays on the current release.\n\n" if pre else "")
+ f"Container image: `{image}` (linux/amd64, linux/arm64); also {also}. "
"amd64 is published first; arm64 is added to the same tag when its build "
"finishes, and the series tags move then.\n\n"
"Binaries for a host install are attached: `inbuxa-linux-amd64.tar.gz` and "
"`inbuxa-linux-arm64.tar.gz`, with `SHA256SUMS`. Each is the binary out of this "
"release's image for that architecture, so it is the same build. The image "
"grants it `cap_net_bind_service`; a host install has to grant that itself "
"(`setcap`, or `AmbientCapabilities` in the unit) to bind port 25.\n\n"
f"inbuxa Relay, the edge relay, ships from the same tag: image `{relay}` "
"(linux/amd64, linux/arm64), and static binaries `inbuxa-relay-x86_64-linux-musl` "
"and `inbuxa-relay-aarch64-linux-musl`, each with a `.sig` file by the inbuxa "
"release key. inbuxa-admin's Relays page takes a binary and its `.sig` to "
"upgrade a relay.")
data = json.dumps({"tag_name": tag, "name": f"INBUXA {version}", "body": body,
"prerelease": pre}).encode()
r = json.load(urllib.request.urlopen(urllib.request.Request(f"{api}/releases", data=data, headers=h)))
print(f"created release {r['tag_name']}")
PY
# The binaries for a host install, taken out of the image that was just
# pushed rather than compiled again.
#
# Building them separately would mean a second Rust build per architecture
# -- the slowest thing this pipeline does -- and would leave two artifacts
# that are supposed to be the same build but only probably are. Extracting
# them makes that identity a fact: the binary in the tarball is the file
# the image runs.
#
# `docker create` does not start anything, so pulling an arm64 image on an
# amd64 runner and copying a file out of it needs no emulation.
binaries:
if: ${{ vars.BUILD_ON != 'github' }}
needs: [version, publish-arm64, publish-relay, release]
runs-on: docker
container:
image: docker:28-cli@sha256:625d9431a9f54c5a2bc90f24f0e1c3d55b1349fd857dd85035f98c2c9acbdd4d # 28-cli
volumes:
- /var/run/docker.sock:/var/run/docker.sock
env:
REGISTRY: ${{ vars.REGISTRY }}
IMAGE: ${{ vars.REGISTRY }}/${{ github.repository }}
VERSION: ${{ needs.version.outputs.version }}
TAG: ${{ github.ref_name }}
REPO: ${{ github.repository }}
PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
TOKEN: ${{ secrets.GITHUB_TOKEN }}
MINISIGN_SECRET_KEY: ${{ secrets.MINISIGN_SECRET_KEY }}
steps:
- name: take the binaries out of the image
run: |
set -euo pipefail
echo "$PACKAGE_TOKEN" | docker login -u jcoffey-dev --password-stdin "$REGISTRY"
mkdir -p /out && cd /out
for arch in amd64 arm64; do
docker pull -q --platform "linux/$arch" "$IMAGE:$VERSION"
id="$(docker create --platform "linux/$arch" "$IMAGE:$VERSION")"
docker cp "$id:/usr/local/bin/inbuxa" "inbuxa"
docker rm -f "$id" >/dev/null
chmod 0755 inbuxa
tar -czf "inbuxa-linux-$arch.tar.gz" inbuxa
rm inbuxa
done
# The relay's, as bare binaries: an upgrade (RL-29) takes the file itself
relay="$REGISTRY/${REPO%%/*}/inbuxa-relay:$VERSION"
for pair in amd64:x86_64 arm64:aarch64; do
arch="${pair%%:*}"; name="inbuxa-relay-${pair#*:}-linux-musl"
docker pull -q --platform "linux/$arch" "$relay"
id="$(docker create --platform "linux/$arch" "$relay")"
docker cp "$id:/usr/local/bin/inbuxa-relay" "$name"
docker rm -f "$id" >/dev/null
chmod 0755 "$name"
done
sha256sum inbuxa-linux-*.tar.gz inbuxa-relay-* > SHA256SUMS
cat SHA256SUMS
# Sign SHA256SUMS with the suite's minisign key, so a download can be
# verified against the public key the docs publish. This is separate
# from the relay .sig (relay-sign), which the server uses only to let a
# relay take a binary as an upgrade. No key (as on a fork, or before the
# key is added) leaves the release unsigned.
- name: sign SHA256SUMS with minisign
run: |
set -euo pipefail
if [ -z "${MINISIGN_SECRET_KEY:-}" ]; then
echo "::warning::MINISIGN_SECRET_KEY is not set; this release will be unsigned"
exit 0
fi
apk add --no-cache -q minisign
umask 077
key="$(mktemp)"; printf '%s\n' "$MINISIGN_SECRET_KEY" > "$key"
minisign -S -s "$key" -m /out/SHA256SUMS -t "inbuxa $TAG"
rm -f "$key"
echo "signed /out/SHA256SUMS -> /out/SHA256SUMS.minisig"
- name: attach them to the release
run: |
set -euo pipefail
apk add --no-cache -q python3
python3 - <<'PY'
import json, os, urllib.request, urllib.error, uuid, pathlib
api = f"{os.environ['CI_SERVER_INTERNAL']}/api/v1/repos/{os.environ['REPO']}"
tok = {"Authorization": f"token {os.environ['TOKEN']}"}
tag = os.environ["TAG"]
def get(path):
return json.load(urllib.request.urlopen(urllib.request.Request(api + path, headers=tok)))
rel = get(f"/releases/tags/{tag}")
assets = {a["name"]: a["id"] for a in get(f"/releases/{rel['id']}/assets")}
for path in ["/out/inbuxa-linux-amd64.tar.gz", "/out/inbuxa-linux-arm64.tar.gz",
"/out/inbuxa-relay-x86_64-linux-musl", "/out/inbuxa-relay-aarch64-linux-musl",
"/out/SHA256SUMS", "/out/SHA256SUMS.minisig"]:
if not pathlib.Path(path).exists():
continue # SHA256SUMS.minisig is absent only on an unsigned release
name = os.path.basename(path)
# A re-run of a tag replaces its assets rather than leaving two
# files with the same name and different contents.
if name in assets:
urllib.request.urlopen(urllib.request.Request(
f"{api}/releases/{rel['id']}/assets/{assets[name]}", headers=tok, method="DELETE"))
boundary = uuid.uuid4().hex
body = b"".join([
f"--{boundary}\r\nContent-Disposition: form-data; name=\"attachment\"; filename=\"{name}\"\r\n".encode(),
b"Content-Type: application/octet-stream\r\n\r\n",
pathlib.Path(path).read_bytes(),
f"\r\n--{boundary}--\r\n".encode(),
])
req = urllib.request.Request(
f"{api}/releases/{rel['id']}/assets?name={name}", data=body, method="POST",
headers={**tok, "Content-Type": f"multipart/form-data; boundary={boundary}"})
urllib.request.urlopen(req)
print("attached", name)
PY
- if: always()
run: docker logout "$REGISTRY" || true
# Signs the relay binaries on the release with the inbuxa release key
# (RL-29), whichever forge put them there: this forge's binaries job, or
# the GitHub run the `github` job waited for. Each .sig names the Cargo
# version the binary's --version prints and the target in its file name.
# The key is written to a file only for the signing, and removed.
relay-sign:
needs: [binaries, github]
if: ${{ always() && (needs.binaries.result == 'success' || needs.github.result == 'success') }}
runs-on: docker
container:
image: docker:28-cli@sha256:625d9431a9f54c5a2bc90f24f0e1c3d55b1349fd857dd85035f98c2c9acbdd4d # 28-cli
env:
TAG: ${{ github.ref_name }}
REPO: ${{ github.repository }}
TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELAY_RELEASE_KEY: ${{ secrets.RELAY_RELEASE_KEY }}
steps:
- uses: coffey-labs/actions/checkout@fab0c4d45e0162963965f1555df27b7bed5e20ec
- shell: sh
run: |
set -eu
if [ -z "$RELAY_RELEASE_KEY" ]; then
echo "::warning::RELAY_RELEASE_KEY is not set: the relay binaries stay unsigned, and no relay takes them as an upgrade"
exit 0
fi
apk add --no-cache -q bash coreutils curl jq openssl python3
version="$(python3 tools/release/version.py --cargo)"
api="$CI_SERVER_INTERNAL/api/v1/repos/$REPO"
auth="Authorization: token $TOKEN"
rel="$(curl -fsS -H "$auth" "$api/releases/tags/$TAG" | jq -r .id)"
assets="$(curl -fsS -H "$auth" "$api/releases/$rel/assets")"
umask 077
key="$(mktemp)"
trap 'rm -f "$key"' EXIT
printf '%s\n' "$RELAY_RELEASE_KEY" > "$key"
for target in x86_64-linux-musl aarch64-linux-musl; do
name="inbuxa-relay-$target"
id="$(echo "$assets" | jq -r --arg n "$name" '.[] | select(.name == $n) | .id' | head -n1)"
[ -n "$id" ] || { echo "the release has no $name" >&2; exit 1; }
curl -fsSL -H "$auth" -o "$name" "$CI_SERVER_INTERNAL/$REPO/releases/download/$TAG/$name"
bash tools/release/sign-relay.sh "$key" "$name" "$version" "$target"
for old in $(echo "$assets" | jq -r --arg n "$name.sig" '.[] | select(.name == $n) | .id'); do
curl -fsS -o /dev/null -X DELETE -H "$auth" "$api/releases/$rel/assets/$old"
done
curl -fsS -o /dev/null -X POST -H "$auth" -F "attachment=@$name.sig" "$api/releases/$rel/assets?name=$name.sig"
echo "signed $name ($version, $target)"
cat "$name.sig"
done
# The release above is made with the job's own token, and Gitea starts no
# workflow for events the Actions bot causes -- announce.yml's
# 'on: release' never fires for it -- so announce it from here.
#
# With BUILD_ON=github the release and binaries come from the GitHub run,
# so the announcement waits for the `github` job instead. The Release that
# run creates for a hand-pushed tag is made with a user token, so
# announce.yml fires for it too; discourse-release keeps one topic per tag.
announce:
needs: [release, binaries, github]
if: ${{ always() && ((needs.release.result == 'success' && needs.binaries.result == 'success') || needs.github.result == 'success') }}
runs-on: light
steps:
- uses: coffey-labs/actions/discourse-release@958b3a35a345744abd236ca5abf679cbedaca7a0
with:
api-key: ${{ secrets.DISCOURSE_RELEASE_KEY }}
discord-webhook: ${{ secrets.DISCORD_RELEASE_WEBHOOK }}
tag: ${{ github.ref_name }}