Compare commits
36 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 steve
|
aa80a3418e | ||
|
steve
|
ac9d7b92ed | ||
|
steve
|
556d65d77f | ||
|
steve
|
7ee8d2311b | ||
|
steve
|
e4dd7f96d6 | ||
|
steve
|
6afde3ce23 | ||
|
steve
|
775e988340 | ||
|
steve
|
e2cf9a68f6 | ||
|
steve
|
1e212db24e | ||
|
steve
|
a9185424d3 | ||
|
steve
|
9c5037ec54 | ||
|
steve
|
ed839aacb4 | ||
|
steve
|
7d2c2ae1c2 | ||
|
steve
|
a131419b1a | ||
|
steve
|
9d727a7b3a | ||
|
steve
|
1cbc856514 | ||
|
steve
|
fb24e42c6e | ||
|
steve
|
a899cc2d04 | ||
|
steve
|
ef2a30a82d | ||
|
steve
|
e7960151fb | ||
|
steve
|
2d27a23e99 | ||
|
steve
|
9d9773cad4 | ||
|
steve
|
b3d033fa11 | ||
|
steve
|
b7033fcfcd | ||
|
steve
|
b1b0d9d1e9 | ||
|
steve
|
f711593549 | ||
|
steve
|
dfc2cd314d | ||
|
steve
|
5e2b88c6dd | ||
|
steve
|
768972d870 | ||
|
steve
|
82a73fad85 | ||
|
steve
|
26bb881c12 | ||
|
steve
|
3873bd9d34 | ||
|
steve
|
1bb31b9c49 | ||
|
steve
|
4985050a0a | ||
|
steve
|
1c7b471e75 | ||
|
steve
|
88216d29d0 |
+4
-46
@@ -1,46 +1,3 @@
|
||||
# CI workflow — runs on every PR into main.
|
||||
#
|
||||
# Notes for anyone editing this file:
|
||||
#
|
||||
# Self-hosted runner expectations
|
||||
# The Gitea runners are provisioned via scripts/provision-gitea-runner.sh.
|
||||
# That script bind-mounts persistent host volumes for /root/go/pkg/mod
|
||||
# (GOMODCACHE), /root/.cache/go-build (GOCACHE), and /root/.cache/act
|
||||
# (action clones) into every job container. As a result:
|
||||
# * `cache: true` on actions/setup-go is intentionally OMITTED — the
|
||||
# action would otherwise tar/untar GOMODCACHE+GOCACHE through the
|
||||
# Gitea cache backend on every job, undoing the host-volume cache
|
||||
# and adding ~10s of redundant zstd round-trip per job.
|
||||
# * Common GitHub actions (actions/checkout, actions/setup-go,
|
||||
# actions/upload-artifact, golangci/golangci-lint-action) are
|
||||
# pre-cloned into /root/.cache/act on the runner, so the per-job
|
||||
# "git clone https://github.com/actions/..." step is a fetch, not
|
||||
# a full clone.
|
||||
# * golangci-lint is pre-installed at /usr/local/bin/golangci-lint
|
||||
# on the runner (latest v2.x). The golangci-lint-action below
|
||||
# still pins a specific version and re-downloads — that's fine
|
||||
# (deterministic CI > marginal speed) but means the host-installed
|
||||
# binary is currently unused. Drop the `version:` arg below to
|
||||
# use the host-installed one if you want to trade determinism
|
||||
# for speed.
|
||||
#
|
||||
# Build matrix
|
||||
# Linux amd64 + arm64 + Windows amd64. CGO_ENABLED=0 throughout —
|
||||
# modernc.org/sqlite is pure-Go so no cross-compile toolchain is
|
||||
# needed. -trimpath + -ldflags="-s -w" for reproducible, smaller
|
||||
# binaries.
|
||||
#
|
||||
# Go version
|
||||
# The GO_VERSION env var anchors all three jobs. Floor is set by the
|
||||
# heaviest dep (modernc.org/sqlite v1.50+ requires Go 1.23+ today;
|
||||
# we run 1.25 so golangci-lint's Go-version compatibility check is
|
||||
# happy — see the version pin in the lint job).
|
||||
#
|
||||
# upload-artifact
|
||||
# Pinned at v3 historically; v3 was deprecated upstream. v4 should
|
||||
# work but hasn't been validated against this runner's act_runner
|
||||
# version yet. Bump when convenient.
|
||||
|
||||
name: CI
|
||||
|
||||
on:
|
||||
@@ -48,6 +5,7 @@ on:
|
||||
branches: [main]
|
||||
|
||||
env:
|
||||
# Floor is set by the heaviest dep (modernc.org/sqlite v1.50+).
|
||||
GO_VERSION: "1.25"
|
||||
|
||||
jobs:
|
||||
@@ -59,7 +17,7 @@ jobs:
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: ${{ env.GO_VERSION }}
|
||||
# cache: true intentionally omitted — see header notes.
|
||||
cache: true
|
||||
- name: go vet
|
||||
run: go vet ./...
|
||||
- name: go test
|
||||
@@ -75,7 +33,7 @@ jobs:
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: ${{ env.GO_VERSION }}
|
||||
# cache: true intentionally omitted — see header notes.
|
||||
cache: true
|
||||
- uses: golangci/golangci-lint-action@v7
|
||||
with:
|
||||
# Must be built against the same Go release as go.mod targets,
|
||||
@@ -105,7 +63,7 @@ jobs:
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: ${{ env.GO_VERSION }}
|
||||
# cache: true intentionally omitted — see header notes.
|
||||
cache: true
|
||||
- name: build server + agent
|
||||
env:
|
||||
GOOS: ${{ matrix.goos }}
|
||||
|
||||
@@ -1,327 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# provision-gitea-runner.sh — one-shot, idempotent host setup for an
|
||||
# act_runner LXC. Speeds up Gitea Actions runs by:
|
||||
#
|
||||
# 1. Disabling forced docker pulls (image refresh moves to a cron).
|
||||
# 2. Mounting persistent host volumes for Go module/build caches and
|
||||
# the act-actions clone cache.
|
||||
# 3. Pre-pulling the runner-images container image.
|
||||
# 4. Pre-cloning a configurable list of GitHub actions into the
|
||||
# act cache so jobs don't fetch them on every run.
|
||||
# 5. Installing golangci-lint (latest v2.x) at /usr/local/bin.
|
||||
# 6. Setting up a nightly cron to refresh image + action clones +
|
||||
# golangci-lint.
|
||||
#
|
||||
# The script is generic — no per-project state. Point it at any LXC
|
||||
# running act_runner as a systemd service and it will provision the
|
||||
# host. Re-runs are safe; they reconcile state.
|
||||
#
|
||||
# Usage: sudo ./provision-gitea-runner.sh
|
||||
#
|
||||
# Configurable via environment variables (defaults shown):
|
||||
#
|
||||
# CACHE_BASE=/var/cache/gitea-runner
|
||||
# ACT_RUNNER_CONFIG=/etc/act_runner/config.yaml
|
||||
# RUNNER_IMAGE=docker.gitea.com/runner-images:ubuntu-latest
|
||||
# ACTIONS_TO_PRECLONE=(actions/checkout@v4 actions/setup-go@v5
|
||||
# actions/upload-artifact@v4
|
||||
# golangci/golangci-lint-action@v7)
|
||||
#
|
||||
# To add more pre-cloned actions later, edit /etc/cron.d/gitea-runner-refresh
|
||||
# (the ACTIONS list is materialised into the cron script).
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# ---------- defaults ---------------------------------------------------
|
||||
|
||||
: "${CACHE_BASE:=/var/cache/gitea-runner}"
|
||||
: "${ACT_RUNNER_CONFIG:=/etc/act_runner/config.yaml}"
|
||||
: "${RUNNER_IMAGE:=docker.gitea.com/runner-images:ubuntu-latest}"
|
||||
|
||||
DEFAULT_ACTIONS=(
|
||||
"actions/checkout@v4"
|
||||
"actions/setup-go@v5"
|
||||
"actions/upload-artifact@v4"
|
||||
"golangci/golangci-lint-action@v7"
|
||||
)
|
||||
# Allow caller to override by exporting ACTIONS_TO_PRECLONE as a
|
||||
# space-separated string (env vars can't carry arrays cleanly).
|
||||
if [[ -n "${ACTIONS_TO_PRECLONE:-}" ]]; then
|
||||
read -r -a ACTIONS <<<"${ACTIONS_TO_PRECLONE}"
|
||||
else
|
||||
ACTIONS=("${DEFAULT_ACTIONS[@]}")
|
||||
fi
|
||||
|
||||
# ---------- helpers ----------------------------------------------------
|
||||
|
||||
log() { printf '\033[1;36m==>\033[0m %s\n' "$*"; }
|
||||
warn() { printf '\033[1;33m==>\033[0m %s\n' "$*" >&2; }
|
||||
die() { printf '\033[1;31m==>\033[0m %s\n' "$*" >&2; exit 1; }
|
||||
|
||||
require_cmd() {
|
||||
command -v "$1" >/dev/null 2>&1 || die "missing: $1 (install it first)"
|
||||
}
|
||||
|
||||
# sha256_url <url> — act_runner names its action-clone dirs after
|
||||
# sha256(URL). Verified against a real run log:
|
||||
# url=https://github.com/actions/checkout
|
||||
# sha256=c3fe249fe73091a17d6638fe1341e7bd0bcc3466ce52323c0688e83e2463a4ab
|
||||
sha256_url() {
|
||||
printf '%s' "$1" | sha256sum | awk '{print $1}'
|
||||
}
|
||||
|
||||
# ---------- pre-flight -------------------------------------------------
|
||||
|
||||
[[ $EUID -eq 0 ]] || die "run as root (the act_runner service writes /var/lib/act_runner as root)"
|
||||
|
||||
require_cmd systemctl
|
||||
require_cmd docker
|
||||
require_cmd git
|
||||
require_cmd curl
|
||||
require_cmd python3
|
||||
|
||||
# PyYAML for the config edit. Install if missing — Ubuntu 24.04 ships
|
||||
# python3-yaml in the default repos.
|
||||
if ! python3 -c 'import yaml' 2>/dev/null; then
|
||||
log "installing python3-yaml (needed for safe YAML edits)"
|
||||
apt-get update -qq
|
||||
apt-get install -y -qq python3-yaml
|
||||
fi
|
||||
|
||||
[[ -f "$ACT_RUNNER_CONFIG" ]] || die "$ACT_RUNNER_CONFIG not found — is act_runner installed?"
|
||||
|
||||
systemctl list-unit-files act_runner.service >/dev/null 2>&1 || \
|
||||
die "act_runner.service not found — register the runner first"
|
||||
|
||||
log "pre-flight OK"
|
||||
log " cache base : $CACHE_BASE"
|
||||
log " config file : $ACT_RUNNER_CONFIG"
|
||||
log " runner image : $RUNNER_IMAGE"
|
||||
log " actions to clone : ${ACTIONS[*]}"
|
||||
|
||||
# ---------- 1. cache directories ---------------------------------------
|
||||
|
||||
log "creating cache directories under $CACHE_BASE"
|
||||
for sub in go-mod go-build act-actions; do
|
||||
install -d -m 0755 -o root -g root "$CACHE_BASE/$sub"
|
||||
done
|
||||
|
||||
# ---------- 2. edit /etc/act_runner/config.yaml ------------------------
|
||||
#
|
||||
# Three keys are reconciled to known values:
|
||||
#
|
||||
# container.force_pull : false (we keep the image fresh via cron)
|
||||
# container.options : "-v <cache mounts...>" (auto-mount caches
|
||||
# into every job container)
|
||||
# container.valid_volumes: [<our cache paths>] (whitelist so the
|
||||
# container.options mounts are accepted)
|
||||
#
|
||||
# Other keys are preserved verbatim. The edit is idempotent: re-running
|
||||
# yields the same file content.
|
||||
|
||||
log "patching $ACT_RUNNER_CONFIG"
|
||||
|
||||
# Backup once (only if no .pre-provision backup exists yet).
|
||||
if [[ ! -f "${ACT_RUNNER_CONFIG}.pre-provision" ]]; then
|
||||
cp -p "$ACT_RUNNER_CONFIG" "${ACT_RUNNER_CONFIG}.pre-provision"
|
||||
log " saved pristine copy to ${ACT_RUNNER_CONFIG}.pre-provision"
|
||||
fi
|
||||
|
||||
CONTAINER_OPTIONS_VALUE="-v ${CACHE_BASE}/go-mod:/root/go/pkg/mod:rw -v ${CACHE_BASE}/go-build:/root/.cache/go-build:rw -v ${CACHE_BASE}/act-actions:/root/.cache/act:rw"
|
||||
|
||||
CACHE_BASE="$CACHE_BASE" CONTAINER_OPTIONS_VALUE="$CONTAINER_OPTIONS_VALUE" \
|
||||
ACT_RUNNER_CONFIG="$ACT_RUNNER_CONFIG" \
|
||||
python3 - <<'PY'
|
||||
import os, sys, yaml
|
||||
cfg_path = os.environ['ACT_RUNNER_CONFIG']
|
||||
cache_base = os.environ['CACHE_BASE']
|
||||
container_options = os.environ['CONTAINER_OPTIONS_VALUE']
|
||||
|
||||
with open(cfg_path) as f:
|
||||
cfg = yaml.safe_load(f) or {}
|
||||
|
||||
cfg.setdefault('container', {})
|
||||
cfg['container']['force_pull'] = False
|
||||
cfg['container']['options'] = container_options
|
||||
|
||||
# Whitelist every cache subdir explicitly so jobs that try to bind-mount
|
||||
# them via workflow-side `volumes:` (rare but possible) are accepted.
|
||||
desired_vols = [
|
||||
f"{cache_base}/go-mod",
|
||||
f"{cache_base}/go-build",
|
||||
f"{cache_base}/act-actions",
|
||||
]
|
||||
existing = cfg['container'].get('valid_volumes') or []
|
||||
merged = list(dict.fromkeys(existing + desired_vols)) # de-dup, preserve order
|
||||
cfg['container']['valid_volumes'] = merged
|
||||
|
||||
# Write back with stable formatting. yaml.dump preserves enough
|
||||
# structure for act_runner to parse; comments in the original config
|
||||
# do get stripped — that's why we preserve the .pre-provision backup.
|
||||
with open(cfg_path + '.tmp', 'w') as f:
|
||||
yaml.safe_dump(cfg, f, default_flow_style=False, sort_keys=False)
|
||||
os.replace(cfg_path + '.tmp', cfg_path)
|
||||
print(f" container.force_pull : false")
|
||||
print(f" container.options : {container_options}")
|
||||
print(f" container.valid_volumes: {merged}")
|
||||
PY
|
||||
|
||||
# ---------- 3. pre-pull the runner image -------------------------------
|
||||
|
||||
log "pulling $RUNNER_IMAGE (one-time; cron refreshes it nightly)"
|
||||
docker pull "$RUNNER_IMAGE"
|
||||
|
||||
# ---------- 4. pre-clone the actions list ------------------------------
|
||||
#
|
||||
# act_runner expects clones at $cache/<sha256(url)> with the ref already
|
||||
# checked out. We clone the default branch then fetch + check out the
|
||||
# requested ref. Re-running fetches updates rather than re-cloning.
|
||||
|
||||
log "pre-cloning actions into $CACHE_BASE/act-actions"
|
||||
for spec in "${ACTIONS[@]}"; do
|
||||
if [[ "$spec" != *@* ]]; then
|
||||
warn " skip '$spec' — must be owner/repo@ref"
|
||||
continue
|
||||
fi
|
||||
repo="${spec%@*}"
|
||||
ref="${spec##*@}"
|
||||
url="https://github.com/${repo}"
|
||||
dir="${CACHE_BASE}/act-actions/$(sha256_url "$url")"
|
||||
|
||||
if [[ -d "$dir/.git" ]]; then
|
||||
log " refresh $repo @ $ref"
|
||||
git -C "$dir" fetch --quiet --tags --prune origin
|
||||
else
|
||||
log " clone $repo @ $ref → $dir"
|
||||
git clone --quiet "$url" "$dir"
|
||||
fi
|
||||
# Detach onto the requested ref. Works for branches, tags, and SHAs.
|
||||
if ! git -C "$dir" -c advice.detachedHead=false checkout --quiet "$ref" 2>/dev/null; then
|
||||
# If `ref` is a remote branch we haven't tracked yet, try origin/<ref>.
|
||||
git -C "$dir" -c advice.detachedHead=false checkout --quiet "origin/$ref"
|
||||
fi
|
||||
done
|
||||
|
||||
# ---------- 5. golangci-lint -------------------------------------------
|
||||
#
|
||||
# Install the latest v2.x at /usr/local/bin/golangci-lint. Workflows
|
||||
# that pin a specific version via the action's `version:` arg will
|
||||
# still re-download — but jobs that don't pin (or pin to "latest"/"v2")
|
||||
# get the host-installed binary for free.
|
||||
|
||||
log "installing/updating golangci-lint (latest v2.x) → /usr/local/bin"
|
||||
GOLANGCI_INSTALL_URL="https://raw.githubusercontent.com/golangci/golangci-lint/HEAD/install.sh"
|
||||
# `-b` = install dir, `-d` = quiet "downloading" lines, no version arg
|
||||
# means "latest" — which install.sh resolves to the latest v2 release
|
||||
# from GitHub releases.
|
||||
curl -fsSL "$GOLANGCI_INSTALL_URL" | sh -s -- -b /usr/local/bin >/dev/null
|
||||
/usr/local/bin/golangci-lint --version || warn "golangci-lint install verification failed"
|
||||
|
||||
# ---------- 6. nightly refresh cron ------------------------------------
|
||||
#
|
||||
# Re-pulls the runner image, refreshes the action clones, and updates
|
||||
# golangci-lint. Runs at 03:17 to dodge top-of-hour CI bursts.
|
||||
|
||||
CRON_PATH=/etc/cron.d/gitea-runner-refresh
|
||||
REFRESH_SCRIPT=/usr/local/sbin/gitea-runner-refresh
|
||||
|
||||
log "writing $REFRESH_SCRIPT and $CRON_PATH"
|
||||
|
||||
# Materialise the actions list into the script so the cron is
|
||||
# self-contained and surviving an edit to this file.
|
||||
ACTIONS_LITERAL=""
|
||||
for s in "${ACTIONS[@]}"; do
|
||||
ACTIONS_LITERAL="${ACTIONS_LITERAL} \"$s\"\n"
|
||||
done
|
||||
|
||||
cat >"$REFRESH_SCRIPT" <<EOF
|
||||
#!/usr/bin/env bash
|
||||
# Auto-generated by provision-gitea-runner.sh. Re-running the
|
||||
# provisioning script regenerates this file.
|
||||
set -euo pipefail
|
||||
CACHE_BASE="$CACHE_BASE"
|
||||
RUNNER_IMAGE="$RUNNER_IMAGE"
|
||||
ACTIONS=(
|
||||
$(printf ' "%s"\n' "${ACTIONS[@]}")
|
||||
)
|
||||
|
||||
sha256_url() { printf '%s' "\$1" | sha256sum | awk '{print \$1}'; }
|
||||
|
||||
# 1. Refresh the runner-images base.
|
||||
docker pull -q "\$RUNNER_IMAGE" >/dev/null
|
||||
|
||||
# 2. Refresh action clones.
|
||||
for spec in "\${ACTIONS[@]}"; do
|
||||
[[ "\$spec" == *@* ]] || continue
|
||||
repo="\${spec%@*}"; ref="\${spec##*@}"
|
||||
url="https://github.com/\$repo"
|
||||
dir="\$CACHE_BASE/act-actions/\$(sha256_url "\$url")"
|
||||
if [[ -d "\$dir/.git" ]]; then
|
||||
git -C "\$dir" fetch --quiet --tags --prune origin || true
|
||||
git -C "\$dir" -c advice.detachedHead=false checkout --quiet "\$ref" 2>/dev/null \\
|
||||
|| git -C "\$dir" -c advice.detachedHead=false checkout --quiet "origin/\$ref" || true
|
||||
fi
|
||||
done
|
||||
|
||||
# 3. Refresh golangci-lint (latest v2.x). Tolerate transient
|
||||
# GitHub-rate-limit failures — next night will retry.
|
||||
curl -fsSL https://raw.githubusercontent.com/golangci/golangci-lint/HEAD/install.sh \\
|
||||
| sh -s -- -b /usr/local/bin >/dev/null 2>&1 || true
|
||||
EOF
|
||||
chmod 0755 "$REFRESH_SCRIPT"
|
||||
|
||||
cat >"$CRON_PATH" <<EOF
|
||||
# Auto-generated by provision-gitea-runner.sh. Refreshes the runner
|
||||
# image, action clones, and golangci-lint every night at 03:17.
|
||||
SHELL=/bin/bash
|
||||
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
||||
17 3 * * * root $REFRESH_SCRIPT >> /var/log/gitea-runner-refresh.log 2>&1
|
||||
EOF
|
||||
chmod 0644 "$CRON_PATH"
|
||||
|
||||
# ---------- 7. restart act_runner --------------------------------------
|
||||
|
||||
log "restarting act_runner.service to pick up the new config"
|
||||
systemctl restart act_runner.service
|
||||
sleep 2
|
||||
systemctl is-active --quiet act_runner.service \
|
||||
|| die "act_runner did not come back up — check 'journalctl -u act_runner -n 50'"
|
||||
|
||||
# ---------- 8. container-create benchmark ------------------------------
|
||||
#
|
||||
# Reports cold + warm `docker run --rm <image> true` time. Sanity check
|
||||
# that overlay setup is fast on this host. Numbers > ~5s indicate a
|
||||
# slow filesystem or DNS issue worth investigating separately.
|
||||
|
||||
log "benchmark: docker run --rm $RUNNER_IMAGE true"
|
||||
{
|
||||
printf ' cold (post-pull) : '
|
||||
/usr/bin/time -f '%e s' docker run --rm "$RUNNER_IMAGE" true 2>&1 | tail -1
|
||||
printf ' warm (immediate) : '
|
||||
/usr/bin/time -f '%e s' docker run --rm "$RUNNER_IMAGE" true 2>&1 | tail -1
|
||||
} || warn "benchmark failed — non-fatal"
|
||||
|
||||
# ---------- done -------------------------------------------------------
|
||||
|
||||
cat <<EOF
|
||||
|
||||
\033[1;32m==> Provisioning complete\033[0m
|
||||
|
||||
What changed on this host:
|
||||
* /etc/act_runner/config.yaml — force_pull off, container.options +
|
||||
valid_volumes set for the cache mounts. Pristine copy preserved
|
||||
at ${ACT_RUNNER_CONFIG}.pre-provision.
|
||||
* $CACHE_BASE/{go-mod,go-build,act-actions} — persistent caches.
|
||||
* /usr/local/bin/golangci-lint — latest v2.x.
|
||||
* $REFRESH_SCRIPT and $CRON_PATH — nightly refresh @ 03:17.
|
||||
* Runner image pre-pulled.
|
||||
|
||||
\033[1;33mNote on Go cache + setup-go:\033[0m if your workflow uses
|
||||
\`actions/setup-go\` with \`cache: true\`, the action will still tar/untar
|
||||
the cache via the Gitea cache backend on every job — partially
|
||||
defeating the persistent volume. For full speed-up, drop \`cache: true\`
|
||||
from the workflow once the persistent volume is warm. Per-project
|
||||
decision; this script doesn't touch workflows.
|
||||
|
||||
EOF
|
||||
Reference in New Issue
Block a user