mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-10-14 13:30:55 +02:00
Make the release verification actually verify the Space and the node
The HF Space step curled the root for HTTP 200 and then emitted a warning and SUCCEEDED when it never got one. The documented contract was three other checks entirely: the requirements pin, the runtime stage, and live identify and visible calls. None was implemented, which is why the demo sat two releases behind while every verification run stayed green. Implemented all three; the pin check fails today, correctly, on 0.34.1 against 0.36.0. The ComfyUI step read the latest run of the node repo's sync workflow. That workflow also runs on a daily schedule, so a green nightly satisfied it whether or not this release was ever synced. The registry does publish a JSON API, contrary to the comment that justified the proxy, so read the node directly and assert the active version declares this library release. Both of those steps warned and continued; only Homebrew failed hard. Now all of them exit non-zero, and the summary is assembled from lines each step appends as it passes rather than an unconditional block that claimed five verified surfaces regardless. No step needs the GitHub token any more, so pin least-privilege permissions here and in test.yml. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
29de21256b
commit
326840bab4
2 files changed
+122
-47
No files matched your search
@@ -5,6 +5,9 @@ on:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: tests-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
@@ -17,13 +17,24 @@ on:
|
||||
workflows: ["Distribute on release"]
|
||||
types: [completed]
|
||||
|
||||
# No step reads the GitHub token any more: the ComfyUI check moved to the
|
||||
# registry's public API. Keep the job read-only so it cannot write on a repo
|
||||
# whose default token permission is read/write.
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
verify:
|
||||
# Only run on a successful distribute (not on manual dispatch, where the
|
||||
# version input or PyPI-latest provides the target).
|
||||
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
timeout-minutes: 25
|
||||
env:
|
||||
# The Space id appears in the Hub API, the raw-file URL and the app host,
|
||||
# spelled three different ways. Name it once so a move cannot half-land.
|
||||
SPACE_REPO: wiltodelta/remove-ai-watermarks
|
||||
SPACE_URL: https://wiltodelta-remove-ai-watermarks.hf.space
|
||||
|
||||
steps:
|
||||
- name: Resolve target version
|
||||
@@ -89,63 +100,124 @@ jobs:
|
||||
echo " Homebrew: formula points to $VERSION"
|
||||
|
||||
# -- ComfyUI ------------------------------------------------------------------
|
||||
- name: Verify ComfyUI sync succeeded
|
||||
- name: Verify ComfyUI registry node requires this release
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Checking ComfyUI node sync..."
|
||||
# The registry itself has no public JSON API; the sync workflow in
|
||||
# the node repo already tests compatibility and publishes only on
|
||||
# success, so verifying its latest run succeeded is the proxy.
|
||||
latest=$(gh run list \
|
||||
--repo wiltodelta/ComfyUI-remove-ai-watermarks \
|
||||
--workflow sync-library.yml \
|
||||
--limit 1 \
|
||||
--json conclusion \
|
||||
| python3 -c "
|
||||
import json, sys
|
||||
runs = json.load(sys.stdin)
|
||||
r = runs[0] if runs else {}
|
||||
print(r.get('conclusion', 'unknown'))
|
||||
")
|
||||
echo " ComfyUI sync: $latest"
|
||||
if [ "$latest" != "success" ]; then
|
||||
echo "::warning::ComfyUI sync latest run is '$latest', not success."
|
||||
echo " Check: https://github.com/wiltodelta/ComfyUI-remove-ai-watermarks/actions"
|
||||
else
|
||||
echo " ComfyUI: sync succeeded, node published"
|
||||
fi
|
||||
echo "Checking the ComfyUI Registry node..."
|
||||
# The registry publishes the node as JSON, including the dependency the
|
||||
# published version declares. Read that directly instead of proxying
|
||||
# through the sync workflow's run status: the node repo runs
|
||||
# sync-library.yml on a daily schedule too, so a green nightly run says
|
||||
# nothing about whether THIS release was ever synced.
|
||||
node=$(curl -fsSL "https://api.comfy.org/nodes/remove-ai-watermarks")
|
||||
echo "$node" | VERSION="$VERSION" python3 -c "
|
||||
import json, os, sys
|
||||
version = os.environ['VERSION']
|
||||
latest = json.load(sys.stdin).get('latest_version') or {}
|
||||
deps = latest.get('dependencies') or []
|
||||
floor = f'>={version}'
|
||||
if latest.get('status') != 'NodeVersionStatusActive':
|
||||
sys.exit(f\"::error::ComfyUI node {latest.get('version')} is {latest.get('status')}, not active\")
|
||||
if not any(floor in d for d in deps):
|
||||
sys.exit(
|
||||
f'::error::ComfyUI node {latest.get(\"version\")} does not require '
|
||||
f'{version}; it declares {deps}'
|
||||
)
|
||||
print(f' ComfyUI: node {latest[\"version\"]} requires {version}')
|
||||
"
|
||||
echo "- ComfyUI: registry node requires $VERSION" >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
# -- HF Space -------------------------------------------------------------------
|
||||
- name: Verify HF Space is alive
|
||||
- name: Verify the HF Space serves this release
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Checking HF Space..."
|
||||
space_url="https://wiltodelta-remove-ai-watermarks.hf.space"
|
||||
# The Space may be sleeping after a factory rebuild; wake it.
|
||||
for attempt in $(seq 1 12); do
|
||||
status=$(curl -s -o /dev/null -w "%{http_code}" --max-time 30 "$space_url/" || echo "000")
|
||||
if [ "$status" = "200" ]; then
|
||||
echo " HF Space: alive (HTTP $status)"
|
||||
exit 0
|
||||
fi
|
||||
echo " HF Space: HTTP $status (attempt $attempt), waiting for rebuild..."
|
||||
echo "Checking the HF Space pin..."
|
||||
# The Space pin bump is a MANUAL step (see docs/release-and-distribution.md):
|
||||
# distribute.yml only factory-rebuilds whatever is already pinned. Failing
|
||||
# here is the point -- a demo left on an older release is an incomplete
|
||||
# release, and only this check can say so.
|
||||
reqs=$(curl -fsSL "https://huggingface.co/spaces/$SPACE_REPO/raw/main/requirements.txt")
|
||||
pinned=$(printf '%s\n' "$reqs" | sed -n 's/^remove-ai-watermarks==\([0-9][0-9a-zA-Z.]*\).*/\1/p' | head -1)
|
||||
if [ -z "$pinned" ]; then
|
||||
echo "::error::Space requirements.txt has no remove-ai-watermarks== pin"
|
||||
exit 1
|
||||
fi
|
||||
if [ "$pinned" != "$VERSION" ]; then
|
||||
echo "::error::HF Space is pinned to $pinned, not $VERSION."
|
||||
echo " Bump remove-ai-watermarks in wiltodelta/raiw-hf-space (pyproject.toml,"
|
||||
echo " uv lock, re-exported requirements.txt) and push; sync-to-hf.yml mirrors it."
|
||||
exit 1
|
||||
fi
|
||||
echo " HF Space: requirements pin is $pinned"
|
||||
|
||||
- name: Verify the HF Space is running and answering
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# A factory rebuild leaves the Space BUILDING for a while; wait for the
|
||||
# runtime stage the API reports rather than for a bare HTTP 200, which a
|
||||
# build page also returns.
|
||||
for attempt in $(seq 1 20); do
|
||||
stage=$(curl -fsSL "https://huggingface.co/api/spaces/$SPACE_REPO" \
|
||||
| python3 -c "import json,sys;print((json.load(sys.stdin).get('runtime') or {}).get('stage',''))")
|
||||
echo " attempt $attempt: runtime stage=$stage"
|
||||
[ "$stage" = "RUNNING" ] && break
|
||||
case "$stage" in
|
||||
BUILD_ERROR|RUNTIME_ERROR|CONFIG_ERROR)
|
||||
echo "::error::HF Space runtime stage is $stage"; exit 1 ;;
|
||||
esac
|
||||
sleep 15
|
||||
done
|
||||
echo "::warning::HF Space did not respond with HTTP 200 after 3 minutes."
|
||||
echo " The Space may still be rebuilding after a factory restart."
|
||||
echo " Verify manually at $space_url"
|
||||
if [ "$stage" != "RUNNING" ]; then
|
||||
echo "::error::HF Space never reached RUNNING (last stage: $stage)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Alive is not the same as working. Drive the two CPU surfaces the demo
|
||||
# exists for, against the Space's own committed example, and require a
|
||||
# real verdict rather than any HTTP 200.
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
curl -fsSL -o /tmp/example.png \
|
||||
"https://huggingface.co/spaces/$SPACE_REPO/resolve/main/examples/doubao-aigc.png"
|
||||
uv run --no-project --with 'gradio_client>=1,<2' python - <<'PYEOF'
|
||||
import os
|
||||
import sys
|
||||
|
||||
from gradio_client import Client, handle_file
|
||||
|
||||
# The committed example is a 5 MB PNG, and httpx's default 5 s write
|
||||
# timeout aborts that upload on a slow link before the Space ever sees
|
||||
# it -- which would read as a broken demo rather than a slow one.
|
||||
client = Client(os.environ["SPACE_URL"], verbose=False, httpx_kwargs={"timeout": 300})
|
||||
|
||||
report = client.predict(path=handle_file("/tmp/example.png"), api_name="/run_identify")
|
||||
if "Verdict:" not in report:
|
||||
sys.exit(f"::error::/run_identify returned no verdict: {report[:200]!r}")
|
||||
print(f" HF Space identify: {report.splitlines()[0]}")
|
||||
|
||||
_image, _file, info = client.predict(
|
||||
path=handle_file("/tmp/example.png"),
|
||||
sensitivity_label="auto (recommended)",
|
||||
api_name="/run_visible",
|
||||
)
|
||||
if "Removed:" not in info:
|
||||
sys.exit(f"::error::/run_visible removed nothing on the Doubao example: {info!r}")
|
||||
print(f" HF Space visible: {info.splitlines()[0]}")
|
||||
PYEOF
|
||||
echo "- HF Space: pinned to $VERSION, RUNNING, identify + visible answered" >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Summary
|
||||
env:
|
||||
VERSION: ${{ steps.v.outputs.version }}
|
||||
run: |
|
||||
cat <<EOF
|
||||
Release $VERSION verified:
|
||||
PyPI: wheel + sdist present
|
||||
Clean install: --version returns $VERSION
|
||||
Homebrew: formula points to $VERSION
|
||||
ComfyUI: sync succeeded
|
||||
HF Space: alive and responding
|
||||
EOF
|
||||
# Every check above appends its own line to the step summary as it passes,
|
||||
# and every one of them fails the job instead of warning. So this heading
|
||||
# can only be reached when all of them actually held -- an unconditional
|
||||
# "verified" block printed here once claimed surfaces nobody had checked.
|
||||
sed -i "1i ## Release $VERSION verified" "$GITHUB_STEP_SUMMARY"
|
||||
cat "$GITHUB_STEP_SUMMARY"
|
||||
Reference in new issue
Block a user