CalvinBackup/ctrld
1
0
Fork 0
mirror of https://github.com/Control-D-Inc/ctrld.git synced 2026-09-04 13:36:35 +02:00
Code Issues Packages Projects Releases Wiki Activity
Files
c48220a404c9c04e37c355be98e3e8b3e40bb5eb
ctrld/docs/provisioning-failure-codes.md
T

18 KiB
Raw Blame History

Provisioning failure codes

When ctrld hits a terminal failure during provisioning, it reports the same stable code on three surfaces:

  • Result file — provision_result.json in the ctrld home directory (next to the persisted internal ctrld.log). JSON with stage, code, exit_code, message, and for listener failures a bounded detail.attempts list of {addr, proto, os_error}. Written atomically, removed on the next successful provisioning. Never contains provision tokens, resolver/device IDs, or configuration contents.
  • Output line — one fixed-format line on the CLI output: provisioning failed: stage=<stage> code=<CODE> (exit <N>). Installer or MDM wrappers can extract exactly this line (fixed charset: stage=[a-z]* code=[A-Z_]* (exit [0-9]*)) into their own logs without risking token leakage from other output.
  • Exit code — stage-scoped: input 20–29, bootstrap 30–39, listener 40–49, service 50–59. This does not change other existing exit-code contracts: ctrld-client status still exits 0–3, and an invalid deactivation pin still exits 126.

A customer or administrator only needs to report the code (or the whole output line). The table below is the maintained support mapping; it must stay in sync with cmd/cli/provision_result.go and changes in the same MR.

Codes

For every code below, first run sudo ctrld-client diag (sudo ctrld-client diag --json gives a paste-safe, machine-readable copy). Get that output before you do anything else. The command needs no running service. Without root, the last-provisioning-result and service-state sections report permission denied, so run it with sudo. It repeats the same stage, code, and message the customer already reported. The Next action / evidence column lists what to look at next.

Code Stage Exit Failure scenario Next action / evidence
PROVISION_TOKEN_MALFORMED input 21 The --cd-org value is not a provisioning code: it is empty, too short, too long, or it contains whitespace or control characters. ctrld checks this before any network call. A missing org-v1- prefix is not this code. That case only logs a warning, because legacy codes can lack the prefix. Make sure that the provisioning code was copied in full, with no extra whitespace. The result file never echoes the value.
CUSTOM_HOSTNAME_INVALID input 22 The --custom-hostname value fails ctrld's own hostname rule (validHostname): it is not 3 to 64 characters, or it is not in RFC1123 hostname format. ctrld checks this before any network call. The API itself does not reject a bad hostname during provisioning. When ControlD registers the device name, it folds or strips characters like space, +, and .. Because of this, a value ctrld accepts can still register under an adjusted name. That case only logs a notice, not a failure. Fix --custom-hostname following the message: it names the offending characters and the allowed format.
INTERCEPT_MODE_INVALID input 23 The --intercept-mode value is not one of off, dns, or hard. ctrld checks this before it installs the service. Re-run with a valid --intercept-mode value.
INVALID_FLAG_COMBINATION input 24 --cd or --cd-org is used together with --nextdns, or --proto is set to anything other than doh/doh3 while --cd is in play. ctrld checks this before any network call or service install. Re-run without the conflicting flag, or fix the invalid value. The message names the exact flags involved.
API_UNREACHABLE bootstrap 30 The Control D API could not be reached or answered with a retryable error (network failure, proxy interference, 5xx, timeout) and retries ran out. The service manager may retry the service later. Make sure that the device has network access to api.controld.com (DNS, proxy, firewall, captive portal). ctrld-client diag's api_reachability block already ran this same probe. Ask whether other TLS traffic works.
API_REJECTED bootstrap 31 The API answered and permanently rejected the configuration (4xx other than 408/429): bad or revoked token, malformed request. ctrld exits without burning service-manager restarts because retrying cannot change the answer. This is also the fallback for the four TOKEN_* codes below. Those codes need the API response to carry error.metadata.reason. An API deployment that does not yet populate that field reports every provisioning-code rejection as API_REJECTED instead. Verify the provision token / org configuration in the Control D dashboard. Re-push after fixing credentials. Evidence: HTTP status in the result file message.
API_DEVICE_INVALID bootstrap 32 The API reports the device/resolver no longer exists (error code 40402). When the daemon's own bootstrap preflight discovers this, it self-uninstalls, because the identity is gone server-side. The direct --cd <uid> install path (ctrld-client start --cd) can also hit this code before the service exists. On that path there is nothing to uninstall yet. Confirm the device was deleted or re-provisioned in the dashboard; re-provision with a current token. No local evidence needed beyond the code.
TOKEN_INVALID bootstrap 33 The API rejected the --cd-org provisioning code with reason token_invalid: the code is not recognized. This code needs the deployed API to send error.metadata.reason: "token_invalid". See API_REJECTED above for the fallback when it does not. Make sure that the code is correct. Re-enter it exactly as given.
TOKEN_EXPIRED bootstrap 34 The API rejected the --cd-org provisioning code with reason token_expired. This has the same API-reason dependency as TOKEN_INVALID. Get a new provisioning code from your administrator.
TOKEN_LIMIT_REACHED bootstrap 35 The API rejected the --cd-org provisioning code with reason token_limit_reached: it reached its device limit. This has the same API-reason dependency as TOKEN_INVALID. Free up a device slot or use a different provisioning code.
TOKEN_DISABLED bootstrap 36 The API rejected the --cd-org provisioning code with reason token_disabled: the code was invalidated. This has the same API-reason dependency as TOKEN_INVALID. Download a profile from an active provisioning code.
LISTENER_BIND_FAILED listener 41 No listen address could be bound after all fallbacks (configured address, 0.0.0.0:53, localhost:53, random) were exhausted. detail.attempts records each tried address with the UDP/TCP OS error, e.g. address already in use (another DNS service owns the port) or can't assign requested address (address not on any interface). Read detail.attempts: address already in use → find the process owning the port (sudo lsof -i :53 -nP); can't assign requested address → the configured IP is not present on the device. Then fix the conflict or the listener config.
LISTENER_CONFIGURED_ADDR_UNAVAILABLE listener 42 An explicitly configured listener address could not be bound and configuration checks forbid falling back to another address, or (macOS intercept mode) the required explicit address is unavailable. The configured ip:port in the listener config is wrong for this device or occupied. Verify the address exists on an interface and nothing else binds it; correct the config rather than expecting fallback.
SERVICE_INSTALL_FAILED service 51 The OS service manager refused to install the service (launchd/systemd/SCM registration failed). Check OS-level constraints: permissions/elevation, MDM policy blocking daemon installation, corrupted previous install. Evidence: result file message (service manager error), plus launchctl print system/ctrld-client / systemctl status ctrld-client / SCM state.
SERVICE_START_FAILED service 52 The service installed but the service manager could not start it. Check the service manager's own log for the start error, then the ctrld home dir ctrld.log. Often permissions or a binary quarantined by security tooling.
SERVICE_SELFCHECK_FAILED service 53 The service started but never became healthy: no fresher failure was reported by the daemon, and the post-install DNS self-check failed. On a fresh install or an upgrade, the just-installed service is rolled back (uninstalled). A restart of an already installed service keeps that service installed. If the daemon itself recorded a more specific failure (e.g. a listener code), that code is reported instead of this one. Ask for the drained service log printed by ctrld-client start and the result file. If the service was running but unreachable, check host firewall rules intercepting DNS to the listener.
UNCLASSIFIED service 59 This code covers terminal failures on the provisioning boundary that predate a dedicated code. Examples include a config unmarshal, and a file-system or environment failure such as writing the config file, reading socketDir, or respawning as a daemon. Other examples are a service-argument update failing mid-upgrade, and a DNS-intercept failure the interface-DNS fallback cannot safely take over from. Every one of these used to be a bare crash with nothing to read. Now they all persist a result file. Read the result file message: it names the specific operation that failed and the underlying OS error. When you report it, treat this the same as any other stage-scoped failure.

The macOS installer script (scripts/pkg/postinstall) emits the package rows below, not ctrld itself, before ctrld ever runs. They only ever exit 1 (the script's own exit code), and they never appear in provision_result.json. The same script also reuses PROVISION_TOKEN_MALFORMED and CUSTOM_HOSTNAME_INVALID from the table above, for its own pre-flight checks on the managed-prefs ProvisionToken/CustomHostname values. It uses the same rule, and also logs it as stage=package (exit 1). So those two codes can appear with either stage, depending on which side rejected the value.

| PROFILE_PREFS_MISSING | package | 1 | The MDM configuration profile (managed prefs domain com.controld.ctrld) has no ProvisionToken, or the value is empty. This shows up after the postinstall script's wait loop ends: 12 attempts, 10 seconds apart, about 2 minutes. | Scope the com.controld.ctrld configuration profile to the device. If the profile is already on the device, set its ProvisionToken value. Then reinstall the package, or run sudo ctrld-client start --cd-org=<token> by hand. | | INTERCEPT_MODE_INVALID | package | 1 | On a fresh install, the profile's InterceptMode is set, but it is not one of the managed-prefs values (empty, standard, intercept-dns). A fresh install fails visibly, instead of silently falling back, because a managed deployment must not provision into an unintended mode. On an upgrade, the same bad value only logs a warning and keeps the existing service mode. So a bad profile edit cannot break an already-working fleet. | Fix InterceptMode in the profile to one of the allowed values. Then reinstall the package. | | SERVICE_RELOAD_FAILED | package | 1 | An upgrade reused the existing service mode (no managed InterceptMode override), and launchctl load on the installed plist failed. | Run sudo launchctl load /Library/LaunchDaemons/ctrld-client.plist by hand, and read the error it gives. | | TEMP_FILE_UNAVAILABLE | package | 1 | The script failed to create the private temp file it uses to capture ctrld's output (/tmp or TMPDIR is full or unwritable). Because ctrld never runs, the script has no diagnostics to add. | Free space in /tmp or TMPDIR, or fix its permissions. Then reinstall the package. |

Customer-visible message shape

Every failure shows the same two-part shape: the fixed identifier line, then one free-text line specific to the code. Neither part ever contains the provision token, the resolver/device ID, or config contents.

  • Input stage — names the offending flag or value and the rule it broke. For example: --cd-org provisioning code is malformed: must be 6-64 characters with no whitespace or control characters. For CUSTOM_HOSTNAME_INVALID, the message names the specific offending characters and the allowed format.
  • Bootstrap, TOKEN_* — one fixed sentence per code (see the table above). For example: the provisioning code has expired; get a new provisioning code from your administrator.
  • Bootstrap, API_UNREACHABLE/API_REJECTED/API_DEVICE_INVALID — When the failure surfaces during a running install's periodic resolver refresh, the message is ControlD API rejected this configuration (HTTP status <n>). When it surfaces during the initial provisioning call, the message is provision token exchange failed: <err> or failed to fetch resolver config: <err>. Neither ever includes the request or response body.
  • Listener — names the address or addresses tried. detail.attempts in the result file carries the OS error for each address and protocol.
  • Service — For SERVICE_INSTALL_FAILED or SERVICE_START_FAILED, the message is <task> failed: <err> (<task> is Install or Start). When the daemon itself reported no fresher failure, SERVICE_SELFCHECK_FAILED shows one of three messages, depending on what the self-check saw. When there is no error and the service is not running, the message is ctrld service did not pass its post-start self-check. When the test query itself errored, the message is An error occurred while performing test query: <err>. When the service was running but the query silently failed, the message is ctrld service was running, but a DNS query could not be sent to its listener; check firewall rules blocking/intercepting/redirecting DNS queries.
  • Package — the plain-language second line the installer script logs right after the identifier line. Each package row's Next action cell above quotes this line verbatim, because the script has no separate result file to carry it.

Where each code surfaces

  • Every ctrld-native code (every row above except package): the CLI output / persisted ctrld.log, provision_result.json, and ctrld-client diag (which reads that same file).
  • Package-stage codes (PROFILE_PREFS_MISSING, SERVICE_RELOAD_FAILED, TEMP_FILE_UNAVAILABLE, the package's own INTERCEPT_MODE_INVALID check on the managed-prefs value, and the package pre-flight's reused PROVISION_TOKEN_MALFORMED and CUSTOM_HOSTNAME_INVALID): only /var/log/install.log. ctrld never ran, so no result file exists yet. sudo ctrld-client diag run right after this reports "none recorded" for the last provisioning result. That is still useful: it shows that the package script's own gate rejected the profile before ctrld had a chance to run.
  • Any ctrld-native code can also reach /var/log/install.log. The postinstall script invokes ctrld in two cases: to provision on a fresh install, or to reapply a managed intercept mode on an upgrade. Either time, it greps ctrld's own identifier line out of the captured output. It re-logs that line, and appends (ctrld exit <n>). So a bootstrap or listener code, not only the three package-only codes, can appear in the MDM install log. Right after any installer-script failure, it also appends a fresh sudo ctrld-client diag run, each line prefixed diag:.

Reading the result file

macOS and Linux (default service home is /etc/controld):

sudo cat /etc/controld/provision_result.json

On Windows the file sits next to ctrld-client.exe in the install directory. A custom homedir config moves it accordingly; routers and mobile use their platform home directory.

The file sits in the same directory as the persisted internal log (ctrld.log) for the user the service runs as. On a healthy install the file is absent.

Support: ctrld-client diag

Run sudo ctrld-client diag (or sudo ctrld-client diag --json) to collect the facts support needs for a provisioning ticket, in one copy-paste-safe command. It collects the client version and the MDM-managed preferences (macOS only, the token reported only as present or absent, an empty value counts as absent). It also collects the last provisioning result, the service state, and whether the Control D API is reachable. It needs no config. It also runs without root, but then the last-provisioning-result and the service-state sections report permission denied, and the text report names the step that gets the rights it needs. Even when it finds a failure, it still exits 0: that failure is part of the report, not a command failure. It never asks for or prints the provisioning token itself.

The provision_result.status field in the JSON report is one of none (no result file), recorded, untrusted (a file that fails the contract check), unreadable (permission denied), or corrupt (a file that could not be parsed).

Rules for maintainers

  • Codes are append-only once released. Never rename, renumber, or reuse a code or exit number; add a new one and note the deprecation here.
  • API rejection reasons are append-only too: tokenFailureCodeForReason maps a fixed set of error.metadata.reason strings to their TOKEN_* code. An unrecognized or absent reason falls back to API_REJECTED (see apiFailureCode). So a new reason string does nothing until both a matching code and a row here exist.
  • Every code added in cmd/cli/provision_result.go needs a row here in the same MR. Tests enforce the code/stage/exit maps and that this table has exactly one stage != package row per ctrld code. The package rows belong to scripts/pkg/postinstall and are not part of that count.
  • Detail must stay bounded and free of secrets: the constructor strips the provision token and cd UID and caps sizes; do not bypass it.
Reference in New Issue View Git Blame Copy Permalink