Troubleshooting the installation

Troubleshooting the installation

Who this is for: the administrator of a self-hosted installation made with . What it covers: where to look first, then faults by symptom, with the command that confirms the cause and the fix. Before you begin you need: shell access to the server, in the release directory (the one with docker-compose.yml).

Self-hosted only. Everything here concerns the server you run. If your problem is a project setting or a right, it is for the project owner: see the .

Look here first

docker compose ps

docker compose logs --tail 100 app

curl -s http://127.0.0.1:3000/api/health
•

docker compose ps shows whether app, db, valkey (and caddy) are up, and how often they restarted.

•

The app log is on standard output. Add --since 1h for a time window, or -f to follow it. The same works for db, valkey and caddy.

•

The health reply names the failing part: "status":"degraded" with checks such as db or valkey set to an error text. "status":"draining" means the app is shutting down. Stack traces in the log point at compiled files, because the image has no application source. Quote log lines as they are when you ask your supplier for help.

•

In the browser, System, Health shows whether the background schedulers run, and Jobs shows background work.

Log messages and what they mean

Message in the app log

Meaning

What to do

[migrate] database not ready (repeating)

The database is still starting, or unreachable. The app waits up to a minute.

docker compose logs db. For an external database, check the host, port, password and firewall.

[migrate] FAILED: cannot enable Postgres extension "vector" (or pg_trgm)

The database role cannot create the extension. Common with a managed database.

Create the extensions as a superuser in that database (CREATE EXTENSION vector; CREATE EXTENSION pg_trgm;), then docker compose restart app. The bundled database does not have this problem.

[migrate] FAILED: with another message

A migration stopped.

Read the message. Fix the cause and restart the app; migrations that were not applied run again. Do not edit migration files.

[DocSearch] FATAL: Required DB objects are missing

The search objects that a migration creates are absent, so migrations did not complete.

Restart the app and watch for [migrate] up to date. If you set SKIP_MIGRATIONS=1, run the migrations once.

[Valkey] FATAL: VALKEY_URL is not set or [Instrumentation] FATAL: Valkey health check failed

Valkey is missing, unreachable, or refuses the password. The app stops on purpose.

docker compose ps valkey; docker compose exec valkey valkey-cli ping; check VALKEY_URL.

AUTH_SECRET is required, ... too short, ... publicly known default

AUTH_SECRET is empty, under 32 characters, or a sample value.

Set a real value (openssl rand -base64 48) in .env, then docker compose up -d. Everybody signs in again.

[Instrumentation] FATAL: SCHEDULERS=...

The SCHEDULERS variable names a scheduler that does not exist.

The message lists the valid ids. Fix or remove the variable.

[Watchdog] Disabled: failed to load ...

The stall detector could not load its native module.

A warning only; the app runs. Report it to your supplier if it persists on the bundled image.

[shutdown] NEXT_MANUAL_SIG_HANDLE is not set

The orderly shutdown is cut short when the container stops.

Add the override in .

The installer says the app did not become healthy

install.sh waits six minutes for /api/health and then prints this. The stack is usually still running.

1.

Read the log for one of the messages above.

docker compose logs --tail 120 app
2.

If the log ends with [migrate] up to date and the server started, wait a minute and test the health address again. A first start on a small host can be slow.

3.

If you set HEALTH_TOKEN, the installer's check gets 401 and always fails. See the caution in the .

4.

Do not edit docker-compose.yml to get past the error. Fix the cause, then run docker compose up -d.

Check: curl -fsS http://127.0.0.1:3000/api/health returns "status":"ok".

The database rejects the app after you changed POSTGRES_PASSWORD

The database image reads POSTGRES_PASSWORD only when it creates a new data volume. Changing the value in .env later does not change the password inside the existing database, so the app gets password authentication failed.

•

Put the old value back in .env, then docker compose up -d; or

•

Change it inside the database and in .env together:

docker compose exec -T db psql -U codedocs -d codedocs -c "ALTER USER codedocs WITH PASSWORD 'NEW_PASSWORD'"

Then set POSTGRES_PASSWORD=NEW_PASSWORD in .env and run docker compose up -d.

HTTPS and the certificate

Symptom

Cause

Fix

The browser cannot connect to https://app.example.com

The domain does not point at this server, or ports 80/443 are blocked

dig +short A app.example.com must print the server's address. Open 80 and 443 in the provider's firewall.

docker compose logs caddy shows challenge errors

DNS is wrong or not yet visible; a stale AAAA record points elsewhere; a DNS proxy answers for the name

Fix DNS (use DNS-only while the certificate is issued), then docker compose restart caddy. Do not retry in a loop: the certificate authority throttles repeated failures for an hour or more.

caddy does not start: port already in use

Another web server holds 80 or 443

Stop it, or use instead of Caddy.

The certificate is requested again and again after a restart

The caddy_data volume was removed

Keep that volume. Losing it re-issues the certificate and runs into the authority's limits.

Check the certificate (subject is your domain, issuer is a public authority, notAfter about 90 days ahead):

echo | openssl s_client -connect app.example.com:443 -servername app.example.com 2>/dev/null | openssl x509 -noout -subject -issuer -dates

Signing in

Symptom

Fix

You lost the admin password and another administrator exists

In System, Users, select admin, enter a New password under Reset Password, press Reset.

You lost the admin password and nobody else can sign in

Clear the stored password and restart the app. At start the app repairs the account and sets its password to the ADMIN_INITIAL_PASSWORD value in .env.

Everybody was signed out

AUTH_SECRET changed. Expected after a rotation; users sign in again.

Passkeys stop working after you moved to a new domain

Passkeys are bound to the domain in PUBLIC_APP_URL. Users register them again.

For the second row:

docker compose exec -T db psql -U codedocs -d codedocs -c "UPDATE users SET password_hash = NULL WHERE email = 'admin'"

docker compose restart app

Check: grep '^ADMIN_INITIAL_PASSWORD=' .env, then sign in as admin with that value and change it.

Emails

Symptom

Cause and fix

A new account shows Account created but failed to send verification email

Outgoing email is not configured or fails. In System, Config, Email, fill the provider, set the From address, press Test Connection. Read the error it prints.

Test fails with a TLS or protocol error on SMTP

Port 465 uses implicit TLS; 587 and others use STARTTLS. Match the port to your provider.

Resend test passes but mail is rejected

The domain of From address is not verified in Resend (SPF and DKIM).

Emailed links point to localhost or the wrong host

Public App URL is wrong or empty. A value saved in Config wins over PUBLIC_APP_URL in .env: fix it in Config.

Inbound mail never arrives

The webhook address <Public App URL>/api/webhooks/inbound/resend is not registered in Resend, or Resend Inbound Signing Secret is missing or wrong.

Files and uploads

Symptom

Cause and fix

Uploading a large file fails at once, with a 413 in the proxy log

client_max_body_size in your proxy is smaller than the file. Raise it to at least Embed File Max Size (MB) (default 100). The bundled Caddy has no such limit.

Uploads fail with Upload failed

Config, Storage & Logs, Test Connection. For S3, check endpoint, region, bucket, keys, and Force path-style addressing for MinIO. Check free disk with df -h for local storage.

Images disappear after a rebuild of the stack

You ran docker compose down -v, which deletes volumes including uploads. Restore from backup. Never use -v on a live installation.

Switching to S3 did not move the old files

Switching affects new uploads only. Use Migrate storage to copy existing files.

Live editing, speed and background work

Symptom

Cause and fix

Editors keep disconnecting, the live inbox stops for a minute

Check docker compose ps: a rising restart count means the app is being restarted. Out of memory is the usual cause: sudo dmesg | grep -i 'killed process'. Add memory or swap. With your own proxy, check the WebSocket headers and the 86400 s timeouts from .

Imports never finish, activity feeds stop, spreadsheet recalculation hangs

System, Jobs and Health show failed or stale work. Check that Valkey is up. With a source install, check that worker-dist/ exists next to the build; the log then shows MODULE_NOT_FOUND for a *.worker.cjs file.

The app answers slowly or hangs, while the proxy is up

curl -s http://127.0.0.1:3000/api/health: pool shows saturation, and degraded names the part. A blocked event loop leaves a stuck_*.trc stack dump in the traces volume. Send the dump and the log lines around it to your supplier.

Document search misses results that match by meaning

The embedding endpoint is not configured or fails. See Embedding Model in the . Keyword search still works.

A connector cannot reach a database on your internal network

Connectors are blocked from private addresses by default. Add CONNECTOR_ALLOW_PRIVATE_HOSTS=true, or list the host in CONNECTOR_HOST_ALLOWLIST, through the override file.

Accounts and plans

Symptom

Fix

A new user hits Plan limit reached at once (2 projects, 20 documents)

The account is on the Free plan. Check that DEFAULT_PLAN_CODE=self_hosted is in .env. It applies to new accounts only; move existing ones in System, Billing (select the account, change the plan to Self-hosted).

Backups are a paid feature

The account's plan has backups off. The Self-hosted plan has them on.

Available plans is empty for a user

No plan is offered for sale and Stripe billing is not set up. Expected on a self-hosted installation.

Disk and logs

•

docker system df shows what Docker uses. Images of old releases stay until you remove them: docker image prune (only unused images).

•

Docker keeps container logs without a size limit by default, so a busy installation can fill the disk. Set a limit once in /etc/docker/daemon.json and restart Docker:

{ "log-driver": "json-file", "log-opts": { "max-size": "50m", "max-file": "5" } }

The setting applies to containers created after the restart; run docker compose up -d --force-recreate to apply it to the stack.

Rolling back an update

There is no downgrade after the migrations ran. To go back, stop the stack, restore the database dump taken before the update into a fresh database volume, and start the previous release. The procedure and the backup contents are in .

Starting over on a failed first install

Only when the installation holds no data you need.

Careful: docker compose down -v deletes the database, Valkey data and uploaded files permanently.

docker compose down -v

./install.sh --domain app.example.com

The existing .env is reused, so the secrets stay the same.

•

•

•

•