open-notebook/docs/5-CONFIGURATION/database.md
Luis Novo 95f76e5a59
docs: complete env reference; troubleshooting keyed on real error messages (#1461)
* docs: complete env reference; troubleshooting keyed on real error messages

Configuration (docs/5-CONFIGURATION):
- environment-reference.md is now the single complete list, checked
  against the code and the content-core, esperanto, surreal-commands and
  langsmith libraries: adds the undocumented variables, a "Read by"
  column, correct defaults (SURREAL_COMMANDS_RETRY_ENABLED=false,
  WAIT_MAX=60, LANGCHAIN_PROJECT=default, SurrealDB defaults), which
  legacy provider vars Migrate to Database copies, and a table of
  variables nothing reads.
- security.md: PBKDF2 derivation, POST /api/credentials/migrate-encryption
  with the back-up-first / no-downgrade warning, password stored in
  localStorage, full list of unauthenticated paths, real upload endpoint.
- reverse-proxy.md: drop v1.0 legacy sections and the non-existent
  multi-container images; real /api/config output; how API_URL
  auto-detection works; the 100 MB Next.js proxy cap.
- ai-providers.md, ollama.md, openai-compatible.md: one shared setup
  procedure with current UI labels, provider/modality table from
  provider_registry.py, no stale model lists or prices; Ollama
  num_ctx and 180 s ESPERANTO_LLM_TIMEOUT; Azure form limits.
- local-tts.md is the shared Speaches page; local-stt.md keeps only
  STT specifics (Whisper models, CCORE_AUDIO_SEGMENT_MINUTES).
- advanced.md, index.md, database.md: link to the reference instead of
  re-listing variables; fix service names, port changes, backups.

Troubleshooting (docs/6-TROUBLESHOOTING):
- Entries start with the exact message from the code or the en-US
  locale. New processing-issues.md covers queued/failed sources (the
  content-core failure reasons), uploads, search, podcasts (profile
  errors and NOTE hints). ai-chat-issues.md covers timeouts, error
  classifier messages, empty responses, Ollama context, encryption
  key and decryption errors.
- Removed invented entries ("too many connections", "Model is not a
  LanguageModel: None", soft delete, retry vars as a connection fix).

SECURITY_REVIEW.md is marked as a historical January 2026 snapshot of
the credentials UI, with the SSRF and security-warning rows corrected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: address review on configuration and troubleshooting pages

- No predictable keys or passwords: placeholders plus the openssl /
  PowerShell generation commands used by the install docs.
- Every 0.0.0.0 bind and firewall rule (Ollama, OpenAI-compatible
  servers, Speaches, LAN access) restricted to the host and Docker
  networks or the LAN, with an explicit exposure warning.
- Diagnostic commands no longer print secrets.
- systemd drop-in includes [Service]; Traefik middleware attached to
  the router.
- Provider pages link to Connect a provider (#1459) instead of
  repeating steps; Vertex and keyless local providers called out;
  "Add Selected" (a select-all toggle) no longer described as the add
  button; "Manage -> Models" instead of "the Models page".
- Startup database failure described correctly (API down, not the
  Database Connection Failed overlay); worker crash causes, Test
  timeout/404 meanings, .env overrides when moving servers, frontend
  restart from source, privacy claims scoped to self-hosted servers.
- Drop claims about bugs fixed in #1457; restore the ollama.md
  GPU acceleration anchor used by quick-start-local.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: address second review round on configuration pages

- Anthropic Compatible always needs an API key and a Base URL
  (enforced in api/models.py), noted in the provider differences.
- Speaches firewall guidance covers a separate Open Notebook host.
- The password check mirrors get_secret_from_env: a readable,
  non-empty _FILE wins, otherwise the variable; never prints values.
- From source, frontend variables go in frontend/.env.local or the
  shell: `make frontend` runs `next dev` in frontend/, which doesn't
  read the root .env.
- The env listing prints every variable name, without values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: correct the default API path for the frontend; deduplicate

Without API_URL the browser calls <host>:5055 directly (the /config
route auto-detects it); INTERNAL_API_URL only serves requests that hit
the Next.js /api forwarding. Full explanation stays in the environment
reference; the configuration index links to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: upload limit depends on whether API calls go through the frontend proxy

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 23:28:50 -03:00

3.6 KiB

Database - SurrealDB Configuration

Open Notebook stores everything except uploaded files and podcast audio in SurrealDB. The API and the background worker both connect to it with the same five variables. Schema migrations run automatically when the API starts.


Connection settings

SURREAL_URL=ws://surrealdb:8000/rpc
SURREAL_USER=root
SURREAL_PASSWORD=root
SURREAL_NAMESPACE=open_notebook
SURREAL_DATABASE=open_notebook

Set all five explicitly. The API and the job queue fall back to different defaults when a variable is missing (open_notebook versus test for namespace and database), and with mismatched values background jobs never get processed. Defaults and legacy names are in the Environment Reference.

SURREAL_USER and SURREAL_PASSWORD must match the --user and --pass that SurrealDB was started with.


Which SURREAL_URL to use

The host in SURREAL_URL is the database as seen from wherever the API and worker run.

Open Notebook SurrealDB SURREAL_URL
Docker Compose (shipped file) surrealdb service in the same compose file ws://surrealdb:8000/rpc
Docker On the host machine ws://host.docker.internal:8000/rpc (on Linux, add extra_hosts: ["host.docker.internal:host-gateway"] to the service)
From source Docker (make database) or installed locally ws://localhost:8000/rpc
Single-container image Inside the same container ws://localhost:8000/rpc (see Single Container)
Anywhere Another machine ws://<db-host>:8000/rpc

localhost inside a container is the container itself, not your machine. Using ws://localhost:8000/rpc in the compose setup is the most common reason the API won't start.

The shipped compose file publishes SurrealDB on 127.0.0.1:8000 only, so other machines can't reach it. To connect from elsewhere, re-publish the port deliberately (see docker-compose.override.yml.example in the repository root), with real credentials and a firewall or SSH tunnel.


Changing the credentials

With the shipped compose file, put the new values in a .env file next to docker-compose.yml:

SURREAL_USER=<your-db-user>
SURREAL_PASSWORD=<generated-password>

Generate the password yourself, for example with openssl rand -hex 32. Don't reuse an example value.

The compose file passes them to both the surrealdb command and the open_notebook environment, so they stay in sync. Apply with docker compose up -d.


Several instances on one SurrealDB

A SurrealDB server can hold many namespaces, and each namespace many databases. To run several independent Open Notebook instances against one server, give each instance its own SURREAL_NAMESPACE or SURREAL_DATABASE.


When the API can't reach the database

While waiting for SurrealDB, the API logs Database is not reachable yet (attempt n/12) and retries. After the last attempt it logs Database did not become reachable after 12 attempts and CRITICAL: Database migration failed, and the API doesn't start, so the browser can't reach it (Unable to Connect to API Server, or the UI doesn't load at all in the Docker image, where the UI waits for the API). If the database goes away later, while the API is running, the UI shows Database Connection Failed. See Connection Issues → Database Connection Failed.

Backups: Advanced → Backup & Restore.