SurfSense/docs/architecture/plugin-registry.md

Plugin registry

The registry is the list of third-party plugins, verified and community, the app offers once third-party plugins are allowed. Publishers fill in a submission issue form, which a bot checks and, once a maintainer accepts it, turns into a pull request to plugins/registry/plugins.json, or open that pull request themselves; CI checks it, a maintainer connects it on dev and reviews it, and CI signs the list on dev; the release pull request carries the signed list to main. A small container serves the signed list, its icons and SurfSense's OAuth files at plugins.surfsense.com, and the app fetches it, verifies the signature against keys it compiles in, and keeps the last good copy. What the app does with an entry, connecting and signing in, is in plugins.

Code: plugins/registry/, surfsense_local/backend/modules/plugins/lists/registry/, .github/workflows/ (plugins-registry*.yml), plugin-submit.yml Decisions: ADR 0053

The repository side

Path	Holds
plugins.json	The source list as authors write it: schema_version, plugins and removed. It lists DeepWiki
icons/	One icon per entry that names one
published/	Written by the sign job only: plugins.json, stamped, and plugins.json.sig. Empty until the first signing
check/	A uv project: python -m registry_check static, live, submission, sign or verify
oauth/	client.json and callback.html, SurfSense's OAuth client (below)
Caddyfile, Dockerfile	The registry container

.github/CODEOWNERS names maintainers for /plugins/registry/, so no change merges without one. The publisher's guide is plugins/registry/README.md.

The entry rules

static checks every entry offline and prints one line per problem, naming the entry and the field (entry_rules.py, list_rules.py):

• Identity. An id of lowercase letters, digits and hyphens, kind: remote, verification verified or community. The license belongs to the first-party list alone, and being in that list is what makes a plugin SurfSense's.
• Verified. Nothing more than verification: verified: it is a maintainer's judgement, which no check can make (below). A domain an older entry carries is ignored.
• Display. name and author up to 80 characters, description up to 200. author is the company or project that runs the server, never a person's name; review checks it.
• Server. An https url with no credentials. hosts lists exact lowercase hostnames, none loopback, none twice, the URL's host among them.
• Sign-in. auth.type is none; token (with an optional title, up to 80 characters, and description, up to 200, wording the prompt for the key; without them the app uses its own, translated); headers, naming at least one header the user's key goes in, each {name, prefix?, label?, description?}: a header name the app would accept, none the transport sets and none twice, a prefix such as Bearer of one line up to 32 characters, a label up to 80 and a description up to 200; or oauth (with an optional client_id and scopes, and user_client: true when the provider takes only apps registered in advance, so each user brings their own). No auth field may carry a secret or a value.
• Transport. transport, when given, is streamable_http or sse, the older HTTP+SSE a server offers alone.
• Access. free, or external with an access_note.
• Pages. privacy_policy is required, and it and homepage are https. min_app_version, when given, is a version like 2.3.0.
• The plugin's page. about, when given, is up to 600 characters; tools is required, a list of MCP tool names, each once, since the plugin's page shows them before anyone connects; added, when given, is a month like 2026-10. Review, not the check, makes sure every entry has an about and says the month it was added.
• Ids. Unique in the registry and across the first-party list, which the check reads too; an id in removed is never listed again, and each removed entry gives a reason. Unknown fields are allowed, so a newer app can add them.
• Icons (icon_rules.py). A plain file name in icons/, .svg or .png, at most 32 KB, square; a PNG at least 64×64; an SVG read with defusedxml, with no script, foreignObject, frame or embed, no event attribute and no reference outside itself. An icon no entry uses fails.

Verified

verified is set by a maintainer in review, when they are satisfied that author runs the server, on whatever evidence convinces them: the publisher's own docs linking the server, the submission issue or pull request coming from the publisher's own account, a contact they know, or a proof on the server's domain. The app shows it as a checkmark (ADR 0055). Every other entry is community. Only a maintainer's approval merges a change to the registry (CODEOWNERS), so only a maintainer sets or keeps it.

Contacting the servers

live connects to each entry a pull request adds or changes, compared with the list it is based on, with the MCP client the app uses, the official SDK at the app's minor (live/). An unchanged entry is not contacted again. A pull request's log is public, so a failure is reported as a reason of the check's own, such as "it answered 404", never with what the server answered, and the SDK's logging, which quotes a bad answer whole, is silenced (failure_reason.py). Problems fail the pull request; what MCP leaves optional only warns, so any server the app could use passes:

auth	Problems	Warnings
Any	the URL cannot be reached, or does not answer MCP over Streamable HTTP	none
none	the server asks for sign-in; the server lists a tool tools leaves out	a tool in tools the server does not list without sign-in, which the maintainer confirms signed in; no tools in the entry, naming the server's; a tool with neither readOnlyHint nor destructiveHint
token, headers	none: without a key it is expected to refuse	the server lists its tools without a key; it refuses with another 4xx than 401 or 403, which some key servers answer
oauth	no authorization server found as the app finds one (RFC 9728, then RFC 8414 on the server's origin); a sign-in host, the authorization server's or one of its endpoints', missing from hosts; an authorization server that takes neither SurfSense's client metadata document nor dynamic registration, with no client_id in the entry and no user_client	the server lists its tools without signing in

A server behind a key or an account is not listed by CI, which holds no credentials: the maintainer connects it on dev and checks it offers exactly the tools in tools. The entry's privacy_policy and homepage are not opened: a site often refuses a script, and the reviewer opens the privacy policy anyway.

Submissions

plugin-submit.yml asks the publisher only what a server cannot say about itself: name, URL, publisher, description, about, privacy policy, homepage, icon, who can use it and its tool names, Nothing about keys or OAuth apps: each user pastes their own key, or registers their own app, when connecting. submission reads the issue as GitHub renders the form, one ### <question> per answer (submission/):

• Before contacting anything. Every required answer is given and the no-secrets confirmation is ticked. The URL, and later the icon's, is https on a host that is neither loopback nor a private or link-local address, since a stranger names it for the runner to fetch (public_address.py).
• What the server says. It is contacted as live contacts it (server_report.py). One that lists its tools without a key is auth: none; one asking for sign-in is oauth when it names an authorization server, with every sign-in host added to hosts, and user_client: true, with a note, when it takes neither SurfSense's client metadata document nor dynamic registration; it is token otherwise. One whose POST is refused with 400, 404 or 405 is contacted again over the older HTTP+SSE, as the app does, and transport: sse when that answers; otherwise it is a problem saying to check it is the MCP endpoint. A key server is token: it answers 401 whichever header it reads, so a reviewer sets headers in the pull request when it reads another. The live findings for that entry are its problems and warnings.
• Tools. The form's tool names are the entry's tools, in the publisher's order: the record, since the plugin's page shows them. A tool the server lists without sign-in that they leave out is a problem naming the server's list; one they name that it does not list without sign-in is a note, since some servers list more once signed in. A server behind sign-in lists nothing, so its names are taken as given, with a note. Either way the maintainer confirms them signed in, in review. A key server is auth: {"type": "token"}, with no wording.
• The entry. id from the name, kind: remote, verification: community, added this month, in the order the list keeps (submitted_entry.py). The icon, dropped into the form or linked, is fetched once, at most 32 KB after up to three redirects, and named <id>.svg or .png (icon_fetch.py). The entry rules, the id rules against both lists and removed, and the icon rules then run, each problem named by the form's question rather than the field.
• The comment. One comment, edited on each run, lists what to fix, or the entry and what happens next, with submitted text unable to mention anyone or add markup (submission_comment.py). --apply appends the entry in the list's own formatting and writes the icon beside it (listing.py).

A submission is found by its plugin-submission label or its [plugin submission] title, since an issue opened through the API by someone without write access, as an agent's gh issue create, loses its labels.

Signing

sign reads the source list and writes published/plugins.json with generated_at, the UTC time with a Z, and each icon's icon_sha256, then plugins.json.sig: the bare base64 Ed25519 signature over that file's exact bytes (sign.py). One formatting, two-space JSON with a final newline, makes a re-sign of the same list byte-identical (signed_bytes.py). The key is the raw 32-byte private key in base64, read from REGISTRY_SIGNING_KEY; the source list is never rewritten. verify checks a list against a hex public key as the app does.

CI

Workflow	Runs	Does
plugins-registry.yml, on a pull request	Pull requests into main or dev that touch plugins/registry/ or the first-party list	Fails a hand edit to published/, except from the bot or the release pull request from dev into main; runs the check's lint, format and tests, static, then live against the list the pull request is based on, and the callback page's Node tests. A manual run checks any branch, without live except on a submission branch. A maintainer then connects the listing on dev and reviews its tools, since a server behind a key or a sign-in is not listed by CI
plugins-registry-submission.yml, on a submission issue	A submission opened, edited or reopened	Runs submission on the issue body against dev's list, read access to the code only, labels the issue plugin-submission, and keeps one comment on it. The body reaches the check as a file, never the shell
plugins-registry-submission.yml, on registry: accept	A maintainer labels an open submission registry: accept	Runs submission --apply again on the issue as it is now, commits the entry and icon to registry/plugin-submission-<issue> from dev, force-pushed if accepted again, and opens the pull request into dev with the template's headings and Fixes #<issue>. A failed check comments why and takes the label off. It acts as github-actions[bot] with GITHUB_TOKEN, which needs Actions allowed to open pull requests. A pull request opened that way starts no workflow, so it then runs plugins-registry.yml on the branch itself, live included, and the checks show on the pull request
plugins-registry.yml, on dev	Pushes to dev that change plugins.json or icons/, or a manual run on dev	Its sign job, the only one with write access to the code: runs static, never live, so a third-party server that is down never holds up signing; signs with the REGISTRY_SIGNING_KEY secret, verifies with the public key the app compiles in, and commits published/ to dev itself. Nothing pushes to main: the release pull request carries it there, and the registry container is built from main

The container

The registry is served by a container of its own beside the plugin host, on the same hostname, so updating the list never restarts the plugins. It is pinned Caddy 2.11.4 serving files copied into the image, read-only, as an unprivileged user, with a health check (Caddyfile):

Path	Serves	Cache
/plugins.json, /plugins.json.sig	The signed copies from published/; the source list is never served	5 minutes
/icons/<name>.svg or .png	An icon, with a CSP that sandboxes it and runs nothing	5 minutes
/oauth/client.json	SurfSense's client ID metadata document	5 minutes
/oauth/callback	callback.html, never framed	none
/health	ok	none

Any other path is 404, and any method but GET or HEAD is 405. The website's /plugins page, and each plugin's page at /plugins/<id>, read /plugins.json and /icons/ too, at most hourly, so a listing reaches them with the release that reaches the app, and a new one gets its page on first visit (registry-plugins.ts). docker build -t surfsense-registry plugins/registry builds it.

OAuth files

SurfSense signs in to OAuth plugins as one public client (plugins). client.json is its client ID metadata document: client_id is its own URL, its one redirect is https://plugins.surfsense.com/oauth/callback, its grants authorization_code and refresh_token, and it authenticates with none.

callback.html forwards the browser to the app. The state the app sends is <port>.<nonce>; the page reads exactly one state, checks its shape and that the port is 1024 to 65535, and replaces the location with http://127.0.0.1:<port>/plugins/oauth/callback and the whole query, the provider's error included. Anything else shows "This sign-in link isn't valid". Its CSP allows only its own inline script and style, by hash, and it sends no referrer.

In the app

current_registry.py holds the one answer to which list the app offers.

• When it fetches. Only through GET /plugins/registry or connecting from it, only while third-party plugins are allowed and plugins.surfsense.com is allowed, and at most every five minutes; a failed or refused fetch also waits five minutes. Readers that must not reach the network, readiness, a plugin's row, icons and custom ids, read the copy held (delisted.py, listed_ids.py).
• Consent. host:plugins.surfsense.com is built in, off by default (egress). A refusal carries purpose: "plugin_registry", so the prompt asks about fetching the list (registry_consent.py).
• Verifying. The list and its .sig are fetched from plugins.surfsense.com (registry_fetch.py) and kept only if the signature verifies with one of the public keys compiled into registry_keys.py, a tuple so a release can trust an old and a new key while CI's is rotated. A list with another schema_version, or a generated_at older than the copy held, is ignored and the copy kept, so an old list cannot bring back a removed plugin.
• Entries dropped (registry_document.py, offered_entry.py): one that does not parse; one claiming SurfSense's license (auth: license or access: license), so no fetched file reaches the license key; one taking a first-party id; one whose min_app_version is above this build's, read from the package's metadata (app_version.py, copied into the frozen API by bundling/api.spec); one listed twice or also in removed. Each is logged and the rest are kept.
• The cache. {data_dir}/plugin-registry/ holds the list, its signature and when it was fetched, written last so a copy cut short reads as none (registry_cache.py). Offline the copy is listed with its date; with no verified copy, listing is 502 registry_unavailable.
• Icons. After a new list, each icon is fetched into the cache's icons/ and kept only if its SHA-256 matches the entry's icon_sha256 (registry_icons.py). GET /plugins/registry/icons/{id} serves only a matching copy, never fetching, with a CSP that sandboxes it; anything else is 404 and the screen shows the name's first letter.
• Development. An unfrozen API reads the checkout's plugins/registry/plugins.json and icons, unsigned and with no consent (registry_origin.py). SURFSENSE_LOCAL_PLUGIN_HOST_URL moves entries on plugins.surfsense.com to the local plugin host, as it does the first-party list (served_entry.py).

Tests

• Check: from plugins/registry/check, uv run pytest. tests/ hold each entry rule and icon rule, unique ids across both lists, removed entries, the live check against real MCP servers from the SDK on loopback (live_servers.py): tools matching or not, missing hints, a server down or with no MCP, sign-in asked where none is declared, a key server and a headers one, an entry over SSE, OAuth with its hosts listed or not, no authorization server, no way to register and a client_id, user_client or client metadata document instead, unchanged entries left alone, and an answer carrying credentials never reaching the output; and signing: a list verifies, one changed byte or another key fails, the stamps, stable formatting, and the source never rewritten. The sign job alone also checks the committed list against the production key. oauth/tests/callback.test.mjs runs the callback page: a valid state forwarded, a provider's error forwarded, each malformed state going nowhere, and a CSP that loads nothing. Submissions (test_submission.py, test_server_report.py, test_submission_command.py, test_icon_fetch.py, test_issue_form.py): an entry from a keyless server, tool names required and compared with the server's, a sign-in server's tools as declared, a key server asking nothing, an SSE server and a provider taking only registered apps recorded as such, paid access, each blank question, a taken id, icon rules and a dropped icon, private addresses never contacted, the server report against loopback servers, the comment kept inert, the entry written in the list's formatting, and the form's questions matching the ones read.
• App: test_registry.py runs the routes against a stand-in plugin host signing with a test key (tests/registry_stand_in.py): nothing fetched while third-party plugins are off, the five-minute refresh, a bad signature, an older list and an unknown schema each ignored, the cached copy offline, no copy, entries only the first-party list may hold dropped, the host's consent, icons, the checkout read unsigned, connecting, removed plugins, and custom ids. test_registry_compiled.py checks the production key is compiled in and the host is built in.

Known gaps

• No workflow builds the registry image: plugins.surfsense.com is deployed outside CI, so a merged list reaches it only when someone deploys.
• A delisting reaches the app only when Settings → Plugins lists the registry; nothing fetches it before an agent turn.
• min_app_version has not been tried in a frozen build; only a test that bundling/api.spec copies the package's metadata guards it.