mirror of
https://github.com/Helldez/BigMoeOnEdge.git
synced 2026-10-19 15:33:46 +00:00
5a07bd6b9a
Some checks failed
ci / changes (push) Has been cancelled
ci / host-windows (push) Has been cancelled
ci / version (push) Has been cancelled
ci / host-macos (push) Has been cancelled
ci / ios-harness (push) Has been cancelled
ci / android-apk (push) Has been cancelled
ci / format (push) Has been cancelled
ci / host-linux (push) Has been cancelled
ci / ui (push) Has been cancelled
A second front-end next to bmoe-cli. bmoe-server keeps one model loaded in one Session and serves a browser UI on 127.0.0.1:8765: chat with the reasoning channel, live engine metrics per token, model management and every engine setting. The settings form is rendered from the parameter table alone (GET /api/params), so a knob added to the engine appears in the UI with no UI change, and the UI names no engine parameter. The same server speaks an OpenAI-compatible /v1/chat/completions, streaming, so any OpenAI client can drive the engine. The API is in docs/server-api.md. Layout. server/ holds HTTP, files and the platform, none of it in core/: conversation, settings_store, event_bus and json_util are pure policy (bmoe_server_core, unit-tested without a model); engine_host owns the one Session; model_library the models folder, the catalog and resumable downloads; planner_adapter offers /api/plan; platform.cpp is the only file with platform conditionals. ui/ is Svelte and TypeScript. The model catalog is data, catalog/models.json. Behaviour. Settings live in layers (defaults, plan, launch flags, user) and only the user's are saved. A chat continues the engine's KV cache when the request extends the previous conversation by one message. With the planner built in every load plans first, with the previous model unloaded so the probes measure the machine and not the model in it. Security model, stated in docs/server-api.md. The server binds to loopback unless --host says otherwise and answers only to loopback host names. A request to /api/* or /v1/* from another site is refused on every method, whether it carries a foreign Origin or is marked Sec-Fetch-Site: cross-site. Loads, unloads and plan measurements are serialised, request bodies are capped at 8 MiB, and a parameter that makes the engine write a file is refused over the API. It does not protect against other local processes or users, and --host on a network has neither authentication nor TLS. Build and release. BMOE_BUILD_SERVER (on by default, off for Android) builds bmoe-server and stages ui/dist and the catalog beside it. CI formats server/ and compile-checks the server on macOS; release-host builds the UI and attaches a bmoe-desktop archive per host target next to bmoe-cli. Tool calling and conversations. /v1/chat/completions takes tools, tool_choice and tool messages and returns tool_calls, streamed or not; the model's own chat template writes the tools and llama.cpp's chat layer parses the calls back, so the server knows no tool syntax. The conversation is the client's: every request carries the whole history and the engine reuses the leading tokens its KV cache holds. Requests that arrive while one runs wait their turn. A coding agent (opencode) fixes a file through it. Hardening. Every POST, PUT and DELETE to /api/* or /v1/* must declare Content-Type: application/json, checked ahead of routing, and the chat view drops form controls and styles from model output: together they stop markup in an answer from posting to the API from the UI's own origin. max_tokens is clamped to the n-predict bound. CI runs the UI's unit tests. The plan's arguments reach /api/plan as the list the plan renders from the parameter table, not as text parsed back. A split model's storage is probed where its experts are. |
||
|---|---|---|
| .. | ||
| models.json | ||