mirror of
https://github.com/chmouel/liseur-sync.git
synced 2026-10-18 04:50:12 +00:00
docs: write down how a fresh device starts up
The batch resolve route joins docs/integrating.md and the OpenAPI description, and ADR-0035 records why a route for many books exists when one for a single book already does. Alongside it, the sequence a device follows on its first day is now spelled out: name everything, ask where each book stands through /v1/heads, then sync normally. Every client wants that order and none of it was written down, which is part of why the first attempt at it took several refreshes to settle.
This commit is contained in:
parent
9d10752c61
commit
9699ca3308
5 changed files with 320 additions and 2 deletions
|
|
@ -90,7 +90,9 @@ identity.
|
|||
and tenant-isolation/concurrency tests. **Implemented.**
|
||||
2. Add an explicit catalog-book resolve operation for sync-capable clients.
|
||||
**Implemented** as `POST /v1/books/{id}/resolve`, which requires both
|
||||
`library-read` and `sync`.
|
||||
`library-read` and `sync`. [ADR-0035](0035-naming-a-shelf-is-one-request.md)
|
||||
adds `POST /v1/books/resolve` for a whole shelf at once; it goes
|
||||
through the same resolver and decides nothing differently.
|
||||
3. Add maintenance backfill and mapping repair for split/merge. **Implemented.**
|
||||
Split and merge repair cover unavailable current files, and
|
||||
`liseur-sync admin backfill-works <user>` pre-resolves a user's whole
|
||||
|
|
|
|||
125
docs/adr/0035-naming-a-shelf-is-one-request.md
Normal file
125
docs/adr/0035-naming-a-shelf-is-one-request.md
Normal file
|
|
@ -0,0 +1,125 @@
|
|||
# ADR-0035: Naming a shelf is one request
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-09-07
|
||||
- **Depends on:** [ADR-0003](0003-catalog-work-identity.md)
|
||||
- **Amends:** [ADR-0003](0003-catalog-work-identity.md), which gave
|
||||
catalog resolution one route for one book. It gets a second shape for
|
||||
many, deciding nothing differently.
|
||||
|
||||
## Context
|
||||
|
||||
Positions and sessions attach to a work, never to a catalog book, so a
|
||||
book with no work on this server can neither send a position nor receive
|
||||
one. `POST /v1/books/{id}/resolve` (ADR-0003) gives a book its work, one
|
||||
book per request.
|
||||
|
||||
That is the right shape for the case it was designed for: a reader
|
||||
downloads a book and the client names it. It is the wrong shape for the
|
||||
other case, which is every device's first day. A phone signing in to an
|
||||
existing account has a name here for *none* of its books, so a library of
|
||||
four hundred needs four hundred requests before anything syncs at all.
|
||||
|
||||
Clients therefore ration it, and rationing has a cost the ration was
|
||||
never meant to pay. Liseur named twenty-five books a run, and a run
|
||||
happened when the reader pulled to refresh. Each refresh named the next
|
||||
twenty-five, fetched their history, and put it on the shelf — so the
|
||||
shelf arrived in batches over several refreshes, each batch landing on
|
||||
top of the last. That is [chmouel/liseur#180], and while the ordering
|
||||
half of it was the client's own bug, the batching half is this route's
|
||||
shape.
|
||||
|
||||
A client cannot fix it by asking faster. Four hundred requests is four
|
||||
hundred round trips against a server that may be a Raspberry Pi behind a
|
||||
domestic uplink, and the reader is watching.
|
||||
|
||||
## Decision
|
||||
|
||||
`POST /v1/books/resolve` resolves up to 500 books in one request.
|
||||
|
||||
**It decides nothing that the single route does not.** Both go through
|
||||
`resolveBookWork`, which is the whole of what resolving means: the same
|
||||
evidence read off the catalog, the same `DecideWorkResolution`, the same
|
||||
per-user mapping. A batch route that could reach a different answer would
|
||||
be a second protocol wearing the first one's name.
|
||||
|
||||
**Each book is resolved in its own transaction, and takes its own
|
||||
answer.** One book the server cannot place settles nothing about the
|
||||
others. This is the load-bearing decision and it is what makes the route
|
||||
worth having: a client naming four hundred books must not lose the three
|
||||
hundred and ninety-eight that worked because two did not.
|
||||
|
||||
**So the response is `200` with per-item results, not a status code.**
|
||||
There is no single status that honestly describes five hundred answers.
|
||||
`error` is the per-item spelling of what the single route answers with —
|
||||
`not_found` for its `404`, `ambiguous` (with the works) for its `409` —
|
||||
and a doubtful title/author match is *not* an error: it comes back with
|
||||
`confidence: "low"` and nothing stored, exactly as it does one at a time.
|
||||
|
||||
**A failure that is the server's, rather than a book's, fails the
|
||||
request.** A database that has gone away would otherwise be reported as a
|
||||
permanent refusal of each book in turn, which a client cannot tell from
|
||||
"this book will never resolve" — and would file away as a settled
|
||||
answer. Only `ErrNotFound` and `ErrConflict` become items.
|
||||
|
||||
**The same scope pair, `library-read` + `sync`.** The reasoning is
|
||||
ADR-0003's unchanged: it reads the catalog and writes the caller's work
|
||||
graph. A batch route with a weaker gate would be a way around the check
|
||||
the single route exists to enforce.
|
||||
|
||||
**`confirmed` applies to the whole batch**, and the documentation says to
|
||||
send it only when there is no reader to ask. Accepting a title/author
|
||||
guess is a question about one book; answering it five hundred at a time
|
||||
is how two different books quietly become one reading history, which is
|
||||
what ADR-0003 exists to prevent. The field is kept rather than dropped
|
||||
because a migration tool has nobody to ask and knows what it is holding.
|
||||
|
||||
**500 ids**, refused above that with `batch_too_large` and the limit, the
|
||||
shape `POST /v1/ops` and `POST /v1/sessions` already use and clients
|
||||
already know how to cut a request against. The bound is not about the
|
||||
body — a list of ids is small — but about the work behind it, which is
|
||||
five hundred short transactions.
|
||||
|
||||
### What is *not* here
|
||||
|
||||
`GET /v1/heads` already answers "where does each of my works stand" for a
|
||||
whole account, and a freshly named library is exactly that question. It
|
||||
needs no server change; it needed a client that knew to ask it. The
|
||||
sequence — batch resolve, then heads, then the ordinary delta feed — is
|
||||
now written down in `docs/integrating.md`, because every client will want
|
||||
it and none of it was spelled out.
|
||||
|
||||
The cursor does not move for a heads seed. `snapshot_seq` belongs to the
|
||||
resync protocol, and the rule that a cursor advances only in the
|
||||
transaction that stores the page it covers is unchanged.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A fresh device names its whole catalog in one request and learns where
|
||||
every book stands in one more. Two round trips instead of four hundred.
|
||||
- A client that predates the route, or a server that does, keeps working:
|
||||
the batch answers `404` and the per-book route is still there. Clients
|
||||
are told to remember the probe rather than repeat it.
|
||||
- The single route stays. It is the right shape for a book the reader
|
||||
just downloaded, and it is the only place a doubtful match can be
|
||||
confirmed for one book.
|
||||
- Per-item errors mean a client must read the results rather than the
|
||||
status. The alternative — a batch that is all-or-nothing — was
|
||||
rejected: one unplaceable book in a library of four hundred is
|
||||
ordinary, and it must not be able to stop the other three hundred and
|
||||
ninety-nine.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- A batch of good ids names each book, once, with the catalog's own
|
||||
evidence, and replays to the same works without creating more.
|
||||
- A mixed batch returns the good books' works alongside `not_found` and
|
||||
`ambiguous` for the ones that failed, and neither refusal leaves
|
||||
anything in the work graph.
|
||||
- A doubtful match returns `confidence: "low"` and stores no mapping.
|
||||
- `library-read`-only and `sync`-only tokens are both refused, and a
|
||||
refused batch maps nothing.
|
||||
- Over 500 ids answers `batch_too_large` with the limit; empty and
|
||||
malformed bodies are `400`, never `5xx`.
|
||||
|
||||
[chmouel/liseur#180]: https://github.com/chmouel/liseur/issues/180
|
||||
|
|
@ -59,6 +59,7 @@ loose ends that must come before any of it, are in
|
|||
| [0032](0032-reader-pages-are-readium-positions.md) | The reader's page is the app's page | Later | Accepted; implemented | Pages counted as Readium positions so the browser and the app name the same page; display only, no server or API change; three recorded patches to the vendored engine; amends [0031](0031-web-reader-footer.md) and [0012](0012-reader-engine-foliate.md) |
|
||||
| [0033](0033-a-device-outlives-its-token.md) | A device outlives its token | MVP | Accepted; implemented | `POST /v1/tokens` inherits a `device_id` the account already carried, `400 unknown_device` otherwise; `account_id` on `GET /v1/token`; the store keeps comparing devices; amends [0016](0016-token-self-introspection.md) |
|
||||
| [0034](0034-live-notifications-say-only-that-something-changed.md) | A live notification says only that something changed | Client | Accepted | `GET /v1/events` as topic-only SSE, published post-commit by the store, authorized per topic; correctness stays in the existing feeds; amends [0007](0007-web-reader.md) |
|
||||
| [0035](0035-naming-a-shelf-is-one-request.md) | Naming a shelf is one request | Client | Accepted; implemented | `POST /v1/books/resolve` for up to 500 books, one transaction and one answer each, `200` with per-item `not_found`/`ambiguous`; same `library-read` + `sync` pair; amends [0003](0003-catalog-work-identity.md) |
|
||||
|
||||
## Convention
|
||||
|
||||
|
|
|
|||
|
|
@ -1161,6 +1161,91 @@ This route needs both `library-read` and `sync`, the only one that does:
|
|||
it reads the catalog and writes your work graph. A catalog-only token
|
||||
gets `403`, and so does a sync-only one.
|
||||
|
||||
#### Naming a whole shelf at once
|
||||
|
||||
One request per book is fine for a book the reader has just downloaded.
|
||||
It is not fine for a device that has just signed in, which has a name
|
||||
here for none of its books — and a book with no work can neither send a
|
||||
position nor receive one, so nothing syncs until they all have names.
|
||||
|
||||
`POST /v1/books/resolve` does the same join for up to 500 books:
|
||||
|
||||
```json
|
||||
POST /v1/books/resolve
|
||||
{"book_ids": ["…", "…", "…"]}
|
||||
```
|
||||
|
||||
```json
|
||||
{"results": [
|
||||
{"book_id": "…", "work_id": "…", "confidence": "high", "created": true,
|
||||
"identifiers": [{"kind": "sha256", "value": "…"}]},
|
||||
{"book_id": "…", "error": "ambiguous", "works": ["…", "…"]},
|
||||
{"book_id": "…", "error": "not_found"}
|
||||
]}
|
||||
```
|
||||
|
||||
Results come back in the order you asked for them.
|
||||
|
||||
Each book is resolved on its own, so one book the server cannot place
|
||||
settles nothing about the others: it takes its refusal in its own entry
|
||||
and the rest of the batch stands. That is why the answer is `200` with
|
||||
per-item results rather than a status code — refusing the batch would
|
||||
throw away the four hundred and ninety-eight that worked.
|
||||
|
||||
`error` is the per-item spelling of what the single route answers with:
|
||||
`not_found` for its `404`, and `ambiguous`, with the works, or
|
||||
`conflict` for its two `409`s. `conflict` means the work graph moved
|
||||
under the write; that book alone is worth sending again, since the next
|
||||
attempt sees the graph that displaced it. The other two are settled
|
||||
answers.
|
||||
|
||||
A doubtful match is *not* an error — it comes back with
|
||||
`confidence: "low"` and nothing stored, exactly as it does one at a
|
||||
time.
|
||||
|
||||
Over 500 ids the whole request is refused with `code: batch_too_large`
|
||||
and the `limit` it will take, so cut the list down and send it again. An
|
||||
empty `book_ids` is a `400`.
|
||||
|
||||
`confirmed` applies to every book in the batch. Send it only if you have
|
||||
no reader to ask: accepting a title/author guess is a question about one
|
||||
book, and the single-book route is where that answer belongs.
|
||||
|
||||
A server that predates this route answers `404` or `405`. Fall back to
|
||||
`POST /v1/books/{id}/resolve` per book, and remember the answer rather
|
||||
than probing on every run.
|
||||
|
||||
### Bootstrapping a fresh device
|
||||
|
||||
A device signing in to an existing account has to do three things before
|
||||
its shelf looks right, and the order matters.
|
||||
|
||||
1. **Name everything.** Batch-resolve the catalog books as above.
|
||||
Anything the reader side-loaded resolves through `POST
|
||||
/v1/works/resolve` with identifiers you compute locally.
|
||||
|
||||
2. **Ask where each book stands.** Everything the account did before
|
||||
this device had a name for a book happened *behind* your cursor, so
|
||||
the delta feed will never mention it. `GET /v1/heads` answers for the
|
||||
whole account at once — the newest op per work — which is exactly the
|
||||
question a device with a freshly named library is asking. Do not
|
||||
advance your cursor from it: `snapshot_seq` belongs to the resync
|
||||
protocol, and a seed is not a pull.
|
||||
|
||||
`/v1/heads` carries only the newest record per work, so if one comes
|
||||
back in a shape you cannot read, ask `GET /v1/works/{id}/positions`
|
||||
for that work alone rather than treating it as "nothing here": an
|
||||
older op behind it may still hold a position you can use.
|
||||
|
||||
3. **Sync normally.** From here `GET /v1/changes?since=…` is enough, and
|
||||
the cursor advances in the transaction that stores the page it
|
||||
covers.
|
||||
|
||||
Steps 1 and 2 are two requests for a whole library. Do them in one go
|
||||
rather than a few books at a time: a reader who sees the shelf fill in
|
||||
batches over several refreshes will refresh again, and every batch that
|
||||
lands stamps its arrival on books that were read weeks ago.
|
||||
|
||||
## KOReader through OPDS
|
||||
|
||||
KOReader can browse the catalog with no plugin at all. Add an OPDS
|
||||
|
|
@ -1211,7 +1296,7 @@ round-trips verbatim to kosync pulls.
|
|||
bug, report it. A refusal a client can recover from also carries a
|
||||
machine-readable `code`, currently only `unknown_work` (see
|
||||
[Pushing](#pushing)).
|
||||
- Batch limits: 500 ops, 1000 sessions per request, 1
|
||||
- Batch limits: 500 ops, 1000 sessions, 500 books per batch resolve, 1
|
||||
MiB body, 16 KiB per `locator`.
|
||||
- Auth endpoints are rate-limited per
|
||||
IP (429 + `Retry-After`).
|
||||
|
|
|
|||
|
|
@ -1955,6 +1955,73 @@ paths:
|
|||
error: { type: string }
|
||||
works: { type: array, items: { type: string } }
|
||||
|
||||
/v1/books/resolve:
|
||||
post:
|
||||
tags: [library, sync]
|
||||
summary: Join many catalog books to your own works
|
||||
security: [{ deviceToken: [] }] # requires BOTH library-read and sync
|
||||
description: |
|
||||
The same join as `POST /v1/books/{id}/resolve`, for a shelf at a
|
||||
time. It exists for the moment a device signs in to an account and
|
||||
has a name here for none of its books: a book with no work can
|
||||
neither send a position nor receive one, so asking one request per
|
||||
book is what made a fresh library arrive in batches.
|
||||
|
||||
Each book is resolved in its own transaction and takes its own
|
||||
answer, so one book the server cannot place settles nothing about
|
||||
the others. The response is therefore `200` with per-item results
|
||||
rather than a status code: no single status could describe five
|
||||
hundred answers, and refusing the batch would throw away the ones
|
||||
that worked.
|
||||
|
||||
Results come back in the order the ids were sent. A refused one
|
||||
carries `error`: `not_found`, `ambiguous` with the works, or
|
||||
`conflict` when the work graph moved under the write — the only
|
||||
one of the three worth sending again.
|
||||
|
||||
`confirmed` applies to the whole batch and should be sent only by a
|
||||
caller with no reader to ask. Accepting a doubtful title/author
|
||||
match is a question about one book, and the single-book route is
|
||||
where the answer belongs.
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
required: [book_ids]
|
||||
properties:
|
||||
book_ids:
|
||||
type: array
|
||||
minItems: 1
|
||||
maxItems: 500
|
||||
items: { type: string, format: uuid }
|
||||
confirmed:
|
||||
type: boolean
|
||||
description: Accept low-confidence title/author matches.
|
||||
responses:
|
||||
"200":
|
||||
description: One result per requested book, in the order asked for
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
required: [results]
|
||||
properties:
|
||||
results:
|
||||
type: array
|
||||
items: { $ref: "#/components/schemas/CatalogResolutionItem" }
|
||||
"400":
|
||||
description: |
|
||||
Malformed or empty body, or more than 500 ids — in which case
|
||||
the answer carries `code: batch_too_large` and the `limit` the
|
||||
server will take.
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: "#/components/schemas/BatchItemError" }
|
||||
"403": { $ref: "#/components/responses/Error", description: "The token lacks library-read or sync" }
|
||||
"413": { $ref: "#/components/responses/Error", description: "Request body too large" }
|
||||
|
||||
/v1/books/{id}/download:
|
||||
get:
|
||||
tags: [library]
|
||||
|
|
@ -2395,6 +2462,44 @@ components:
|
|||
kind: { type: string, enum: [sha256, partial-md5, source, dc, ta] }
|
||||
value: { type: string }
|
||||
|
||||
CatalogResolutionItem:
|
||||
type: object
|
||||
description: |
|
||||
One book's answer within a batch resolve. Either it resolved, and
|
||||
carries the same fields `CatalogResolution` does, or it did not,
|
||||
and carries the reason instead.
|
||||
|
||||
A doubtful match is not a refusal: the work comes back with
|
||||
`confidence: low` and nothing is stored, exactly as on the
|
||||
single-book route.
|
||||
required: [book_id]
|
||||
properties:
|
||||
book_id: { type: string }
|
||||
work_id: { type: string }
|
||||
confidence: { type: string, enum: [high, low] }
|
||||
created: { type: boolean }
|
||||
identifiers:
|
||||
type: array
|
||||
description: The catalog's evidence, strongest first.
|
||||
items:
|
||||
type: object
|
||||
required: [kind, value]
|
||||
properties:
|
||||
kind: { type: string, enum: [sha256, partial-md5, source, dc, ta] }
|
||||
value: { type: string }
|
||||
error:
|
||||
type: string
|
||||
enum: [not_found, ambiguous, conflict]
|
||||
description: |
|
||||
Why this book alone was refused. `not_found` is the batch
|
||||
spelling of the single route's `404`, `ambiguous` of its
|
||||
`409`. `conflict` means the work graph moved under the write
|
||||
and the book is worth asking about again.
|
||||
works:
|
||||
type: array
|
||||
description: With `ambiguous`, the works the identifiers landed between.
|
||||
items: { type: string }
|
||||
|
||||
CatalogBook:
|
||||
type: object
|
||||
description: |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue