mirror of
https://github.com/readest/readest.git
synced 2026-10-19 10:13:29 +00:00
* chore: update agent memories * chore: update agent memories * fix(sync): closing unclosed void tags before XML re-parsing, closes #5271 * chore(agent): update agent memories * fix(sync): preserve block starts for dropped XPointer whitespace * docs(agent): sanitize support notes and correct review findings
This commit is contained in:
Huang Xin
GitHub
parent
7c8a0a468a
commit
d05fb5338a
27 changed files with 5126 additions and 138 deletions
|
|
@ -49,6 +49,7 @@ Index only — one line per memory. Detail lives in the topic files; do not rest
|
|||
- [#5923 storage search auto-fired mid-IME](storage-manager-search-button-5923.md) MERGED #5925; no browser verify (auth-gated)
|
||||
- [#5910 reader menu ignored third-party sync](reader-menu-third-party-sync-status-5910.md) MERGED #5922 via shared useCloudSyncStatus
|
||||
- [#6029 manual sync](kosync-manual-sync-6029.md) MERGED #6034 (9e316c7d7); **BookOrbit ONLY** (chrox rescope, KOSync untouched); autoSync on KosyncEngineConfig not KOSyncSettings; default ON; gate the CALLERS not pushProgress; pulls stay automatic; `detail.provider` addressing; increment-throttling half NOT built; live-server verify PENDING
|
||||
- [#5931 S3 Full Sync never restores Readest-Cloud-restored covers](pr-6116-cover-repair-review.md) MERGED #6116 (0041b7b3e) not device-verified; repair = pull-side fullSync pass so Receive Only heals; #5911 test fake needed a local cover
|
||||
- [#5900 file sync never converged](file-sync-converge-5900.md) MERGED #5905; RULE incremental sync = O(changed), never a whole dir read
|
||||
- [#5883 file sync never moved the live reader](file-sync-live-view-progress-5883.md) MERGED #5886; debounce 15s→5s; never device-tested
|
||||
- [#5839 Qiniu S3 auth on `()` keys](s3-key-rfc3986-wire-encoding-5839.md) MERGED #5849; Qiniu verify pending
|
||||
|
|
@ -58,10 +59,13 @@ Index only — one line per memory. Detail lives in the topic files; do not rest
|
|||
- [koplugin local_present sweep](koplugin-local-present-sweep-noop.md) UNFIXED; fix = rm readest_library.sqlite3
|
||||
- [#5838 koplugin auto sync Wi-Fi prompts](koplugin-auto-sync-no-wifi-bringup-5838.md) MERGED #5848; OP's turn_on shape not fixed by design
|
||||
- [#5625 loadDocument parsererror fallback](loaddocument-xhtml-parsererror-5625.md) MERGED #5630 + foliate#70
|
||||
- [#5271 KO highlights at page 1 / wrong place](kosync-html-fallback-dom-mismatch-5271.md) FIXED in foliate epub.js (void-tag XML repair, PR pending); text/html fallback DOM ≠ crengine (<a/> swallows <p>s); fix = close void tags, re-parse XML; luajit probe recipe
|
||||
- [crengine XPointer oracle](crengine-xpointer-oracle.md) luajit dumps real-engine XPointers per word, vitest checks xcfi both ways; found+FIXED text().N cumulative read, single-spine DocFragment, whitespace-collapsed offsets
|
||||
- Resolved/stable → [Sync Fixes](sync-fixes.md)
|
||||
|
||||
## Build, Testing & CI
|
||||
- [tauri fork bump: exclude + swift-rs relabel](tauri-fork-bump-workspace-exclude-swift-rs.md) MERGED #6081; root must EXCLUDE packages/tauri; `patch not used` = fork silently dropped; NEVER orphan a pinned fork commit; tao patch dropped; upstream 0.37.0 CarPlay VERIFIED on iOS 18.5 sim (Now Playing rendered, no crash); packages/tao submodule REMOVED (a362ac598, unpushed); iOS 26 sim + device unchecked
|
||||
- [Ad-hoc visual checks need theme tokens](adhoc-visual-check-daisyui-theme-tokens.md) a scratch browser test gets daisyUI defaults (`--depth: 1`) = phantom input hairlines; inject `themeVariables`
|
||||
- [tauri fork bump: exclude + swift-rs relabel](tauri-fork-bump-workspace-exclude-swift-rs.md) MERGED #6081; root must EXCLUDE packages/tauri; `patch not used` = fork silently dropped; NEVER orphan a pinned fork commit; tao patch dropped; upstream 0.37.0 CarPlay VERIFIED on iOS 18.5 sim (Now Playing rendered, no crash); packages/tao submodule REMOVED PR #6085; iOS 26 sim + device unchecked
|
||||
- [TypeScript 7 upgrade #5260](typescript-7-upgrade-5260.md) MERGED #5893; no tsserver/tsgo (lint = `tsc`); next 16.3.3; rootDir fix
|
||||
- [Nix FOD hash staleness](nix-fod-hash-staleness.md) MERGED #5779; hash from the PR check's `got:` line, NEVER docker/OrbStack; `--keep-going` since #6081; PR head may be chrox/readest-app (remote `chrox`)
|
||||
- [git push needs the SOCKS proxy](git-push-socks-proxy.md) ssh ProxyCommand only; `--no-verify` + ServerAliveInterval
|
||||
|
|
@ -93,13 +97,16 @@ Index only — one line per memory. Detail lives in the topic files; do not rest
|
|||
- Translation: [#1582 markup lost](translation-inline-markup-1582.md) OPEN, deepl CORRUPTS · [en→zh providers](translation-providers-device-verification-2026-08.md) #5913 · [#5772 iframe observer](translation-iframe-observer-5772.md) INDEX-COUPLED · [#5600 PDF quota toast](pdf-translation-quota-toast-5600.md) OPEN
|
||||
- [#6071 PDF TTS skips/loops](fixed-layout-missing-primaryindex-tts-6071.md) FIXED + Xiaomi-VERIFIED; ResizeObserver re-render wiped .textLayer under live TTS ranges; guard = renderedFor signature in pdf.js (submodule, needs PR + re-pin)
|
||||
- [#5011 Ctrl/Cmd+R read whole book](read-aloud-selection-onetime-5011.md) Xiaomi-VERIFIED + FIXED #6076; `handleSpeakText(oneTime=false)` default; toolbar btn was right ONLY because React passes a truthy MouseEvent
|
||||
- [iOS TTS support mail + landing docs recipe](landing-docs-support-reply-tts-ios.md) Edge is the silent default; voice ONLY in the player; landing PR #35 MERGED
|
||||
- TTS: [#5755 lyric view](tts-lyric-view-5755.md) #5908+#5909 · [#5690 download queue](tts-download-queue-5690.md) · [#5750 pause](tts-pause-inconsistency-5750.md) · [#5414 Edge silence iOS](edge-tts-baked-silence-ios-native-5414.md) · [#5230 Edge stall](edge-tts-tauri-ws-hang-5230.md) · [word highlight vs ruby](tts-word-highlight-wordlens-ruby-collapse.md)
|
||||
- [MO narration rewinds on Android](media-overlay-android-rewind-stall.md) MERGED #6058 (cef42c907); 0.12.6 put Android on native ExoPlayer; JS stall > next clip = seek BACK to clipBegin; Xiaomi repro = CDP busy-loop + MediaCodec flush
|
||||
- Narration: [#5480 Media Overlays](media-overlay-narration-5480.md) UNFIXED · [#5562 iOS AVPlayer](media-overlay-ios-native-playout-5562.md) · [Audiobookshelf 1+2](audiobookshelf-integration-phase1.md) · [#5863 transport+WebP](abs-audio-transport-5863.md) · [#5807 read-along](abs-read-along-5807.md)
|
||||
- [PDF selection ignores eink inversion](eink-pdf-textlayer-selection-accentcolor.md) Leaf5-VERIFIED; pdf.js `.textLayer ::selection` outranks ours; OP likely never had E-Ink Mode ON
|
||||
- e-ink: [`[class*=]` matchers](eink-class-substring-matchers.md) fire on variants, beat inline styles · [#5795 per-device CSS](eink-per-device-css-data-eink-5795.md) Boox verify PENDING · [#5667 highlight invisible dark](eink-highlight-difference-mask-5667.md) transientHighlight UNFIXED
|
||||
- CSS/layout: [#5711 fixed-attachment](css-fixed-attachment-negative-margin-5711.md) OPEN · [#5641 FXL autosizing](fxl-chrome-android-text-autosizing-5641.md) · [#5582 SE word gaps](se-text-wrap-pretty-justify-5582.md) · [#5662 Alert sizing](alert-flex-item-content-sizing-5662.md) w-full · [#5852 TOC headings](toc-multiline-headings-5852.md) min-w-0
|
||||
- Rendering: [#5924 RTL blank pages](rtl-skip-link-blank-pages-5924.md) · [#5918 AZW3 garbled](azw3-loadraw-concurrency-5918.md) loadRaw races · [#5745 CBZ order](cbz-split-folder-page-order-5745.md) · [#5822 PDF page labels](pdf-page-labels-reference-pages-5822.md) · [#5635 Auto Scroll frozen](autoscroll-progress-relocate-maxwait-5635.md) OPEN
|
||||
- [#5790/#5930/#5943 PDF highlights invisible in dark mode](overlayer-blend-mode-follows-page-not-theme-5790.md) MERGED #6066 (53c0c1686); blend mode must follow the PAGE not the theme; mix-blend-mode does not cross the iframe in WebKit so Apple hid it
|
||||
- [#6089 swipe anywhere to dismiss a sheet](sheet-swipe-dismiss-anywhere-6089.md) MERGED #6097 NOT device-verified; Dialog + useSwipeToDismiss = TWO sheet impls; useDrag shield leaked on touchcancel
|
||||
- [#6036 selection toolbar off by gridcell.left](annotation-popup-fixed-wrapper-offset-6036.md) MERGED #6068 (65287bd3e); `fixed` wrapper vs cell-relative coords; popups = cell coords, handles = window coords
|
||||
- [Reader overlay z-layers](annotator-overlay-z-layers.md) selection toolbar z-[43] < handles z-[44] < z-[45] panels < z-50 popups/dialogs; the toolbar is the ONE surface that must sit BELOW the handles (it opens on the selection, so it overlapped and swallowed the end handle after #6013) and that broke the Android E2E lane (MERGED #6036, 88ea2de55); the `div.cursor-grab` helper query also counted #6031 highlight strip; iOS native grabbers are UIKit ABOVE the web layer, UNSTACKABLE; suppressNativeSelectionHandles() removes them without spending the selection; iOS verify PENDING
|
||||
- [#5987/#5957 new annotations invisible after Annotate](annotations-hub-scroll-to-new-note-5987-5957.md) MERGED #6013 (6df90139d); fix = editor moved ONTO the selection (popup + snapHeight 0.6 sheet) NOT hub scrolling; Insert-into-Notebook REMOVED; chrox: ALWAYS sort by CFI
|
||||
|
|
@ -110,7 +117,10 @@ Index only — one line per memory. Detail lives in the topic files; do not rest
|
|||
- [#5939 edge gestures vs mid-touch selection](edge-gesture-selection-after-touchstart-5939.md) MERGED #5958; re-check selection EVERY move + `scrollLocked`; autoscroll-speed twin UNFIXED
|
||||
- [#5979 Steam Deck gamepad double inputs](gamepad-support-toggle-5979.md) MERGED #6027 (b12932da9) UNVERIFIED on a Deck; toggle ONLY, remapping skipped; gate MUST sit in ReaderContent (only consumer of BOTH gamepad hooks) and read the STORE not the `settings` prop
|
||||
- [#6048 reading ruler on e-ink](reading-ruler-eink-6048.md) PR #6055 UNVERIFIED on device; drop the sepia band tint on B&W eink, crisp outline; surround-tint + opacity-threshold outline DECLINED; `eink-bordered` forces bg-base-100 so it is WRONG for see-through overlays
|
||||
- [#6088 `[xml|lang]`/`:lang()` dead on xml:lang](xml-lang-namespace-selector-6088.md) MERGED #6090 (04379b8f8); `xml` prefix never declared; body{color} loss BY DESIGN; Use Book Layout ON skips paragraph-layout styles and body{line-height:unset}
|
||||
- [#6047 embedded book fonts](embedded-font-generic-rewrite-6047.md) MERGED f5ec1e5f3 (+24cc21e/cdc9305 review fixes); rewrite generic families PER LIST ITEM (park `!important` first); `!important` does NOT beat specificity, swap the pre/code selector with the toggle; assert the RESOLVED cascade in a browser test; fork-PR push recipe inside
|
||||
- [#6107 two-column curl hinged at the spine](two-column-spine-curl-6107.md) REVIEWED not merged; iOS-only cover; 1 test fails deterministically locally; Xiaomi = paper-back+fade fallback
|
||||
- [#6079 bottom dead zone](page-nav-a11y-bottom-dead-zone-6079.md) hidden a11y nav buttons; 0.12.6 lacked #6001; `h-4` LOST to `h-20` by Tailwind sort; MERGED #6102 (f5399cdcc) drops the Android carve-out entirely, hidden = pointer-events-none ALWAYS
|
||||
- Reader chrome: [#5938 header/footer style](header-footer-style-5938.md) MERGED #5960 · [#5652/#5634 header vs footer](reader-header-footer-dedup-5652-5634.md) · [#4977 top bar blocks selection](header-trigger-overlaps-text-4977.md) · [#5888 cross-page selection](cross-page-selection-edge-turn-5888.md) · [#3772 custom shortcuts](custom-shortcuts-3772.md) · [#5501 Apple Pencil](apple-pencil-page-turner-5501.md)
|
||||
- Annotator: [#5538 resize orphan bubble](highlight-resize-orphan-note-bubble-5538.md) drag-race UNFIXED · [#5585 Instant Dictionary](instant-dictionary-deselect-5585.md) clear isTextSelected FIRST · [Proofread gate](proofread-gate-reflowable-formats.md) born dead
|
||||
- Images/PDF: [#5633 iOS zoom blurry](ios-imageviewer-zoom-blur-5633.md) TableViewer UNFIXED · [#5250 invert img](invert-img-dark-override-5250.md) PR #5383 · [#5813 cover full screen](book-cover-fullscreen-viewer-5813.md) · [#5776 title/series attrs](book-meta-data-attrs-5776.md)
|
||||
|
|
@ -118,12 +128,15 @@ Index only — one line per memory. Detail lives in the topic files; do not rest
|
|||
- [#6003 calibre OPDS pubdate](opds-calibre-pubdate-vs-date-added-6003.md) fix = foliate-js#88 MERGED (799ab10) + readest#6008 OPEN; foliate-js#10 flipped atom:published above dc:date; #5477 made it CORRUPT the library; calibre vs Calibre-Web mean OPPOSITE by atom:published
|
||||
- [#5982 import from ReadEra](readera-import-5982.md) MERGED #6032 (8b8fbe14d) UNVERIFIED on device; ReadEra doc_md5 = FULL-file md5, matched when titles decide nothing; per-book dialog row; the export is `.bak` (a plain zip), NO book files (match by title/author); ReadEra XPointers carry an EXTRA `body` (`/body/DocFragment[N]/body/body/...`) that MUST be stripped; anchor = text search -> XPointer -> section start; page-only locators map to a section ONLY on paged books
|
||||
- OPDS/koplugin: [OPDS fixes](opds-fixes.md) aggregator · [#5583 format filter](opds-download-format-filter-5583.md) PR #5593 · [#5645 self-update crash](koplugin-selfupdate-unpackarchive-5645.md) unpackArchive DROPPED upstream
|
||||
- [#6109 proofread panel](proofread-panel-design-pass-6109.md) MERGED 2e021e4c3; `toggleRule` dropped `scope` so library-scoped rules were never written; hidden toggle was the symptom
|
||||
- Known-unfixed: [#4584 tap-death](issue-4584-tap-death-investigation.md) WebView-148 · [#5353 italic clipped](italic-synthetic-oblique-clip-5353.md) · [#5749 iOS hyphenation](hyphenation-engines-5749.md) WebKit dict SEALED · [virtuoso first-paint blank](mobile-sheet-virtuoso-first-paint-blank.md)
|
||||
- [PR #5389 library full-text search review](pr-5389-library-search-review.md) plan in .agents/plans
|
||||
- Resolved/stable → [Reader Feature Fixes](reader-feature-fixes.md)
|
||||
|
||||
- [Storage vs customization entitlement split](storage-customization-entitlement-split.md) MERGED #5996 DEPLOYED; Stripe SKU LIVE $19.99 sells sync/TTS/email-in NOT themes+fonts; PREMIUM_PLANS excludes purchase BY DESIGN; grandfather = synthetic payments row; SELF_HOSTED unlocks all
|
||||
- [#6091/#6093 self-hosted Docker](selfhosted-docker-6091-6093.md) MERGED #6100 (731b13ab6); /api/* body clone TRUNCATES past proxyClientMaxBodySize; Premium chip keyed on sign-in not entitlement; image now ENVs SELF_HOSTED=true
|
||||
- [Move a storage purchase between accounts](storage-purchase-account-transfer.md) reassign the store row + a synthetic audit row on the loser; `scripts/db/transfer-storage-purchase.mjs`
|
||||
- [Merge one account into another](account-merge-recipe.md) `scripts/db/merge-accounts.mjs` MERGED #6121 (64d594380); files = R2 copy + key rewrite; stamp updated_at on configs/notes/stats
|
||||
- [Yearly subscriptions + plans grid](yearly-subscriptions-store-front.md) MERGED #5989; Stripe yearly prices on EXISTING Plus/Pro products (getSubscriptionPlan reads product.metadata.plan); App Store/Play yearly SKUs NOT created; prices.list needs limit:100 + month-before-year
|
||||
- [#1750 Notion sync](notion-sync-pr-5949-review.md) MERGED #5949 as 295d6e79; 4 defects SHIPPED UNFIXED, worst DELETES the user's own Notion blocks (remoteNoteGroups slice); needs a follow-up issue
|
||||
|
||||
|
|
@ -132,6 +145,7 @@ Index only — one line per memory. Detail lives in the topic files; do not rest
|
|||
- [#5959 updated EPUB imports as a duplicate](duplicate-book-calibre-uuid-5959.md) MERGED #5961 (053aba67f) UNVERIFIED on device; metaHash is a WIRE KEY (server + koplugin meta_hash_v1), never change its output
|
||||
- [#5935 group by reading status](status-grouping-i18n-5935.md) MERGED 82658d8ed; grouping MUST partition (414/750 fell out, Unread held 1); `BooksGroup.localized` gates `_()`; fork push = NEVER the worktree:new head
|
||||
- [#5148 no overscroll on mobile = LIBRARY grid](overscroll-library-not-reader-5148.md) MERGED #5867; NEVER for foliate view; goToFraction(1) marks FINISHED
|
||||
- [#6104 bookshelf clicks reopened the URL-launched book](deeplink-coldstart-hijacks-spawned-windows-6104.md) getCurrent() is PROCESS-global + never cleared; each new reader window re-read it; TWO paths: desktop new-window + mobile Android sticky intent/iOS WebContent respawn; fix = per-run SET marker on cold-start reads only (live path never deduped; 10s window was a dead end); MERGED #6111, NOT device-verified
|
||||
- [#5775 in-app web browser as book source](in-app-browser-book-source-5775.md) MERGED #5870; iOS flow never verified; target=_blank bug OPEN
|
||||
- [Windows clip/browser window 0x8007139F](webview2-env-options-scrollbar-clip-window.md) MERGED #5873; every extra WebviewWindow MUST set FluentOverlay
|
||||
- [#5837 backup exports orphan book dirs](backup-orphan-book-files-5837.md) MERGED #5851; `readDirectory('', 'Books')` MISSES the Rust walk
|
||||
|
|
|
|||
31
apps/readest-app/.claude/memory/account-merge-recipe.md
Normal file
31
apps/readest-app/.claude/memory/account-merge-recipe.md
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
---
|
||||
name: account-merge-recipe
|
||||
description: "How to merge one Readest account's cloud data (books, configs, notes, stats, replicas, R2 files) into another account of the same person; scripts/db/merge-accounts.mjs (MERGED #6121, dry run by default)"
|
||||
metadata:
|
||||
node_type: memory
|
||||
type: project
|
||||
originSessionId: b7f213ca-a1ea-4168-943a-0d8e08ead9cb
|
||||
modified: 2026-09-07T13:56:44.698Z
|
||||
---
|
||||
|
||||
**General account-merge procedure:**
|
||||
1. Confirm ownership of both accounts and identify the source and destination. Inspect their data and purchases with the read-only script, then review the merge dry run.
|
||||
2. Preserve purchases on the destination; data merging does not transfer purchase entitlements. Handle any required purchase transfer separately using [[storage-purchase-account-transfer]].
|
||||
3. Copy source R2 objects to destination keys and verify their sizes before re-pointing database rows. Resolve overlapping records by their update timestamps and refresh sync timestamps so destination devices receive moved data.
|
||||
4. Delete source objects only after verified copies and row updates. Recompute storage usage for both accounts, then verify destination data, purchase entitlements, quota, and device sync. Inspect dangling file rows separately before any cleanup.
|
||||
5. Sign out and back into the destination on each device to refresh account state and sync the preserved local library.
|
||||
|
||||
**Scripts (MERGED #6121 (64d594380), in `scripts/db/` on main, need `node --env-file=.env --env-file=.env.local`, must live in the repo so ESM finds `@supabase/supabase-js` and `aws4fetch`)**:
|
||||
- `inspect-accounts.mjs <email>...` read-only: GoTrue identities, per-table counts, plans, payments, book titles for ownership checks, files rows vs actual R2 objects, cross-account overlap.
|
||||
- `merge-accounts.mjs --from <email> --to <email> [--apply]`: R2 CopyObject + HEAD size verify -> re-point rows -> delete source objects -> recompute `plans.storage_usage_bytes` on both.
|
||||
- The `transfer-storage-purchase.mjs` from [[storage-purchase-account-transfer]] was never committed and is gone; rebuild from that memory if a purchase must move.
|
||||
|
||||
**Facts the merge depends on:**
|
||||
- `files.file_key` is `${user_id}/Readest/Books/<hash>/<name>` and UNIQUE, so a merge is a real R2 copy plus a key rewrite, not just a `user_id` update. R2 has no rename.
|
||||
- `books` pull keys on `synced_at`, which a BEFORE INSERT/UPDATE trigger stamps `now()`, so re-pointing `user_id` alone reaches every device on the target. `book_configs`/`book_notes`/`stat_*` pull on `updated_at > cursor`: a device already signed into the target never sees a moved row with an old timestamp, so the script stamps `updated_at = now()` on those (stat pushes are server-stamped anyway; for configs/notes it only reaffirms the row that already won).
|
||||
- Same primary key on both sides: newer `updated_at` wins, loser is deleted on the target / left on the source.
|
||||
- `plans.storage_usage_bytes` == SUM(`file_size`) over live `files` rows. No writer exists in the repo, so prod maintains it by trigger; the script recomputes both users last so it lands where the app would put it.
|
||||
- **Dangling files rows are common**: `pages/api/storage/upload.ts` inserts the row BEFORE returning the signed URL, so a PUT that never completes (over quota, network) leaves a live row with no object. The merge leaves them behind; nothing in the app garbage-collects them (worth a follow-up).
|
||||
- Sign-out (`AuthContext.logout`) keeps the local library on disk, so after the merge, sign out and sign into the destination account on each device; a fresh sign-in pushes the full local state, so any progress written to the old account after the merge is not lost. Both accounts show the new quota only after a token refresh.
|
||||
|
||||
See [[storage-purchase-account-transfer]] and [[apple-iap-lost-storage-purchase-restore-verify]].
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
---
|
||||
name: adhoc-visual-check-daisyui-theme-tokens
|
||||
description: "A throwaway *.browser.test.tsx that imports globals.css renders with daisyUI's DEFAULT tokens (--depth: 1), painting inset hairlines above every .input/.select that the real app never shows - inject themeVariables' token block before screenshotting or you will 'fix' phantom bugs in shared primitives"
|
||||
metadata:
|
||||
node_type: memory
|
||||
type: project
|
||||
originSessionId: 36e8617f-0012-4044-b2b6-5059e5ad403c
|
||||
modified: 2026-09-07T10:05:06.526Z
|
||||
---
|
||||
|
||||
2026-09-07, while redesigning the Proofread Rules panel (PR #6109). Rendering a
|
||||
component in a scratch `src/__tests__/**/zz*.browser.test.tsx` that only does
|
||||
`import '@/styles/globals.css'` is **not** what the app looks like. Readest pins
|
||||
daisyUI's shape tokens per theme in `src/styles/themes.ts` (`themeVariables`),
|
||||
and nothing applies them unless a theme is mounted. Without them you get
|
||||
daisyUI's defaults — notably `--depth: 1`, which turns `.input` / `.select`'s
|
||||
`box-shadow: 0 1px ... inset` into a visible hairline above every field.
|
||||
|
||||
I read that hairline as a real defect in `SettingsInput` / `SettingsSelect`
|
||||
(DESIGN.md §5 does say the chromeless recipe must kill box-shadow, and the
|
||||
primitives only suppress it on `:focus`) and was about to patch both shared
|
||||
primitives — a change that would have touched every settings panel. The app
|
||||
ships `--depth: 0`, so the resting shadow is transparent and there was nothing
|
||||
to fix.
|
||||
|
||||
**How to apply** — before `toMatchScreenshot`, set the token block on
|
||||
`document.documentElement`:
|
||||
`--radius-selector: 1.9rem, --radius-field: 0.5rem, --radius-box: 1rem,`
|
||||
`--size-selector: 0.25rem, --size-field: 0.25rem, --border: 1px,`
|
||||
`--depth: 0, --noise: 0` (copy from `themeVariables` in `src/styles/themes.ts`).
|
||||
Set `data-eink='true'` on the same element for the e-ink pass.
|
||||
|
||||
Two more traps in the same harness: the test iframe is only ~415 CSS px wide,
|
||||
so a `.modal-box` forced wider than that gets **cropped**, not scaled — check
|
||||
mobile width and screenshot inner sections (`ul`, a single row) for detail;
|
||||
and `position: static` on the `.modal-box` to un-clip it reflows the whole
|
||||
dialog. Un-clip with `height/max-height` + `inset-inline-start` overrides
|
||||
instead. Delete the scratch test **and** its `__screenshots__` dir when done.
|
||||
|
||||
Related: [[vitest-screenshot-baseline-relative-path]],
|
||||
[[settings-panel-screenshot-via-playwright]], [[daisyui-v5-tailwind-v4-migration]].
|
||||
|
|
@ -10,7 +10,7 @@ metadata:
|
|||
|
||||
Driving the on-device Readest WebView via CDP to run JS probes/benchmarks **inside the live app** (no rebuild) — used for the NativeFile/RemoteFile I/O study ([[android-nativefile-remotefile-io]]).
|
||||
|
||||
**Setup:** app must be running → `adb shell cat /proc/net/unix | grep webview_devtools_remote_<pid>` → `adb forward tcp:9222 localabstract:webview_devtools_remote_<PID>`. Discover targets with a Node `http.get` to `/json/list` (set header `Host: localhost`) — **curl mishandles the WebView's HTTP framing and hangs/returns empty**. Connect `ws://127.0.0.1:9222/devtools/page/<id>`, then `Runtime.enable` + `Runtime.evaluate {expression:'(async()=>{...})()', awaitPromise:true, returnByValue:true}`. Helper scripts kept in `/tmp/cdp/` (eval.mjs, disc.mjs).
|
||||
**Setup:** app must be running → `adb shell cat /proc/net/unix | grep webview_devtools_remote_<pid>` → `adb forward tcp:9222 localabstract:webview_devtools_remote_<PID>`. Discover targets with a Node `http.get` to `/json/list` (set header `Host: localhost`), or with **`curl --noproxy '*'`**. The earlier note here blamed curl for "mishandling the WebView's HTTP framing" — that was wrong: this shell exports `http_proxy=https_proxy=http://127.0.0.1:8118` (see [[git-push-socks-proxy]]), so a bare `curl http://127.0.0.1:9333/json` is proxied and comes back EMPTY with exit 0. With `--noproxy '*'` both `/json` and `/json/version` return normal JSON (verified 2026-09-06 against the 0.12.6 build on the Xiaomi). Reach for a Node helper only when you need the WebSocket anyway. Connect `ws://127.0.0.1:9222/devtools/page/<id>`, then `Runtime.enable` + `Runtime.evaluate {expression:'(async()=>{...})()', awaitPromise:true, returnByValue:true}`. Helper scripts kept in `/tmp/cdp/` (eval.mjs, disc.mjs).
|
||||
|
||||
**Getting a debuggable build at all (2026-08, Xiaomi 13):**
|
||||
- A **store/release APK has NO `webview_devtools_remote_<pid>` socket** — there is nothing to forward to. Rebuild+install with `pnpm dev-android` (adds `--features devtools`), then re-read `/proc/net/unix`.
|
||||
|
|
@ -21,6 +21,8 @@ Driving the on-device Readest WebView via CDP to run JS probes/benchmarks **insi
|
|||
- Watch for duplicate labels across mounted-but-hidden dialogs: several Readest import dialogs each render an "Import"/"Cancel" button, so `find(b => text === 'Import')` can hit the wrong one. Anchor on a sibling unique to the target dialog (e.g. the "Back" button) and search within its row.
|
||||
|
||||
**Gotchas that burned time:**
|
||||
- **A stale CDP client wedges the whole endpoint.** One hung node/WebSocket client holds the WebView's single debugger slot, and then `/json/list` returns EMPTY over curl (exit 0, no body) and every new `ws://` connect hangs with no error. It looks exactly like "CDP is broken on this build". Fix: `pkill -f <your-client>` first, then re-discover. Verified 2026-09-07 after three failed probes in a row.
|
||||
- **A long build locks the screen and refreezes the page.** After a ~10 min `pnpm dev-android`, the device auto-locks, so `/json/list` comes back empty again for the reason above the fold (`visible:false`). `input keyevent KEYCODE_WAKEUP` + a swipe, then `svc power stayon true`, BEFORE probing. Revert `stayon false` when done, and read `settings get system screen_off_timeout` first if you plan to change it — there is no way to recover the original value afterwards.
|
||||
- **Locked device freezes `fetch`.** When the screen is locked the page is `visible:false`; Chromium freezes the network task queue so EVERY `fetch()` (same-origin and asset) hangs forever — but Tauri `invoke()` still resolves. Must have the user **unlock + keep Readest foregrounded**. Set `svc power stayon true` + `settings put system screen_off_timeout 1800000` after unlock (revert `stayon false` when done).
|
||||
- **`visible:false` also throttles `setTimeout`** (background timer coalescing → ~60 s). Don't rely on setTimeout guards in probes when the page may be hidden; `invoke`-only probes still work hidden.
|
||||
- `window.__TAURI_INTERNALS__` is ALWAYS injected (independent of `withGlobalTauri`) → use `.convertFileSrc(path)` and `.invoke(cmd,args)` from injected JS. Android asset URL = `http://asset.localhost/<encodeURIComponent(path)>`.
|
||||
|
|
|
|||
60
apps/readest-app/.claude/memory/crengine-xpointer-oracle.md
Normal file
60
apps/readest-app/.claude/memory/crengine-xpointer-oracle.md
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
---
|
||||
name: crengine-xpointer-oracle
|
||||
description: "Local crengine XPointer oracle (apps/readest.koplugin/scripts/xpointer-oracle.lua + src/__tests__/utils/xcfi.crengine-oracle.test.ts): dumps the real engine's XPointer for every word of an EPUB and checks xcfi.ts both ways; first run found and fixed three xcfi.ts semantics gaps (unindexed text().N was read cumulatively, single-spine /body/DocFragment/ rejected, offsets counted raw not whitespace-collapsed) — 2026-09-07"
|
||||
metadata:
|
||||
type: project
|
||||
originSessionId: 85280a2e-03e0-46da-8144-1e680cc6af56
|
||||
modified: 2026-09-07T14:31:57.186Z
|
||||
---
|
||||
|
||||
Built 2026-09-07 after #5271 (chrox: "for now only local validation, we may want to ship a cre bin
|
||||
in the future"). No crengine is compiled or shipped: the KOReader emulator's `luajit` already links
|
||||
the real engine, and the oracle script runs it headlessly through the spec harness
|
||||
(`require("setupkoenv")` + `spec/front/unit/commonrequire`). Usage is in `docs/testing.md`.
|
||||
|
||||
**Oracle output** (`<book>.crengine.json`): per DocFragment, `{xp, xp_end, text}` for every
|
||||
visible word (`getNextVisibleWordStart/End` + `getTextFromXPointers`), `every` N sampling, arrays
|
||||
forced with luajson `InitArray` (empty Lua tables encode as `{}` otherwise). Committed fixture:
|
||||
`src/__tests__/fixtures/crengine/sample-alice.json` (every=40, 707 words, 114 KB). Full alice at
|
||||
every=1 (27,921 words) runs in ~95 s and passes 0/0 both ways after the fix.
|
||||
|
||||
**Harness rule**: the push check builds its CFI on the range the PULL check resolved (and only when
|
||||
that range's text equals crengine's word). A sequential text search looked independent but with
|
||||
sampling it found an earlier "the"/"of" and reported 313 phantom push mismatches. crengine writes
|
||||
`/body/DocFragment/…` (no index) for a single-spine book; the harness normalizes it to `[1]`, which
|
||||
crengine also resolves.
|
||||
|
||||
## crengine facts the oracle established (lvtinydom.cpp ldomXPointer::toString + a synthetic EPUB)
|
||||
|
||||
- `text()[K]` counts the parent's TEXT children only; `[K]` is omitted when there is exactly one.
|
||||
The offset is inside that node, NEVER cumulative over the element's descendants. `xcfi.ts` read
|
||||
unindexed `text().N` cumulatively, so `h2/text().1` after a `<span>Chapter 1</span>` landed on
|
||||
"hapt" instead of "Down" (58/27,921 alice words, all headings and `<li>`s).
|
||||
- Whitespace-only text before the FIRST element child of a block is dropped
|
||||
(`<p>\n <em>x</em>…` loses the leading node); whitespace between or after inline children
|
||||
survives as one space (`<em>a</em> <em>b</em>` keeps " " as `text()[1]`).
|
||||
- Runs of whitespace inside a text node collapse to one space and offsets count the collapsed
|
||||
text (`lambda mu` → "mu" at 7, raw DOM 9). Nothing is trimmed at block start.
|
||||
- With one spine item the path is `/body/DocFragment/body/…`; `[1]` still resolves.
|
||||
- Inside `pre`, `code`, `listing`, `plaintext`, `xmp`, `textarea` (fb2def.h built-in
|
||||
`white-space: pre`) whitespace is kept VERBATIM and offsets are raw; `code` counts even outside
|
||||
`<pre>`. Book CSS `white-space: pre` on other elements is NOT modelled (no computed styles
|
||||
off-screen) — a known residual.
|
||||
- The leading-blank drop (lvtinydom.cpp:8958) applies only to BLOCK parents and not under pre;
|
||||
block-ness is CSS-driven in crengine, so xcfi uses a tag list (`BLOCK_TAGS`).
|
||||
- A word at raw offset 0 of a text node that follows an element (`eta<br/>theta`,
|
||||
`<a id=.../>upsilon`) is `text()[2].0` in crengine.
|
||||
|
||||
**Fix shipped in `src/utils/xcfi.ts`** (same session, tests in
|
||||
`xcfi.crengine-semantics.test.ts`): `crengineTextChildren()` (drop the leading blank node),
|
||||
`toCollapsedOffset/toRawOffset` (whitespace runs), unindexed `text().N` → the sole text child,
|
||||
`DocFragment(?:\[N\])?` in `extractSpineIndex` and `resolveXPointerPath`, and push builds the
|
||||
pointer straight from the range's text node (`textNodeXPointer`) instead of a cumulative round
|
||||
trip that snapped boundary offsets to the previous node. Full suite 11,018 passed; readera,
|
||||
kosync and bookorbit consumers unchanged. Synthetic book (45 words incl. pre/code) 0/0 both ways.
|
||||
|
||||
**What it does NOT fix**: #5271 itself — see [[kosync-html-fallback-dom-mismatch-5271]]; on those
|
||||
books the oracle still reports ~99% pull mismatches because the DOM shape differs (text/html
|
||||
fallback), which is a foliate parser fix, not an xcfi one.
|
||||
|
||||
Related: [[koreader-emulator-headless-verify]], [[kosync-percentage-reanchor-impossible-path-5980]].
|
||||
|
|
@ -0,0 +1,76 @@
|
|||
---
|
||||
name: deeplink-coldstart-hijacks-spawned-windows-6104
|
||||
description: "#6104 every bookshelf click re-opened the URL-launched book: getCurrent() is process-global and each new reader window re-read it"
|
||||
metadata:
|
||||
node_type: memory
|
||||
type: project
|
||||
originSessionId: d348f44e-279a-4d7d-98cd-a86fcf364f51
|
||||
modified: 2026-09-07T08:13:39.108Z
|
||||
---
|
||||
|
||||
MERGED #6111 (2026-09-07); NOT device-verified on macOS/Android/iOS.
|
||||
|
||||
#6104 (macOS, reported 2026-09-07): after `open readest://book/<hash>`, every
|
||||
bookshelf click opened that same book instead of the clicked one, until the app
|
||||
was quit.
|
||||
|
||||
**Root cause.** `tauri-plugin-deep-link` stores the launch URL in
|
||||
PROCESS-global state and never clears it — macOS `RunEvent::Opened` does
|
||||
`current.replace(urls)` (plugin `src/lib.rs` `.on_event`), Windows/Linux do the
|
||||
same in `handle_cli_arguments`. So `getCurrent()` keeps returning
|
||||
`readest://book/<hash>` for the whole session. Both JS consume-once guards are
|
||||
PER-WEBVIEW: the module-scoped `coldStartConsumed` flag and the
|
||||
`consumedColdStartBookUrl` sessionStorage key. `openBookInNewWindow` defaults to
|
||||
**true** (`services/constants.ts`), so every library tap calls
|
||||
`showReaderWindow` → a brand-new `WebviewWindow` (`reader-N`) with a fresh JS
|
||||
context and empty sessionStorage → `useOpenBookLink` re-reads the stale URL →
|
||||
`window.location.pathname` starts with `/reader` → dispatches
|
||||
`open-book-in-reader` → `useBooksManager.openBookInReader` does
|
||||
`setBookKeys([newKey])`, which REPLACES the clicked book. Quitting "fixes" it
|
||||
only because the process dies.
|
||||
|
||||
**Mobile is a SECOND path, same root cause.** chrox confirmed Android + iOS
|
||||
reproduce, but only after a background/resume. Android's `activity.intent` is
|
||||
STICKY, so `DeepLinkPlugin.load()` re-reads it and re-emits it as a FRESH
|
||||
delivery on every Activity recreation -> lands on `app-incoming-url`, the LIVE
|
||||
path, which is deliberately never deduped. iOS: WebKit recycles the WebContent
|
||||
process on resume, the document reloads, `coldStartConsumed` + sessionStorage
|
||||
both die while `getCurrent()` still holds the URL. Checked and RULED OUT:
|
||||
`onOpenUrl` does NOT replay getCurrent (2.4.9/2.4.10 just `listen`), and the
|
||||
native bridge's pending-event queue drains once.
|
||||
|
||||
**Fix 2** (`src/utils/deeplinkConsume.ts` `markLaunchUrl(scope, url)` +
|
||||
`app_run_id` in the lib.rs init script): the marker was scoped to the DOCUMENT,
|
||||
the deep link lives as long as the PROCESS. Rust mints `__READEST_APP_RUN_ID__`
|
||||
once per run; the marker is localStorage keyed on it, ONE key per scope holding
|
||||
`{run, urls[]}` (a per-run SET capped at 16 - a single last-seen stamp lets a
|
||||
replayed A through once B was opened; CodeRabbit caught that). Cold-start reads
|
||||
(`getCurrent()`) skip a URL already in the set; LIVE deliveries are ALWAYS
|
||||
processed and only RECORDED, so a reload re-reporting them is a replay.
|
||||
Re-tapping the same bookmark still works (that IS the reporter's workflow).
|
||||
|
||||
**Dead end, do not repeat:** a 10s "launch replay window" on the live path.
|
||||
CodeRabbit flagged it, and it was guarding a path that does not exist: on
|
||||
Android `register_android_plugin` (-> Kotlin `load()`) runs BEFORE
|
||||
`setEventHandler`, so `channel` is null in `load()` and the re-emit is a no-op;
|
||||
the launch URL reaches JS ONLY via `getCurrent()`. Tauri does not buffer emits
|
||||
either (the native bridge's pending-event queue exists for exactly that).
|
||||
|
||||
**Fix 1** (`src/utils/window.ts` `isMainAppWindow()`, gated into the
|
||||
`if (!coldStartConsumed ...)` branch of `useOpenBookLink`,
|
||||
`useOpenAnnotationLink`, `useOpenShareLink`): only the window labelled `main`
|
||||
consumes a cold-start URL. Windows the app spawns itself carry their own intent
|
||||
in their URL (`/reader?ids=`, `/library?file=`) and never need it. Live taps
|
||||
(`app-incoming-url`) are untouched — those are genuine user actions.
|
||||
|
||||
**Gotchas.**
|
||||
- The Rust `WebviewWindowBuilder::new(app, "main", ...)` in `src-tauri/src/lib.rs`
|
||||
is NOT `#[cfg(desktop)]`-gated, so mobile webviews are labelled `main` too and
|
||||
keep working.
|
||||
- `useOpenWithBooks` already had the same idea as `isFirstWindow()` for the LIVE
|
||||
path — precedent, not a new pattern.
|
||||
- Unit-testing this needs `vi.resetModules()` (the guard is module state) plus a
|
||||
static `import '@/hooks/useOpenBookLink'` to warm the graph, or the first case
|
||||
times out under parallel load.
|
||||
|
||||
Related: [[bug-patterns]]
|
||||
|
|
@ -0,0 +1,64 @@
|
|||
---
|
||||
name: eink-pdf-textlayer-selection-accentcolor
|
||||
description: "PDF text selection ignores the e-ink ::selection inversion because pdf.js's `.textLayer ::selection` outranks Readest's bare `::selection`"
|
||||
metadata:
|
||||
node_type: memory
|
||||
type: project
|
||||
originSessionId: a0277d08-901d-4dda-be8a-19be816afdfa
|
||||
modified: 2026-09-06T06:46:45.304Z
|
||||
---
|
||||
|
||||
Support case EINK-PDF-01: on e-ink, selecting a word produces ghosting and a grey
|
||||
dither that makes the selected text illegible. Selection needs high contrast with
|
||||
100% solid backgrounds.
|
||||
|
||||
**Verified on the Boox Leaf5 with a sample PDF.**
|
||||
|
||||
**Root cause (CONFIRMED).** `getEinkSelectionStyles()` in `src/utils/style.ts` injects a
|
||||
bare `::selection { color: var(--theme-bg-color); background: var(--theme-fg-color) }`
|
||||
into the book document, and it is present in the PDF iframe too. But pdf.js's own
|
||||
stylesheet, loaded first in that same iframe, carries
|
||||
|
||||
.textLayer ::selection { background: color-mix(in srgb, AccentColor, transparent 75%); }
|
||||
|
||||
`.textLayer ::selection` outranks the bare selector on specificity, so **every PDF
|
||||
selection paints as 25%-opacity system AccentColor** — a translucent lavender/teal wash
|
||||
over still-black text. That is the reporter's "grey dithering", and it is exactly what a
|
||||
B&W panel cannot render. The same selection in an EPUB inverts correctly to solid
|
||||
black-on-white, which is why this reads as PDF-only.
|
||||
|
||||
**Fix (verified live).** Injecting
|
||||
`.textLayer ::selection{color:var(--theme-bg-color)!important;background:var(--theme-fg-color)!important}`
|
||||
into the PDF iframes turned the selection into a solid black block with white text,
|
||||
matching the EPUB. Not committed — verification only.
|
||||
|
||||
**Second finding: the toolbar chrome is already correct.** In e-ink mode the selection
|
||||
popup is solid `bg-base-100` with a 1px `base-content` border and no shadow
|
||||
(`not-eink:shadow-2xl` etc. all behave). The reporter's photos show a *grey translucent
|
||||
panel with a drop shadow*, which reproduces only with **E-Ink Mode OFF** — so his install
|
||||
is probably not in e-ink mode at all. Auto-detection exists
|
||||
(`src-tauri/src/android/eink.rs`, ONYX/`leaf` both match, sets `window.__READEST_IS_EINK`)
|
||||
but `loadSettings` merges as `{...getDefaultViewSettings(ctx), ...settings.globalViewSettings}`
|
||||
— **the stored value wins**, so any settings file written before the detection shipped
|
||||
(2026-01-08, #2887) keeps `isEink: false` forever with no prompt.
|
||||
|
||||
**Third finding: the teal selection handles are the system's.** Readest renders its own
|
||||
`Handle` (e-ink-coloured) only when `selection.handlesSuppressed` is set, which happens
|
||||
only for lookup surfaces. A plain toolbar selection keeps the Android WebView's native
|
||||
handles, drawn in the platform accent colour, and they stay bright teal on e-ink.
|
||||
|
||||
**Also seen, not chased:** PDF-embedded link colours stay blue on e-ink (the
|
||||
`[class*='text-blue']` rules only touch Tailwind classes, not PDF content), and with Color
|
||||
E-Ink Mode ON body text renders grey rather than black.
|
||||
|
||||
**Device gotchas.** The Leaf5 had **Color E-Ink Mode ON** (wrong for a mono Carta panel) —
|
||||
that alone keeps full-colour swatches, grey text and a translucent selection, so check it
|
||||
before blaming code; restored as found. Instant Dictionary was on, which swallows the
|
||||
long-press before the toolbar can appear — turn the quick action off to see the toolbar.
|
||||
`adb shell input swipe x y x y 900` does NOT long-press; `input motionevent DOWN/UP` with a
|
||||
`sleep` between them does, and CDP `Input.dispatchTouchEvent` does not drive Android's
|
||||
native text-selection gesture at all. `curl` to the forwarded devtools port needs
|
||||
`--noproxy '*'` or the SOCKS proxy eats it.
|
||||
|
||||
Related: [[feedback-always-verify-on-xiaomi]], [[eink-highlight-difference-mask-5667]],
|
||||
[[eink-per-device-css-data-eink-5795]], [[eink-class-substring-matchers]]
|
||||
|
|
@ -9,3 +9,9 @@ Run a specific test file with `pnpm test <path>` or `pnpm test:browser <path>`
|
|||
**Why:** Adding `--` before the path (e.g. `pnpm test:browser -- <path>`) causes vitest to ignore the file filter and run all test files. Without `--`, pnpm appends the path directly to the vitest command, which correctly filters to that file only.
|
||||
|
||||
**How to apply:** Always use `pnpm test src/__tests__/foo.test.ts` or `pnpm test:browser src/__tests__/foo.browser.test.tsx` when verifying a specific test file.
|
||||
|
||||
**Agent shells (2026-09-07):** the `test` script is bare `vitest`, so from a background
|
||||
Bash it can sit in watch mode and a `| tail` never returns. Use
|
||||
`pnpm exec dotenv -e .env -e .env.test.local -- vitest run <path>` for a guaranteed
|
||||
one-shot run (the dotenv wrapper is required: `src/utils/supabase.ts` atob-decodes an
|
||||
env var at import time and throws without it).
|
||||
|
|
|
|||
|
|
@ -0,0 +1,83 @@
|
|||
---
|
||||
name: kosync-html-fallback-dom-mismatch-5271
|
||||
description: "#5271 (KOReader highlights land on page 1 / Readest shows KO highlights in the wrong place) was NOT fixed by #5630 and is now FIXED in foliate epub.js (closeVoidElements + parseContentDocument, 2026-09-07, PR pending): the text/html fallback for malformed XHTML builds a DOM where self-closed <a id=page_N/> anchors swallow the following <p>s, so it no longer matches crengine's; fix = close void tags and re-parse as XML before falling back to HTML (verified 19/20 byte-identical XPointers against the emulator's crengine, 2026-09-07)"
|
||||
metadata:
|
||||
type: project
|
||||
originSessionId: 85280a2e-03e0-46da-8144-1e680cc6af56
|
||||
modified: 2026-09-07T13:58:56.955Z
|
||||
---
|
||||
|
||||
Issue #5271 (ivanbeldad, Readest 0.11.20 macOS + KOReader with readest.koplugin). Reporter's two
|
||||
EPUBs are at `~/Documents/books/issues/5271/` ("It's OK That You're Not OK", "Understanding and
|
||||
Responding to Self-Harm"; the Gmail draft holds the same two files). chrox CLOSED it 2026-09-04
|
||||
with no linked PR, presumably on the strength of #5630 (same unclosed `<meta charset>` as the
|
||||
#5625 book) and #6014. **Verified 2026-09-07 on `dev` at 0041b7b3e (both fixes in): NOT fixed.**
|
||||
Every one of 20 probed highlights fails in BOTH directions. Recommend REOPENING.
|
||||
|
||||
## Mechanism (not the #5625 null-body crash — that part is fixed)
|
||||
|
||||
Every spine file in both books is malformed XML the InDesign way (`<meta charset="utf-8">`
|
||||
unclosed; `xmllint` says `Opening and ending tag mismatch: meta line 3 and head`). foliate's
|
||||
`loadReplaced` (render) and, since #5630, `loadDocument` (`createDocument`) both retry such a file
|
||||
as `text/html`. The HTML parser does NOT honour `/>` on non-void elements, and `<a>` is a
|
||||
*formatting element*: the books' page anchors `<a id="page_25"/>` inside `<p>` stay open, the
|
||||
newline after `</p>` reconstructs the `<a>` at BODY level, and every following `<p>` becomes a
|
||||
child of that `<a>` until the next anchor (a mid-paragraph anchor runs the adoption agency
|
||||
algorithm and hoists THAT `<p>` back to body level). Confirmed in real Chrome 152 via DOMParser and
|
||||
in jsdom. crengine (`LVHTMLParser`, honours `/>`) keeps `<a/>` empty and all `<p>` under `<body>`.
|
||||
|
||||
Consequences, measured with the emulator's crengine (246 / 196 pages):
|
||||
- Readest -> KO: Readest emits `/body/DocFragment[12]/body/a[1]/p[3]/text().0`; crengine has no
|
||||
`body/a[1]` -> `createXPointer` null -> `getPageFromXPointer` returns its default **1**
|
||||
(cre.cpp `int page = 1`). That is the reporter's "always page 1". 20/20 unresolvable.
|
||||
- KO -> Readest: crengine emits `/body/DocFragment[12]/body/p[4]/text().0`; in Readest's DOM the
|
||||
only DIRECT `p` children of body are the chapter-number line plus paragraphs carrying a
|
||||
mid-paragraph anchor, so `p[4]` lands on an unrelated paragraph ("random places", 16/20) or
|
||||
`text()[1]` finds 0 direct text nodes and throws -> note dropped ("don't appear", 4/20).
|
||||
- DocFragment indices themselves are right (`DocFragment[N]` == section N-1), so #6014 is not
|
||||
involved.
|
||||
|
||||
## Fix — IMPLEMENTED 2026-09-07 in packages/foliate-js/epub.js (chrox: "Can we solve #5271?")
|
||||
|
||||
`parseContentDocument(parser, str, mediaType)` is shared by `loadReplaced` (render) and
|
||||
`loadDocument` (createDocument): XML parse -> on parsererror `closeVoidElements` (lowercase void
|
||||
tags only, quoted-attribute-aware regex, skips already self-closed) -> XML re-parse -> HTML only if
|
||||
still broken. Oracle result on the reporter's books after the fix: It's OK 2946/2946 and Self-Harm
|
||||
1845/1845 sampled words exact in BOTH directions (was ~99% failing). Tests in
|
||||
`src/__tests__/foliate-epub-malformed-xhtml.test.ts` (#5271 block: body children, empty anchors,
|
||||
XPointer both ways, render path via captured blob, beyond-repair `&` still falls back to HTML).
|
||||
Not browser-verified. Stored CFIs for such books DO go stale (accepted).
|
||||
|
||||
### Original fix direction note
|
||||
|
||||
In `packages/foliate-js/epub.js` `loadDocument` AND `loadReplaced`: on `parsererror` for XHTML,
|
||||
first close unclosed void elements (`meta link br img hr input col area base embed param source
|
||||
track wbr`) and re-parse as `application/xhtml+xml`; fall back to `text/html` only if that still
|
||||
fails. Experiment on the real books: repaired-XML DOM gives XPointers byte-identical to crengine's
|
||||
for 19/20 passages; the 20th differs only at a `<span class="hide">` boundary
|
||||
(`p[2]/span/text().12` vs crengine `p[2]/text().0`, same visual point). Caveats: existing CFIs
|
||||
stored for such books encode the `<a>`-wrapper structure and would go stale; the malformation is
|
||||
ubiquitous in InDesign/ADE exports, so the blast radius is wide (mostly for the better: today
|
||||
those books render every paragraph inside an anchor element).
|
||||
|
||||
## Headless crengine probe recipe (no GUI, no sync server)
|
||||
|
||||
`cd ~/dev/koreader/koreader-emulator-arm64-apple-darwin24.6.0-debug/koreader &&
|
||||
KO_HOME=<scratch> ./luajit probe.lua in.json out.json` with a script that does
|
||||
`require("setupkoenv")`, `package.path = "spec/front/unit/?.lua;" .. package.path`,
|
||||
`require("commonrequire")` (dummy framebuffer), then `DocumentRegistry:openDocument(epub)`,
|
||||
`doc:requestDomVersion(doc:getLatestDomVersion())`, `doc:render()`, and per case
|
||||
`doc:isXPointerInDocument(xp)`, `doc:getPageFromXPointer(xp)`, `doc:getTextFromXPointers(a, b)`,
|
||||
and `doc:findAllText(text, false, 0, 5, false)` -> `[{start, end}]` for crengine's OWN XPointers of
|
||||
a passage. `require("json")` works. KO_HOME keeps `settings.tests.lua`/cache out of the emulator
|
||||
dir. Readest side: vitest + `DocumentLoader(File)` on the real EPUB, `section.createDocument()`,
|
||||
`CFI.joinIndir(section.cfi, CFI.fromRange(range))`, `getXPointerFromCFI(cfi, undefined,
|
||||
undefined, book)`; reverse via `new XCFI(doc, index).xPointerToCFI(start, end)` +
|
||||
`book.resolveCFI(cfi).anchor(doc).toString()`. Note `CFI.toRange` wants the spine step SHIFTED off
|
||||
first. Runs in ~1 s each; keep probe files out of the repo.
|
||||
|
||||
Related: [[loaddocument-xhtml-parsererror-5625]] (the crash half, fixed),
|
||||
[[kosync-percentage-reanchor-impossible-path-5980]] (DocFragment mapping, not involved),
|
||||
[[koreader-highlight-deletion-dedupe-5818]], [[koreader-emulator-headless-verify]].
|
||||
|
||||
**Follow-up 2026-09-07:** a crengine oracle now exists ([[crengine-xpointer-oracle]]); on these two books it reports ~99% KOReader->Readest mismatches, all from the DOM shape, so the void-tag repair above remains the fix.
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
---
|
||||
name: landing-docs-support-reply-tts-ios
|
||||
description: "Support case TTS-IOS-01 exposed two Listen-docs errors; how TTS actually starts and picks voices, and the readest-landing docs edit recipe"
|
||||
metadata:
|
||||
node_type: memory
|
||||
type: project
|
||||
originSessionId: fa50820b-e361-4dee-85c0-93aeee148071
|
||||
modified: 2026-09-06T05:00:07.244Z
|
||||
---
|
||||
|
||||
Support case TTS-IOS-01 (iPhone): docs said "press `T` or tap the speaker icon in the top toolbar" and "iOS uses the system voices". Both wrong for the app as shipped.
|
||||
|
||||
**Ground truth (verified in code 2026-09-06):**
|
||||
- TTS starts from the headphones icon in the BOTTOM toolbar on every platform (`footerbar/NavigationBar.tsx` mobile, `DesktopFooterBar.tsx`). `t` is only a keyboard shortcut (`src/helpers/shortcuts.ts`).
|
||||
- Default engine = `availableClients[0]` in `TTSController.init` = Edge TTS whenever it inits (online), on iOS/Android too. Native is never picked on its own.
|
||||
- Voice/engine choice lives ONLY in the expanded Read Aloud player (mini player cover/title -> `TTSPlayerSheet` -> Voice), grouped "Edge TTS" / "System TTS" (iOS native groupId = `default`). Settings -> TTS has highlighting, speech, media info, audio cache only.
|
||||
- Native picks the FIRST system voice for the language (`NativeTTSClient.getVoiceIdFromLang`), not the iOS Spoken Content default voice. Possible follow-up: honor `AVSpeechSynthesisVoice(language:)` when no preferred voice is stored.
|
||||
|
||||
**Landing docs recipe** (`~/dev/readest-landing`, branch `dev`): edit `src/app/[locale]/docs/<slug>/page.mdx`, bump `lastModified` for that slug in `docs/_nav.ts`, run `pnpm generate:llms` (regenerates the TRACKED `src/app/llms-full.txt/_content.ts`), then `pnpm test && pnpm lint`. `pnpm format:check` has 7 pre-existing failures in legal/blog/sync MDX; ignore unless touched. Listen page fix = readest-landing PR #35 (branch `docs/listen-tts-start-and-voice-picker`, e02df9d) MERGED 2026-09-06.
|
||||
|
||||
**Why:** the same two questions will recur as long as Edge is the silent default on iOS.
|
||||
**How to apply:** for TTS support mail, point users to the player's Voice button, never to Settings; keep the docs and this note in step if the default-engine logic changes.
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
---
|
||||
name: page-nav-a11y-bottom-dead-zone-6079
|
||||
description: "#6079 bottom dead zone in scroll mode = the four hidden a11y page-nav buttons; 0.12.6 lacked #6001, and h-4 lost to h-20 by Tailwind sort order"
|
||||
metadata:
|
||||
node_type: memory
|
||||
type: project
|
||||
originSessionId: 091f163d-ec2e-40fe-875f-db5e8b295342
|
||||
modified: 2026-09-06T16:12:10.773Z
|
||||
---
|
||||
|
||||
`PageNavigationButtons.tsx` renders four page/section buttons that stay mounted even when
|
||||
hidden, because Android screen readers only announce them while they still hit-test.
|
||||
That hack has produced a bottom-of-screen dead zone four times: #4501, #5966/#5974,
|
||||
#6001, and now #6079 ("Dead zone with scroll mode", Android, 0.12.6, ~1/8 screen height).
|
||||
|
||||
**#6079's actual cause is a release gap, not new code.** Neither #5974 nor #6001 is an
|
||||
ancestor of `v0.12.6` (v0.12.6 = f6146c217, 2026-08-28; #6001 = cd4a76fef, 2026-09-01).
|
||||
In 0.12.6 the button className was `'flex h-20 w-20 ...'` plus a conditional `'h-4 w-4'`.
|
||||
**Same specificity — Tailwind emits `.h-4` before `.h-20`, so `h-20` won and the override
|
||||
was dead.** Result: two 80px-tall, ~164px-wide hit boxes in each bottom corner. #6001
|
||||
fixed it by picking one size string (`navigationButtonSize` = `'h-2 w-2 overflow-hidden'`
|
||||
or `'h-20 w-20'`), never two conflicting ones. **Never "override" a Tailwind sizing
|
||||
utility with another of the same property — swap the whole token.**
|
||||
|
||||
**The fix, MERGED #6102 (f5399cdcc, 2026-09-06):** the Android exception is GONE.
|
||||
`!isPageNavigationButtonsVisible && !appService?.isAndroidApp ? 'pointer-events-none'`
|
||||
became just `!isPageNavigationButtonsVisible ? 'pointer-events-none'` on both
|
||||
containers, so the hidden controls stop hit-testing on EVERY platform and there is no
|
||||
bottom dead zone in any configuration. They take taps only once actually visible
|
||||
(needs `showPaginationButtons` on AND the reader chrome shown). chrox rejected a
|
||||
first attempt that kept the taps whenever the setting was off (gated on a
|
||||
`captureHiddenTaps` flag) — simpler is right here, don't preserve the Android
|
||||
carve-out.
|
||||
|
||||
`opacity-0` + `pointer-events:none` still leaves the buttons in Chrome's a11y tree, so
|
||||
TalkBack linear (swipe) navigation reaches them either way; only explore-by-touch needs
|
||||
the hit test.
|
||||
|
||||
Test: `src/__tests__/components/page-navigation-buttons.browser.test.tsx` (browser lane,
|
||||
real Chromium + real `globals.css`). It asserts hit targets with
|
||||
`document.elementFromPoint` at the button centre, so it catches CSS-order regressions
|
||||
that a jsdom test cannot. `mockPlatform.showPaginationButtons` toggles the two modes.
|
||||
|
||||
Related: [[layout-ui-fixes]], [[eink-class-substring-matchers]],
|
||||
[[daisyui-v5-tailwind-v4-migration]]
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
---
|
||||
name: pr-6116-cover-repair-review
|
||||
description: "#5931 S3 Full Sync never restores covers for rows Readest Cloud restored; PR #6116 (jadhavgaurav) MERGED 2026-09-07 as 0041b7b3e (my 636819a47 on top of the contributor); repair is a pull-side fullSync pass so Receive Only heals too; not device-verified"
|
||||
metadata:
|
||||
type: project
|
||||
---
|
||||
|
||||
Issue #5931: Readest Cloud restores metadata-only rows (no file, no cover); the S3/WebDAV
|
||||
Full Sync then never pulls the covers because discovery skips any hash already in
|
||||
`allBooksMap`, metadata reconciliation only pulls a cover when the remote clock is
|
||||
STRICTLY newer, and `pushBookCover` resolves `no-source` and nothing pulls.
|
||||
|
||||
PR #6116 (@jadhavgaurav, head 7e4d1e126, base main) reviewed 2026-09-07, verdict =
|
||||
needs one change before merge. Fix = `pushBookCover` reports `remoteExists` from
|
||||
the HEAD it already pays; the push loop pulls the cover on `no-source && remoteExists
|
||||
&& canPull`, saves it, stamps `coverDownloadedAt`, persists via
|
||||
`store.updateBookMetadata` (which regenerates `coverImageUrl`, does NOT bump
|
||||
`updatedAt`, so no Readest Cloud upsert storm). Tests (4) pass; lint clean; full
|
||||
suite 10984 passed, 3 timeout flakes pass in isolation.
|
||||
|
||||
**Defect (CodeRabbit r-comment, VERIFIED by probe):** the repair lives inside
|
||||
`if (canPush && booksToPush.length > 0)`, so `strategy: 'receive'` ("Receive only",
|
||||
a real FileSyncForm option) + Full Sync repairs nothing (`coversDownloaded` = 0).
|
||||
Probe test kept at the session scratchpad `engine-cover-repair-receive.probe.test.ts`.
|
||||
|
||||
**Fix pushed 2026-09-07 as 636819a47 on top of the contributor's 7e4d1e126** (fast-forward to
|
||||
`jadhavgaurav:fix/file-sync-repair-missing-covers-5931`, maintainerCanModify true; PR body
|
||||
rewritten + comment posted; MERGED 2026-09-07 as 0041b7b3e). Design = a pull-side pass right after
|
||||
deletion propagation, `if (fullSync && canPull && remoteIndex?.books)`: for each live indexed
|
||||
row, `loadBookCover(local)`; if missing, `pullBookCover` (GET, 404 = null), `saveBookCover`,
|
||||
stamp `coverDownloadedAt`, `updateBookMetadata`, `allBooksMap.set`, `coversDownloaded++`.
|
||||
The `remoteExists` plumbing was dropped. Test file now 6 cases incl. Receive Only and an
|
||||
incremental guard (mutation-checked: removing the `fullSync` gate flips exactly that test).
|
||||
TRAP: `engine-group-metadata-5911.test.ts` "a Full Sync repair pulls no cover it does not
|
||||
need" used a fake store with NO local cover, so the repair legitimately GETs; fixed by giving
|
||||
that fake a local cover (its intent is "field repair never pulls a cover already here").
|
||||
Full suite 10976 passed. Not device-verified.
|
||||
|
||||
Worktree removed after merge. It had to be reset to the REAL PR head before committing
|
||||
(worktree:new had rebased it onto main) ([[worktree-new-rebases-pr-force-push]]).
|
||||
Fixing on top of a contributor PR recipe: [[file-sync-converge-5900]].
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
---
|
||||
name: proofread-panel-design-pass-6109
|
||||
description: "PR #6109 (54wedge, TTS rules in the proofread panel) MERGED 2026-09-07 as squash 2e021e4c3; toggleRule dropped `scope` so library-scoped rules were never written - the hidden toggle was a symptom, not the bug"
|
||||
metadata:
|
||||
node_type: memory
|
||||
type: project
|
||||
originSessionId: 36e8617f-0012-4044-b2b6-5059e5ad403c
|
||||
modified: 2026-09-07T10:05:21.712Z
|
||||
---
|
||||
|
||||
2026-09-07. PR #6109 from **54wedge** adds "Only for TTS" to the Proofread
|
||||
Replacement Rules panel. Reviewed, reworked, pushed onto their branch as
|
||||
`104060e95`, and **MERGED** as squash commit `2e021e4c3` on origin/main.
|
||||
Worktree and local branches cleaned up.
|
||||
|
||||
Process note worth repeating: `pnpm worktree:new` rebased the branch, so the
|
||||
fix was committed there first and then **cherry-picked onto the real PR head**
|
||||
before pushing — a fast-forward (`2e5a4d754..104060e95`), contributor commits
|
||||
intact, no history rewrite. See [[worktree-new-rebases-pr-force-push]]. Both
|
||||
pre-flight checks came back empty (`git diff <pr-head> HEAD~1 -- <touched
|
||||
files>` and the `git merge-tree` equivalence proof), so the rebase was not
|
||||
load-bearing. `maintainerCanModify=true` is what allows the push to the fork.
|
||||
Because GitHub squash-merged, the branch tips are NOT ancestors of
|
||||
origin/main — verify the content landed (grep the file on origin/main) rather
|
||||
than trusting `merge-base --is-ancestor` before deleting branches.
|
||||
|
||||
**The review finding worth keeping:** the PR hid the enable/disable switch on
|
||||
library-scoped rules because "the toggle doesn't work at all". The switch was
|
||||
fine; `proofreadStore.toggleRule` was not. It called
|
||||
`updateRule(env, bookKey, ruleId, { enabled: !rule.enabled })` **without
|
||||
`scope`**, and `updateRule` routes on `updates.scope === 'library'` — so a
|
||||
library rule went down the book branch, which maps over the *book's*
|
||||
`proofreadRules`, never finds it, and writes nothing. `removeRule` and
|
||||
`saveEdit` both pass scope explicitly, which is why only the toggle looked
|
||||
dead. Fixed by passing `scope: rule.scope` and using `enabled: rule.enabled ===
|
||||
false` (the list renders `enabled !== false` as on, so `!undefined` would have
|
||||
re-enabled a legacy flag-less rule). Regression tests live in
|
||||
`src/__tests__/store/proofread-store.test.ts`.
|
||||
|
||||
Still open on that surface: 54wedge reports selection-scoped rules stop
|
||||
applying after any toggle or edit (untouched here), and `ProofreadPopup`'s
|
||||
`range.deleteContents()` path can drop markup when the selection spans
|
||||
elements.
|
||||
|
||||
Related: [[proofread-gate-reflowable-formats]],
|
||||
[[adhoc-visual-check-daisyui-theme-tokens]].
|
||||
|
|
@ -0,0 +1,90 @@
|
|||
---
|
||||
name: selfhosted-docker-6091-6093
|
||||
description: "Two self-hosted Docker bugs fixed together — Next 16 truncates any request body over proxyClientMaxBodySize (10MB default) because middleware.ts matches /api/*, and the Cloud Sync Premium chip keyed off sign-in instead of entitlement"
|
||||
metadata:
|
||||
node_type: memory
|
||||
type: project
|
||||
originSessionId: 1a5dc2d9-f502-4d77-b951-d9c557c40b00
|
||||
modified: 2026-09-06T15:03:45.199Z
|
||||
---
|
||||
|
||||
**MERGED #6100** (squash `731b13ab6`, 2026-09-06). Both fixed in one pass; UNRELEASED, so a
|
||||
self-hoster on `latest` only picks it up after the next image build from main.
|
||||
|
||||
## #6091 — bodies over 10MB silently truncated
|
||||
|
||||
`middleware.ts` matches `/api/*`, so Next buffers a **clone** of every non-GET
|
||||
request body. `next/dist/server/body-streams.js` `getCloneableBody` caps that
|
||||
clone at `experimental.proxyClientMaxBodySize` (**default 10485760**) and, past
|
||||
the limit, pushes `null` into **both** streams — `p1` (handed to the middleware)
|
||||
and `p2`, which `finalize()` splices back over the `IncomingMessage`. So the
|
||||
**route handler** sees a short body ending cleanly: no error, no 413, a stored
|
||||
file of exactly 10,485,760 bytes. Only a `console.warn` on the server.
|
||||
|
||||
This bites upstream today at `pages/api/send/inbox/file.ts`, which accepts
|
||||
`SEND_INBOX_FILE_MAX_BYTES` = 40 MB. Book uploads are NOT affected upstream:
|
||||
`getUploadSignedUrl` only knows `r2`/`s3` and presigns straight to the bucket,
|
||||
so the body never crosses the Next server. The reporter's "local storage
|
||||
backend" is not in this repo — a fork, or a custom proxying setup.
|
||||
|
||||
Fix: `proxyClientMaxBodySize: standaloneOutput ? 1GB : 32MB` in
|
||||
`next.config.mjs` (chrox's numbers). The clone is held in RAM for the life of the
|
||||
request and the buffering happens BEFORE any handler can reject, so the ceiling
|
||||
doubles as an unauthenticated memory-exhaustion budget on every `/api/*` route —
|
||||
only the self-hosted image gets real headroom. Number (bytes), not the string
|
||||
form, so `src/__tests__/next-config.test.ts` can compare the two builds.
|
||||
|
||||
**32MB is deliberately BELOW `SEND_INBOX_FILE_MAX_BYTES` (40MB)**, so a 32-40MB
|
||||
inbox EPUB still truncates on a plain `next start`. Accepted: Cloudflare
|
||||
(OpenNext/workerd) and Vercel never run `next-server.js`'s `attachRequestMeta`,
|
||||
so no production deployment reaches this code path except the Docker image. The
|
||||
"covers the largest route body" assertion therefore lives on the SELF-HOSTED
|
||||
config; the default build only has to beat Next's 10MB.
|
||||
|
||||
**`standaloneOutput` (i.e. `BUILD_STANDALONE`) is the self-host discriminator at
|
||||
BUILD time**: the Dockerfile is the only thing that sets it, and web.readest.com
|
||||
runs on Cloudflare/Vercel, never on that image. `SELF_HOSTED` cannot serve here —
|
||||
CI builds the image without it (it is a compose-time runtime var), and a
|
||||
standalone build bakes next.config into `required-server-files.json`, so a
|
||||
runtime env knob would be dead anyway. Test the standalone branch by stubbing
|
||||
BOTH `BUILD_STANDALONE=true` and `NEXT_PUBLIC_APP_PLATFORM=web` — without the
|
||||
latter, `exportOutput` is true under vitest and forces `standaloneOutput` false.
|
||||
|
||||
Next dev prints `· proxyClientMaxBodySize: 268435456` under "Experiments" —
|
||||
cheapest confirmation the value took.
|
||||
|
||||
## #6093 — WebDAV shows "Premium" on a self-hosted deployment
|
||||
|
||||
`isCloudSyncAllowed` → `isCustomizationAllowed` → `isSelfHosted()` already
|
||||
returned true, so the row **opened fine and sync was never paused**. The only
|
||||
defect was the chip:
|
||||
|
||||
```ts
|
||||
!user || (userProfilePlan !== undefined && !isCloudSyncPremium) // old
|
||||
```
|
||||
|
||||
`!user` short-circuits, so a signed-out self-hoster (the normal state right
|
||||
after `docker compose up`) got a Premium badge on every third-party provider —
|
||||
which reads as "not available in this deployment". Fix is
|
||||
`shouldShowCloudProviderBadge` in `integrations/cloudSyncStatus.ts`:
|
||||
`!isPremium && (!signedIn || !planLoading)`. **Entitlement decides, sign-in does
|
||||
not** — self-hosting unlocks with or without a user.
|
||||
|
||||
Verified live both ways: `SELF_HOSTED=true pnpm dev-web`, signed out →
|
||||
`/runtime-config.js` serves `{"selfHosted":true}`, no badges, WebDAV form opens;
|
||||
then `window.__READEST_RUNTIME_CONFIG = {}` + a settings-tab round trip →
|
||||
badges return. The second half matters, it proves the hosted paywall still works.
|
||||
|
||||
Second cause for the same symptom: `SELF_HOSTED` only reached the container
|
||||
through `compose.yaml` (`${SELF_HOSTED:-true}`, added in #5996 on 2026-09-01).
|
||||
**A compose file copied before that date does not pass it through and `docker
|
||||
compose pull` never updates it**, so the deployment stayed gated with nothing on
|
||||
screen to explain why. Fixed in the image itself: **`ENV SELF_HOSTED=true` in the
|
||||
Dockerfile production stage**, on the same premise as the build-time gate above —
|
||||
the published image IS the self-hosted artifact. A compose file that does pass
|
||||
the variable still overrides it, so `SELF_HOSTED=false` remains the escape hatch
|
||||
for anyone selling plans off this image. Guarded by
|
||||
`src/__tests__/docker-image-defaults.test.ts`, which greps the production stage —
|
||||
dropping that ENV silently re-gates every self-hoster.
|
||||
|
||||
See [[storage-customization-entitlement-split]].
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
---
|
||||
name: sheet-swipe-dismiss-anywhere-6089
|
||||
description: "#6089 dismiss bottom sheets by swiping down from anywhere - MERGED #6097; Dialog is the shared sheet chassis, useSwipeToDismiss is a parallel one, and useDrag leaked its shield on touchcancel"
|
||||
metadata:
|
||||
node_type: memory
|
||||
type: project
|
||||
---
|
||||
|
||||
**#6089 → PR #6097, MERGED 2026-09-06 as squash `1b4180c39`. NEVER verified on a
|
||||
device** — no build was ever installed; the Xiaomi CDP session was set up and then
|
||||
abandoned. Everything below is unit-test evidence only.
|
||||
|
||||
**Two commits:** `feat(ui)` swipe-from-anywhere, then `fix(ui)` touchcancel teardown
|
||||
(CodeRabbit caught the second, correctly).
|
||||
|
||||
## The sheet chassis is split in two — check both
|
||||
|
||||
`src/components/Dialog.tsx` is the bottom sheet for the dictionary, note editor, TTS
|
||||
player, Settings, BackupWindow, ReadingRuler — every `<Dialog>`. It owns `snapHeight`.
|
||||
|
||||
`src/hooks/useSwipeToDismiss.ts` is a SEPARATE, older implementation of the same
|
||||
gesture (PR #3548/#3849) used ONLY by SideBar and Notebook. **Dialog does not use it
|
||||
and cannot** — it has no `snapHeight` support. A first sweep for "the bottom sheet
|
||||
code" finds one and misses the other; grep `useDrag(` to find all of them
|
||||
(`Dialog`, `useSwipeToDismiss`, `usePanelResize`).
|
||||
|
||||
## The swipe gate (Dialog)
|
||||
|
||||
Armed on `touchstart` on `.modal-box`, takes over only after 8px of mostly-vertical
|
||||
downward movement — below that the touch still belongs to the content, so taps and
|
||||
scrolls are untouched. Backs off for: a target with `scrollTop > 0` anywhere up to
|
||||
the modal box (that scroller owns the gesture), `input`/`textarea`/`select`/
|
||||
`contenteditable` (catches the TTS seek bar for free), sideways or upward movement,
|
||||
`!dismissible`, multi-touch, desktop viewport, `.drag-handle` (it starts its own drag).
|
||||
|
||||
Drag positioning is now **grab-offset relative** (`clientY - modal.getBoundingClientRect().top`
|
||||
captured at drag start) instead of pinning the sheet's top edge to the finger. The old
|
||||
absolute mapping only looked right because the handle sits at that edge.
|
||||
|
||||
## useDrag leaked its shield on touchcancel (pre-existing, fixed here)
|
||||
|
||||
`useDrag` covers the viewport with a `.drag-shield` (`z-index: 2147483647`,
|
||||
`pointer-events: auto`, added for #5043 so a release over a PDF iframe still ends the
|
||||
drag) and removed it ONLY on `touchend`/`mouseup`. There was no `touchcancel`
|
||||
listener, so a system-stolen touch (Android edge-swipe back, notification shade,
|
||||
incoming call) left the shield installed, `userSelect` pinned off and the window
|
||||
listeners attached. **The next tap anywhere landed on the shield, was swallowed, and
|
||||
was then read as a drag release** with whatever velocity the stale coordinates implied
|
||||
— which for a sheet can dismiss it.
|
||||
|
||||
Fix: `touchcancel` runs the same teardown and reports `canceled: true` on the
|
||||
`onDragEnd` payload (new required field — a cancelled drag carries no decision).
|
||||
`Dialog` restores to its resting snap and skips the haptic; `useSwipeToDismiss` slides
|
||||
the panel back; `usePanelResize` reads only absolute `clientX` and passes no
|
||||
`onDragEnd`, so it needed nothing.
|
||||
|
||||
## Test notes
|
||||
|
||||
jsdom has no `Touch`/`TouchEvent` constructor — build a stand-in with
|
||||
`new Event(type, {bubbles:true})` + `Object.assign(e, {touches:[t], changedTouches:[t]})`;
|
||||
React's synthetic handlers pick it up fine. `isMobile` is read from
|
||||
`window.innerWidth/innerHeight` at render, so redefine both BEFORE `render()`.
|
||||
jsdom reports every scroll metric as 0, so pin `scrollTop` with `Object.defineProperty`.
|
||||
|
||||
Related: [[eink-class-substring-matchers]], [[feedback-always-verify-on-xiaomi]].
|
||||
|
|
@ -7,7 +7,7 @@ metadata:
|
|||
|
||||
**Done 2026-09-02** (OP had two accounts and wanted the space on the one carrying the subscription): a 2 GB Apple add-on moved from an Apple-sign-in free account to a Google-sign-in plus account. Buyer emails, user ids, the payment id and the `apple_original_transaction_id` are deliberately omitted - see [[feedback-no-prod-metrics-in-public]]; re-resolve them from the DB with `scripts/db/inspect-accounts.mjs`. Result: source back to 0 GB purchased (free-tier quota), target +2 GB.
|
||||
|
||||
**Script: `scripts/db/transfer-storage-purchase.mjs`** (`--from` / `--to` emails, dry run by default, `--apply` writes), plus read-only `scripts/db/inspect-accounts.mjs`. Both need `node --env-file=.env --env-file=.env.local` and must live in the repo, not the scratchpad (ESM resolves `node_modules` from the FILE).
|
||||
**Script: `scripts/db/transfer-storage-purchase.mjs`** (NEVER COMMITTED, gone by 2026-09-07; `inspect-accounts.mjs` was rebuilt, see [[account-merge-recipe]]) (`--from` / `--to` emails, dry run by default, `--apply` writes), plus read-only `scripts/db/inspect-accounts.mjs`. Both need `node --env-file=.env --env-file=.env.local` and must live in the repo, not the scratchpad (ESM resolves `node_modules` from the FILE).
|
||||
|
||||
**Reassign the store row, do NOT credit the target with a synthetic +N and cancel the source with a -N.** The store dedupe key has to follow the entitlement:
|
||||
- Left on the source, a Restore Purchases re-verifies that transaction and `createOrUpdatePayment` rewrites `status`/`storage_gb` from the product id, so a `refunded`/zeroed row silently re-credits.
|
||||
|
|
|
|||
|
|
@ -53,9 +53,9 @@ iOS home indicator). Upstream ba3490b3e "Use workspace dependency management
|
|||
CarPlay and showed the CPNowPlaying template with the current book, process
|
||||
alive, no crash report, no NSGenericException. So upstream 0.37.0 is FINE
|
||||
for CarPlay despite the TaoSceneDelegate override; the theoretical concern
|
||||
above did NOT materialize. `packages/tao` submodule REMOVED in local dev
|
||||
commit a362ac598 (2026-09-06, `git submodule deinit` + `git rm`, workflow
|
||||
comment updated; NOT pushed). Not yet checked: iOS 26.3 sim, real device.
|
||||
above did NOT materialize. `packages/tao` submodule REMOVED: PR #6085
|
||||
(branch `chore/remove-tao-submodule` on origin, rebased onto 77e44eb92;
|
||||
`git submodule deinit` + `git rm`, workflow comment updated). Not yet checked: iOS 26.3 sim, real device.
|
||||
|
||||
Also: `rust-version` bumped 1.77.2 -> 1.90 in root `[workspace.package]` and
|
||||
`src-tauri/Cargo.toml` (user request). MSRV-aware fallback is NOT active with
|
||||
|
|
@ -110,9 +110,9 @@ Outcome: shipped as PR #6081 "chore: bump tauri to version 2.11.5" (head on the
|
|||
CI fixes: cargoHash + pnpmDeps hash bumps and the `as_chunks` clippy rewrite in
|
||||
epub_parser.rs. Main then took #6083 (nix Linux package on CEF). The temporary
|
||||
fork tag `cef-stub-old-base` was DELETED once main pinned 3156d92b7; the
|
||||
leftover `packages/tauri-plugins/` dir is gone. STILL OPEN: `packages/tao`
|
||||
submodule is registered in .gitmodules but unused (remove in a follow-up PR);
|
||||
iOS build with the relabelled swift-rs + upstream tao 0.37 not yet verified.
|
||||
leftover `packages/tauri-plugins/` dir is gone. `packages/tao` was removed by
|
||||
PR #6085. The iOS 18.5 simulator build and CarPlay check passed as recorded above;
|
||||
iOS 26.3 simulator and real-device verification remain open.
|
||||
|
||||
Sim recipe that worked (2026-09-06): boot `xcrun simctl boot <udid>` + `open -a
|
||||
Simulator` FIRST, `pnpm dev-ios-sim` (Next export + cargo sim build + xcodebuild
|
||||
|
|
|
|||
|
|
@ -0,0 +1,25 @@
|
|||
---
|
||||
name: two-column-spine-curl-6107
|
||||
description: "PR #6107 (issue #6106) two-column Page Curl hinged at the spine — review findings, iOS-only cover, deterministic test failure, Xiaomi handoff"
|
||||
metadata:
|
||||
node_type: memory
|
||||
type: project
|
||||
originSessionId: ce290db5-d7d6-4b12-b726-6e3f5331a846
|
||||
modified: 2026-09-07T08:17:04.736Z
|
||||
---
|
||||
|
||||
PR #6107 by @kad-air (branch `feat/two-column-spine-curl`, worktree `/Users/chrox/dev/readest-pr-6107`), closes #6106. Reviewed 2026-09-07; CI all green; NOT merged at review time.
|
||||
|
||||
**What it does:** with `renderer.columnCount >= 2` the curl turns only the outer column as a leaf hinged at the spine (`uLeaf` open x-interval in the vertex shader; radius decays to 0 so progress 1 is an exact reflection about x=w/2). The leaf's back samples a second texture — a capture of the incoming inner column taken *after* the instant nav, under a native iOS cover view (`cover_webview_region`/`uncover_webview_region`, a `resizableSnapshotView` sibling above the WKWebView that `takeSnapshot` does not render). Android/desktop reject the cover, so the back is theme paper and the canvas fades over the last 20% (`LEAF_FADE_START`).
|
||||
|
||||
**Math verified by hand (correct):** a leaf point at `w/2+d` lands at `w/2−d` and samples incoming `u = 2 − 2·vUv.x = 1 − 2d/w`, exactly that point's position inside the captured inner half; the rtl/backward branch (`u = 1 − 2·vUv.x`) matches too. Because both the hinge and `#captureIncoming` use `rect.width/2`, the landed leaf is pixel-exact against the live page **regardless of where the true gutter is** — asymmetric margins / landscape notch only shift the crease line by `(marginLeft − marginRight)/8`, a few px.
|
||||
|
||||
**Findings reported to chrox (not posted to the PR):**
|
||||
1. `captured-turn.browser.test.ts > commits the incoming column when no overlay interferes` **fails deterministically on this Mac** (4/4, isolated, idle machine) while CI is green. Instrumented cause: the `this.#active !== active` guard after `createImageBitmap` (capturedTurn.ts ~line 844) bails because the turn already ended — headless Chromium's software WebGL makes the two `waitForPaint()` double-rAFs cost ~570 ms vs the 120 ms wait + 450 ms animation. Product-safe (losing the race degrades to paper back + fade) but the test asserts a race the code does not guarantee.
|
||||
2. `INCOMING_WAIT_MS = 120` is awaited *before* `#playTo`, so every programmatic two-column turn on iOS freezes on the flat captured page for up to 120 ms before the curl starts. The wait could overlap the early animation (the back is barely visible at low progress).
|
||||
3. Late `setIncoming` flips `canvas.style.opacity` from a partly-faded value back to `''` → one-frame brightness pop if the capture lands after progress 0.8.
|
||||
4. Fixed-layout (`fixed-layout.js` has no `columnCount`) keeps the old whole-spread curl — PDF/FXL two-page spreads are not covered.
|
||||
|
||||
**Device state:** built + installed on the Xiaomi 13 (`pnpm dev-android` from the worktree, md5 `b158c52e21fe4e81308ce8c3cc405d89`, matches the device APK, installed 2026-09-07 16:13). CDP-probed: portrait `columnCount 1`, landscape `columnCount 2`, `captured-turn-style="curl"` — the leaf path IS reachable there, and a `view.next()` turn completes with no JS error. Android therefore exercises the **paper-back + fade** fallback only; the incoming-page-on-the-back half is iOS-only.
|
||||
|
||||
**CDP gotcha:** `Page.captureScreenshot` against the Android WebView appears to stall the renderer's rAF loop — five back-to-back shots during a 450 ms turn all came back at progress ~0 (old spread), then the final frame showed the new spread. Mid-turn frames need a held drag (touchMove without release), not screenshot polling; and my synthetic touchMove sequence was consumed as swipe-flips (pages advanced) rather than a scrubbed drag. See [[feedback-always-verify-on-xiaomi]], [[page-turn-styles-viewtransitions-555]].
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
---
|
||||
name: xml-lang-namespace-selector-6088
|
||||
description: "#6088 book CSS [xml|lang=\"en\"] and :lang() never matched xml:lang in EPUB sections; the srcdoc HTML parse drops the namespace and applyNamespacedAttributes only knew declared prefixes; body{color} loss in the same report is BY DESIGN; getParagraphLayoutStyles is omitted when Use Book Layout is ON, so body{line-height:unset} is not applied and the book's body line-height inherits"
|
||||
metadata:
|
||||
type: project
|
||||
---
|
||||
|
||||
Issue #6088 (email report, Windows 11, 0.12.6, 2026-09-05): a Sigil test book with
|
||||
`@namespace xml "http://www.w3.org/XML/1998/namespace"; [xml|lang="en"]{color:red}` and
|
||||
`<p xml:lang="en">` renders in the theme color. `p.matches(':lang(en)')` was `false` too.
|
||||
|
||||
**Root cause:** sections load via `iframe.srcdoc` (HTML parse, see
|
||||
[[srcdoc-html-parsing-namespaced-attrs-6038]]), so `xml:lang` is a null-namespace attribute
|
||||
named literally `xml:lang`. `applyNamespacedAttributes` (`src/utils/style.ts`) re-attached only
|
||||
prefixes declared with `xmlns:*` (+ a fixed `epub` fallback); `xml` is bound by XML itself and is
|
||||
NEVER declared, so it was skipped. An older test even asserted that skip as intended.
|
||||
|
||||
**Fix (MERGED #6090, squash 04379b8f8 on main, 2026-09-06; UNRELEASED):** `prefix === 'xml'` -> XML namespace twin via
|
||||
`setAttributeNS`. Side benefit: `:lang()` (hyphenation, our `:lang(zh|ja|ko)` widows rule) now sees
|
||||
`xml:lang`-only books. Playwright dev-web probe: before `rgb(23,23,23)`/false/null, after
|
||||
`rgb(255,0,0)`/true/"en". Repro + `probe.cjs` archived in `~/Documents/books/issues/6088/`.
|
||||
|
||||
**Same reporter's second pair (body-color.epub / body-color-important.epub) is BY DESIGN
|
||||
(chrox decision 2026-09-06):** with override OFF, `body{color:red; line-height:3}` still loses:
|
||||
Readest's stylesheet is APPENDED after the book's (`html,body{color}` wins on equal specificity),
|
||||
`body{line-height:unset}` is an explicit "dirty hack" in `getPageLayoutStyles`, and
|
||||
`p{line-height}` beats inheritance. `!important` in the book flips color only. **Use Book Layout
|
||||
did NOT help either** (verified via the settings UI in the probe): `body{line-height:unset}` sat in
|
||||
the always-on page-layout chunk, so `<p>` got `normal`, never the book's 3. FIXED (2nd commit of #6090, in the same squash): the reset moved into `getParagraphLayoutStyles`, so Use Book Layout ON -> 48px at 16px,
|
||||
OFF unchanged 22.4px. Color stays theme-controlled BY DESIGN. foliate's fork
|
||||
has an UNUSED `setStyles([before, after])` slot ("defaults the book can override") if this is ever
|
||||
revisited; moving `html,body{color}` there would make dark-mode books with `body{color:#333}`
|
||||
unreadable, which is why it was declined.
|
||||
|
||||
Related: [[srcdoc-html-parsing-namespaced-attrs-6038]], [[footnote-aside-namespace-order-4438]],
|
||||
[[css-style-fixes]].
|
||||
|
|
@ -201,6 +201,32 @@ the same port — `chrome://inspect` in a local Chrome, or
|
|||
parses the same argv for open-with paths and warns on every launch. Prefer the
|
||||
env var.
|
||||
|
||||
## crengine XPointer Oracle (KOReader sync)
|
||||
|
||||
KOReader stores positions as crengine XPointers; `src/utils/xcfi.ts` converts them to and from
|
||||
CFIs. The only trustworthy reference for what crengine emits is crengine itself, so
|
||||
`apps/readest.koplugin/scripts/xpointer-oracle.lua` runs headlessly inside a KOReader emulator
|
||||
build and dumps crengine's XPointer pair and text for every visible word of an EPUB:
|
||||
|
||||
```bash
|
||||
cd <koreader-emulator>/koreader
|
||||
KO_HOME=/tmp/ko-oracle ./luajit /path/to/readest/apps/readest.koplugin/scripts/xpointer-oracle.lua \
|
||||
/path/to/book.epub /path/to/book.crengine.json 40 # keep one word in 40
|
||||
```
|
||||
|
||||
`src/__tests__/utils/xcfi.crengine-oracle.test.ts` then checks both sync directions for every
|
||||
sampled word: crengine's pointers must resolve to exactly that word, and a highlight on that word
|
||||
must convert back to exactly crengine's pointers. Committed fixtures live in
|
||||
`src/__tests__/fixtures/crengine/` (generated from EPUBs under `fixtures/data/`). To validate a
|
||||
local book that cannot be committed:
|
||||
|
||||
```bash
|
||||
XPOINTER_ORACLE=/path/to/book.crengine.json pnpm test src/__tests__/utils/xcfi.crengine-oracle.test.ts
|
||||
```
|
||||
|
||||
The JSON's `epub` field is an absolute path or a file next to the JSON. The emulator is a local
|
||||
build (`kodev` in a KOReader checkout); nothing in CI needs it.
|
||||
|
||||
## Test File Naming
|
||||
|
||||
| Suffix | Runner | Environment |
|
||||
|
|
|
|||
3648
apps/readest-app/src/__tests__/fixtures/crengine/sample-alice.json
Normal file
3648
apps/readest-app/src/__tests__/fixtures/crengine/sample-alice.json
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -13,7 +13,8 @@ import { describe, expect, it } from 'vitest';
|
|||
|
||||
import { EPUB } from 'foliate-js/epub.js';
|
||||
import type { BookDoc } from '@/libs/document';
|
||||
import { getCFIFromXPointer } from '@/utils/xcfi';
|
||||
import * as CFI from 'foliate-js/epubcfi.js';
|
||||
import { getCFIFromXPointer, getXPointerFromCFI, XCFI } from '@/utils/xcfi';
|
||||
|
||||
const CONTAINER = `<?xml version="1.0" encoding="UTF-8"?>
|
||||
<container version="1.0" xmlns="urn:oasis:names:tc:opendocument:xmlns:container">
|
||||
|
|
@ -54,7 +55,7 @@ const malformed = `<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-GB">
|
|||
</body>
|
||||
</html>`;
|
||||
|
||||
type Section = { createDocument: () => Promise<Document> };
|
||||
type Section = { createDocument: () => Promise<Document>; load: () => Promise<string> };
|
||||
|
||||
const openEpub = async (files: Record<string, string>) => {
|
||||
const epub = new EPUB({
|
||||
|
|
@ -122,3 +123,123 @@ describe('createDocument on a section that is not well-formed XML (#5625)', () =
|
|||
expect(cfi).toMatch(/^epubcfi\(/);
|
||||
});
|
||||
});
|
||||
|
||||
// #5271: the HTML retry is not a faithful parse of such a file. The HTML
|
||||
// parser ignores `/>` on non-void elements and re-opens formatting elements
|
||||
// across blocks, so an InDesign page anchor `<a id="page_25"/>` at the top of
|
||||
// a paragraph swallows every following <p> until the next anchor. Positions
|
||||
// then disagree with the book's real structure and with KOReader's crengine,
|
||||
// which honours `/>`. When the only fault is unclosed void tags, closing them
|
||||
// and re-parsing as XML gives the intended DOM; HTML stays the last resort.
|
||||
const inDesign = `<html xmlns="http://www.w3.org/1999/xhtml">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<title>Chapter 3</title>
|
||||
</head>
|
||||
<body>
|
||||
<p class="cn" id="ch3"><a id="page_25"/><span class="hide"> </span>3</p>
|
||||
<p class="ct1">IT’S NOT YOU, IT’S US</p>
|
||||
<p class="bodytext1">When someone you love has just died, why does it matter?</p>
|
||||
<p class="bodytext">Your personal experience is <a id="page_26"/>affected by the wider culture.</p>
|
||||
<p class="bodytextb">You aren’t crazy. The culture is crazy.<br>It’s not you.</p>
|
||||
</body>
|
||||
</html>`;
|
||||
|
||||
// Still not XML after closing void tags: a bare ampersand.
|
||||
const beyondRepair = `<html xmlns="http://www.w3.org/1999/xhtml">
|
||||
<head><meta charset="utf-8"><title>Notes</title></head>
|
||||
<body><p>Rock & roll</p></body>
|
||||
</html>`;
|
||||
|
||||
const openInDesignFixture = () =>
|
||||
openEpub({
|
||||
'META-INF/container.xml': CONTAINER,
|
||||
'OEBPS/content.opf': opf([
|
||||
{ id: 'ch1', href: 'ch1.html' },
|
||||
{ id: 'ch3', href: 'ch3.html' },
|
||||
{ id: 'notes', href: 'notes.html' },
|
||||
]),
|
||||
'OEBPS/ch1.html': wellFormed,
|
||||
'OEBPS/ch3.html': inDesign,
|
||||
'OEBPS/notes.html': beyondRepair,
|
||||
});
|
||||
|
||||
const XHTML_NS = 'http://www.w3.org/1999/xhtml';
|
||||
|
||||
/** Text covered by a range CFI, resolved in `doc` like the reader does. */
|
||||
const rangeText = (doc: Document, cfi: string) => {
|
||||
const parts = CFI.parse(cfi);
|
||||
(parts.parent ?? parts).shift(); // drop the spine step
|
||||
return CFI.toRange(doc, parts).toString();
|
||||
};
|
||||
|
||||
const bodyTags = (doc: Document) =>
|
||||
Array.from(doc.body.children).map((el) => el.tagName.toLowerCase());
|
||||
|
||||
describe('a file whose only fault is unclosed void tags is repaired as XML (#5271)', () => {
|
||||
it('createDocument keeps the paragraphs under <body> and the anchors empty', async () => {
|
||||
const sections = await openInDesignFixture();
|
||||
const doc = await sections[1]!.createDocument();
|
||||
|
||||
expect(doc.querySelector('parsererror')).toBeNull();
|
||||
expect(doc.documentElement.namespaceURI).toBe(XHTML_NS);
|
||||
expect(bodyTags(doc)).toEqual(['p', 'p', 'p', 'p', 'p']);
|
||||
expect(doc.getElementById('page_25')!.childNodes.length).toBe(0);
|
||||
expect(doc.getElementById('page_26')!.childNodes.length).toBe(0);
|
||||
expect(doc.querySelector('br')!.namespaceURI).toBe(XHTML_NS);
|
||||
});
|
||||
|
||||
it('converts positions the way crengine numbers this chapter', async () => {
|
||||
const sections = await openInDesignFixture();
|
||||
const bookDoc = { sections } as unknown as BookDoc;
|
||||
const doc = await sections[1]!.createDocument();
|
||||
|
||||
// Readest -> KOReader: a highlight on the third paragraph.
|
||||
const node = doc.body.children[2]!.firstChild as Text;
|
||||
const range = doc.createRange();
|
||||
range.setStart(node, 0);
|
||||
range.setEnd(node, 12);
|
||||
const cfi = CFI.joinIndir(CFI.fake.fromIndex(1), CFI.fromRange(range));
|
||||
const xp = await getXPointerFromCFI(cfi, undefined, undefined, bookDoc);
|
||||
expect(xp.pos0).toBe('/body/DocFragment[2]/body/p[3]/text().0');
|
||||
expect(xp.pos1).toBe('/body/DocFragment[2]/body/p[3]/text().12');
|
||||
|
||||
// KOReader -> Readest: the text after the mid-paragraph anchor is the
|
||||
// paragraph's second text child in crengine's DOM.
|
||||
const back = new XCFI(doc, 1).xPointerToCFI(
|
||||
'/body/DocFragment[2]/body/p[4]/text()[2].0',
|
||||
'/body/DocFragment[2]/body/p[4]/text()[2].8',
|
||||
);
|
||||
expect(rangeText(doc, back)).toBe('affected');
|
||||
});
|
||||
|
||||
it('renders the repaired XML, not the HTML re-parse', async () => {
|
||||
const originalCreate = URL.createObjectURL;
|
||||
const blobs: Blob[] = [];
|
||||
URL.createObjectURL = (blob: Blob) => {
|
||||
blobs.push(blob);
|
||||
return `blob:test/${blobs.length}`;
|
||||
};
|
||||
try {
|
||||
const sections = await openInDesignFixture();
|
||||
await sections[1]!.load();
|
||||
} finally {
|
||||
URL.createObjectURL = originalCreate;
|
||||
}
|
||||
expect(blobs).toHaveLength(1);
|
||||
const rendered = new DOMParser().parseFromString(
|
||||
await blobs[0]!.text(),
|
||||
'application/xhtml+xml',
|
||||
);
|
||||
expect(rendered.querySelector('parsererror')).toBeNull();
|
||||
expect(bodyTags(rendered)).toEqual(['p', 'p', 'p', 'p', 'p']);
|
||||
});
|
||||
|
||||
it('still falls back to HTML when the file is broken beyond that repair', async () => {
|
||||
const sections = await openInDesignFixture();
|
||||
const doc = await sections[2]!.createDocument();
|
||||
|
||||
expect(doc.body).not.toBeNull();
|
||||
expect(doc.body.querySelector('p')?.textContent).toBe('Rock & roll');
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -0,0 +1,119 @@
|
|||
// Validates Readest's CFI <-> XPointer conversion against XPointers produced
|
||||
// by the real crengine (KOReader's engine), captured with
|
||||
// apps/readest.koplugin/scripts/xpointer-oracle.lua.
|
||||
//
|
||||
// Every fixture under src/__tests__/fixtures/crengine/ lists, for each spine
|
||||
// item, crengine's XPointer pair and text for a sample of visible words. Both
|
||||
// sync directions are checked for every word:
|
||||
// KOReader -> Readest: xPointerToCFI(xp, xp_end) must land on exactly that word
|
||||
// Readest -> KOReader: a CFI built on that word must convert back to exactly
|
||||
// crengine's xp / xp_end
|
||||
//
|
||||
// Ad hoc validation of any local EPUB (not committed, e.g. a reporter's book):
|
||||
// XPOINTER_ORACLE=/path/to/book.crengine.json pnpm test src/__tests__/utils/xcfi.crengine-oracle.test.ts
|
||||
// where the JSON's `epub` field is an absolute path or lives next to the JSON.
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { existsSync, readdirSync, readFileSync } from 'fs';
|
||||
import { dirname, isAbsolute, join, resolve } from 'path';
|
||||
import * as CFI from 'foliate-js/epubcfi.js';
|
||||
import { DocumentLoader } from '@/libs/document';
|
||||
import type { BookDoc } from '@/libs/document';
|
||||
import { XCFI } from '@/utils/xcfi';
|
||||
|
||||
type OracleWord = { xp: string; xp_end: string; text: string };
|
||||
type Oracle = {
|
||||
epub: string;
|
||||
fragments: { index: number; docfragment: number; word_count: number; words: OracleWord[] }[];
|
||||
};
|
||||
|
||||
const FIXTURES = resolve(__dirname, '../fixtures/crengine');
|
||||
const DATA = resolve(__dirname, '../fixtures/data');
|
||||
|
||||
const oracles = (): { name: string; oracle: Oracle; epub: string }[] => {
|
||||
const override = process.env['XPOINTER_ORACLE'];
|
||||
const files = override
|
||||
? [override]
|
||||
: readdirSync(FIXTURES)
|
||||
.filter((f) => f.endsWith('.json'))
|
||||
.map((f) => join(FIXTURES, f));
|
||||
return files.map((file) => {
|
||||
const oracle = JSON.parse(readFileSync(file, 'utf8')) as Oracle;
|
||||
const candidates = isAbsolute(oracle.epub)
|
||||
? [oracle.epub]
|
||||
: [join(dirname(file), oracle.epub), join(DATA, oracle.epub)];
|
||||
const epub = candidates.find((p) => existsSync(p));
|
||||
if (!epub) throw new Error(`EPUB for ${file} not found: ${candidates.join(', ')}`);
|
||||
return { name: oracle.epub, oracle, epub };
|
||||
});
|
||||
};
|
||||
|
||||
const openBook = async (file: string): Promise<BookDoc> => {
|
||||
const buffer = readFileSync(file);
|
||||
const f = new File([buffer], file.split('/').pop()!, { type: 'application/epub+zip' });
|
||||
const { book } = await new DocumentLoader(f).open();
|
||||
return book;
|
||||
};
|
||||
|
||||
// crengine writes /body/DocFragment/... (no index) when the book has a single
|
||||
// spine item; it resolves the indexed form too, and Readest always emits it.
|
||||
const indexed = (xp: string) => xp.replace(/^\/body\/DocFragment\//, '/body/DocFragment[1]/');
|
||||
|
||||
const cases = oracles();
|
||||
|
||||
describe.each(cases)('crengine XPointer oracle: $name', ({ oracle, epub }) => {
|
||||
it('converts every sampled word both ways exactly as crengine does', async () => {
|
||||
const book = await openBook(epub);
|
||||
expect(book.sections!.length).toBe(oracle.fragments.length);
|
||||
|
||||
const pull: string[] = [];
|
||||
const push: string[] = [];
|
||||
let words = 0;
|
||||
for (const fragment of oracle.fragments) {
|
||||
const { index } = fragment;
|
||||
const section = book.sections![index]!;
|
||||
const doc = await section.createDocument();
|
||||
const converter = new XCFI(doc, index);
|
||||
const base = section.cfi ?? CFI.fake.fromIndex(index);
|
||||
|
||||
for (const word of fragment.words) {
|
||||
words++;
|
||||
const xp = indexed(word.xp);
|
||||
const xpEnd = indexed(word.xp_end);
|
||||
|
||||
// KOReader -> Readest: the range must cover exactly crengine's word.
|
||||
let range: Range | null = null;
|
||||
try {
|
||||
const cfi = converter.xPointerToCFI(xp, xpEnd);
|
||||
const anchored = book.resolveCFI!(cfi)?.anchor?.(doc);
|
||||
const got =
|
||||
!anchored || typeof anchored === 'number' ? '<unresolved>' : anchored.toString();
|
||||
if (got === word.text) range = anchored as Range;
|
||||
else pull.push(`${xp}: expected "${word.text}", got "${got}"`);
|
||||
} catch (e) {
|
||||
pull.push(`${xp}: ${String(e).split('\n')[0]}`);
|
||||
}
|
||||
if (!range) continue; // no trusted location to push from; reported above
|
||||
|
||||
// Readest -> KOReader: a highlight on that range must come back as
|
||||
// crengine's own pointers.
|
||||
try {
|
||||
const cfi = CFI.joinIndir(base, CFI.fromRange(range));
|
||||
const back = converter.cfiToXPointer(cfi);
|
||||
if (back.pos0 !== xp || back.pos1 !== xpEnd) {
|
||||
push.push(
|
||||
`"${word.text}": expected ${xp} .. ${xpEnd}, got ${back.pos0} .. ${back.pos1}`,
|
||||
);
|
||||
}
|
||||
} catch (e) {
|
||||
push.push(`"${word.text}": ${String(e).split('\n')[0]}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const summarize = (label: string, list: string[]) =>
|
||||
`${label}: ${list.length}/${words} mismatches\n ${list.slice(0, 10).join('\n ')}`;
|
||||
// Report both directions in one run rather than stopping at the first.
|
||||
expect.soft(pull, summarize('KOReader -> Readest', pull)).toEqual([]);
|
||||
expect.soft(push, summarize('Readest -> KOReader', push)).toEqual([]);
|
||||
}, 120000);
|
||||
});
|
||||
|
|
@ -0,0 +1,188 @@
|
|||
// XPointer semantics pinned against the real crengine (KOReader's engine),
|
||||
// found with apps/readest.koplugin/scripts/xpointer-oracle.lua. See
|
||||
// crengine/src/lvtinydom.cpp ldomXPointer::toString: a text step is
|
||||
// `/text()[K]` with K counting the parent's TEXT children only, and the `[K]`
|
||||
// is omitted when the parent has a single text child. The offset is within
|
||||
// that text node, never cumulative over the element's descendants. Likewise
|
||||
// `DocFragment[N]` loses its `[N]` when the book has a single spine item.
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import * as CFI from 'foliate-js/epubcfi.js';
|
||||
import { XCFI } from '@/utils/xcfi';
|
||||
|
||||
const html = (body: string) =>
|
||||
new DOMParser().parseFromString(`<html><body>${body}</body></html>`, 'text/html');
|
||||
|
||||
/** Text covered by a range CFI, resolved in `doc` like the reader does. */
|
||||
const rangeText = (doc: Document, cfi: string) => {
|
||||
const parts = CFI.parse(cfi);
|
||||
(parts.parent ?? parts).shift(); // drop the spine step
|
||||
return CFI.toRange(doc, parts).toString();
|
||||
};
|
||||
|
||||
describe('unindexed /text().N is the element’s only text child, offset N inside it', () => {
|
||||
// sample-alice.epub chapter headings: crengine emits h2/text().1 for the
|
||||
// "D" of "Down", i.e. offset 1 of the "\nDown the Rabbit Hole" text node.
|
||||
// Cumulative-over-descendants reading lands on "hapter" instead.
|
||||
const doc = html(
|
||||
'<h2><span class="chapterHeader"><span class="translation">Chapter</span> <span class="count">1</span></span>\nDown the Rabbit Hole</h2>',
|
||||
);
|
||||
const converter = new XCFI(doc, 3);
|
||||
|
||||
it('resolves a KOReader range after inline children to the right word', () => {
|
||||
const cfi = converter.xPointerToCFI(
|
||||
'/body/DocFragment[4]/body/h2/text().1',
|
||||
'/body/DocFragment[4]/body/h2/text().5',
|
||||
);
|
||||
expect(rangeText(doc, cfi)).toBe('Down');
|
||||
});
|
||||
|
||||
it('round-trips its own XPointer for a selection after an inline child', () => {
|
||||
const doc2 = html('<p><em>x</em> rest of it</p>');
|
||||
const c = new XCFI(doc2, 0);
|
||||
const node = doc2.querySelector('p')!.lastChild as Text; // " rest of it"
|
||||
const range = doc2.createRange();
|
||||
range.setStart(node, 1);
|
||||
range.setEnd(node, 5);
|
||||
const cfi = CFI.joinIndir(CFI.fake.fromIndex(0), CFI.fromRange(range));
|
||||
const xp = c.cfiToXPointer(cfi);
|
||||
expect(xp.pos0).toBe('/body/DocFragment[1]/body/p/text().1');
|
||||
expect(rangeText(doc2, c.xPointerToCFI(xp.pos0!, xp.pos1!))).toBe('rest');
|
||||
});
|
||||
|
||||
it('skips the whitespace-only text node crengine drops at the start of a block', () => {
|
||||
// crengine parses "<p>\n<em>delta</em>\nepsilon zeta\n</p>" as
|
||||
// <p><em>delta</em> epsilon zeta </p>: the leading whitespace node is gone
|
||||
// and " epsilon zeta " is the ONLY text child, so it writes p/text().1.
|
||||
const doc3 = html('<p>\n<em>delta</em>\nepsilon zeta\n</p>');
|
||||
const c = new XCFI(doc3, 0);
|
||||
const cfi = c.xPointerToCFI(
|
||||
'/body/DocFragment[1]/body/p/text().1',
|
||||
'/body/DocFragment[1]/body/p/text().8',
|
||||
);
|
||||
expect(rangeText(doc3, cfi)).toBe('epsilon');
|
||||
});
|
||||
});
|
||||
|
||||
describe('indexed /text()[K].N counts text children the way crengine keeps them', () => {
|
||||
// crengine: <p><em>b</em> c <em>d</em> e</p> — the leading "\n" is dropped,
|
||||
// " c " and " e" survive, so "e" is text()[2].1 (not [3] as raw DOM counting says).
|
||||
const doc = html('<p>\n<em>b</em> c <em>d</em> e</p>');
|
||||
const converter = new XCFI(doc, 0);
|
||||
|
||||
it('resolves text()[2] to the second surviving text child', () => {
|
||||
const cfi = converter.xPointerToCFI(
|
||||
'/body/DocFragment[1]/body/p/text()[2].1',
|
||||
'/body/DocFragment[1]/body/p/text()[2].2',
|
||||
);
|
||||
expect(rangeText(doc, cfi)).toBe('e');
|
||||
});
|
||||
|
||||
it('emits text()[2] for a selection in that text child', () => {
|
||||
const node = doc.querySelector('p')!.lastChild as Text; // " e"
|
||||
const range = doc.createRange();
|
||||
range.setStart(node, 1);
|
||||
range.setEnd(node, 2);
|
||||
const cfi = CFI.joinIndir(CFI.fake.fromIndex(0), CFI.fromRange(range));
|
||||
const xp = converter.cfiToXPointer(cfi);
|
||||
expect(xp.pos0).toBe('/body/DocFragment[1]/body/p/text()[2].1');
|
||||
expect(xp.pos1).toBe('/body/DocFragment[1]/body/p/text()[2].2');
|
||||
});
|
||||
});
|
||||
|
||||
describe('dropped leading whitespace stays at the block start', () => {
|
||||
it.each([
|
||||
'',
|
||||
'\nepsilon',
|
||||
'\nepsilon<em>zeta</em>eta',
|
||||
])('preserves a selection starting before inline content with trailing text %j', (trailingText) => {
|
||||
const doc = html(`<p>\n<em>delta</em>${trailingText}</p>`);
|
||||
const converter = new XCFI(doc, 0);
|
||||
const range = doc.createRange();
|
||||
range.setStart(doc.querySelector('p')!.firstChild!, 0);
|
||||
range.setEnd(doc.querySelector('em')!.firstChild!, 5);
|
||||
const cfi = CFI.joinIndir(CFI.fake.fromIndex(0), CFI.fromRange(range));
|
||||
|
||||
const xp = converter.cfiToXPointer(cfi);
|
||||
|
||||
expect(xp.pos0).toBe('/body/DocFragment[1]/body/p');
|
||||
expect(rangeText(doc, converter.xPointerToCFI(xp.pos0!, xp.pos1!)).trim()).toBe('delta');
|
||||
});
|
||||
});
|
||||
|
||||
describe('offsets count crengine’s whitespace-collapsed text, not the raw source', () => {
|
||||
// crengine keeps "lambda mu nu" as "lambda mu nu": "mu" is text().7..9
|
||||
// there, while the raw DOM text node has it at 9..11.
|
||||
const doc = html('<p>lambda mu nu</p>');
|
||||
const converter = new XCFI(doc, 0);
|
||||
|
||||
it('maps a crengine offset back to the raw text node', () => {
|
||||
const cfi = converter.xPointerToCFI(
|
||||
'/body/DocFragment[1]/body/p/text().7',
|
||||
'/body/DocFragment[1]/body/p/text().9',
|
||||
);
|
||||
expect(rangeText(doc, cfi)).toBe('mu');
|
||||
});
|
||||
|
||||
it('emits collapsed offsets for a raw selection', () => {
|
||||
const node = doc.querySelector('p')!.firstChild as Text;
|
||||
const range = doc.createRange();
|
||||
range.setStart(node, 9);
|
||||
range.setEnd(node, 11);
|
||||
const cfi = CFI.joinIndir(CFI.fake.fromIndex(0), CFI.fromRange(range));
|
||||
const xp = converter.cfiToXPointer(cfi);
|
||||
expect(xp.pos0).toBe('/body/DocFragment[1]/body/p/text().7');
|
||||
expect(xp.pos1).toBe('/body/DocFragment[1]/body/p/text().9');
|
||||
});
|
||||
});
|
||||
|
||||
describe('preformatted text keeps its whitespace, so offsets are raw there', () => {
|
||||
// crengine: <pre>alpha2 beta2</pre> has "beta2" at pre/text()[1].8, and the
|
||||
// same holds inside inline <code> outside any <pre>: "eta2" at code/text().8.
|
||||
const doc = html('<pre>alpha2 beta2\n gamma2</pre><p><code>zeta2 eta2</code> theta2</p>');
|
||||
|
||||
it.each([
|
||||
[
|
||||
'pre',
|
||||
'/body/DocFragment[1]/body/pre/text().8',
|
||||
'/body/DocFragment[1]/body/pre/text().13',
|
||||
'beta2',
|
||||
],
|
||||
[
|
||||
'code',
|
||||
'/body/DocFragment[1]/body/p/code/text().8',
|
||||
'/body/DocFragment[1]/body/p/code/text().12',
|
||||
'eta2',
|
||||
],
|
||||
])('resolves a raw offset inside %s', (_tag, start, end, word) => {
|
||||
const cfi = new XCFI(doc, 0).xPointerToCFI(start, end);
|
||||
expect(rangeText(doc, cfi)).toBe(word);
|
||||
});
|
||||
|
||||
it('emits raw offsets for a selection inside code', () => {
|
||||
const node = doc.querySelector('code')!.firstChild as Text;
|
||||
const range = doc.createRange();
|
||||
range.setStart(node, 8);
|
||||
range.setEnd(node, 12);
|
||||
const cfi = CFI.joinIndir(CFI.fake.fromIndex(0), CFI.fromRange(range));
|
||||
const xp = new XCFI(doc, 0).cfiToXPointer(cfi);
|
||||
expect(xp.pos0).toBe('/body/DocFragment[1]/body/p/code/text().8');
|
||||
expect(xp.pos1).toBe('/body/DocFragment[1]/body/p/code/text().12');
|
||||
});
|
||||
});
|
||||
|
||||
describe('a single-spine-item book has /body/DocFragment/ with no index', () => {
|
||||
const doc = html('<p>only chapter</p>');
|
||||
|
||||
it('extracts spine index 0', () => {
|
||||
expect(XCFI.extractSpineIndex('/body/DocFragment/body/p/text().5')).toBe(0);
|
||||
});
|
||||
|
||||
it('converts crengine’s unindexed path to a CFI in section 0', () => {
|
||||
const cfi = new XCFI(doc, 0).xPointerToCFI(
|
||||
'/body/DocFragment/body/p/text().5',
|
||||
'/body/DocFragment/body/p/text().12',
|
||||
);
|
||||
expect(cfi.startsWith('epubcfi(/6/2!')).toBe(true);
|
||||
expect(rangeText(doc, cfi)).toBe('chapter');
|
||||
});
|
||||
});
|
||||
|
|
@ -12,6 +12,14 @@ type XPointer = {
|
|||
pos1?: string;
|
||||
};
|
||||
|
||||
/** Where an XPointer lands: an exact text node position when the pointer
|
||||
* names one, else a cumulative text offset within `element` (or its start). */
|
||||
type XPointerTarget = {
|
||||
element: Element;
|
||||
textOffset?: number;
|
||||
point?: { node: Text; offset: number };
|
||||
};
|
||||
|
||||
export class XCFI {
|
||||
private document: Document;
|
||||
private spineItemIndex: number;
|
||||
|
|
@ -34,12 +42,13 @@ export class XCFI {
|
|||
// CFI uses even numbers starting from 2: 2, 4, 6, 8, ...
|
||||
// Convert to 0-based: (step - 2) / 2 = 0, 1, 2, 3, ...
|
||||
return Math.floor((spineStep - 2) / 2);
|
||||
} else if (cfiOrXPath.startsWith('/body/DocFragment[')) {
|
||||
} else if (cfiOrXPath.startsWith('/body/DocFragment')) {
|
||||
// Note that all indices in XPointer/XPath are 1-based
|
||||
// but the text() offsets are 0-based
|
||||
const match = cfiOrXPath.match(/DocFragment\[(\d+)\]/);
|
||||
// but the text() offsets are 0-based. crengine omits the [N] when
|
||||
// the book has a single spine item: /body/DocFragment/body/...
|
||||
const match = cfiOrXPath.match(/^\/body\/DocFragment(?:\[(\d+)\])?\//);
|
||||
if (match) {
|
||||
return parseInt(match[1]!, 10) - 1;
|
||||
return match[1] ? parseInt(match[1], 10) - 1 : 0;
|
||||
}
|
||||
throw new Error('Cannot extract spine index from XPath');
|
||||
} else {
|
||||
|
|
@ -132,64 +141,38 @@ export class XCFI {
|
|||
* Convert a single point XPointer to CFI
|
||||
*/
|
||||
private convertPointXPointerToCFI(xpointer: string): string {
|
||||
const { element, textOffset } = this.parseXPointer(xpointer);
|
||||
|
||||
const { node, offset } = this.anchorPoint(this.parseXPointer(xpointer));
|
||||
const range = this.document.createRange();
|
||||
if (textOffset !== undefined) {
|
||||
const textNode = this.findTextNodeAtOffset(element, textOffset);
|
||||
if (textNode) {
|
||||
range.setStart(textNode.node, textNode.offset);
|
||||
range.setEnd(textNode.node, textNode.offset);
|
||||
} else {
|
||||
// Fallback to element positioning
|
||||
range.setStart(element, 0);
|
||||
range.setEnd(element, 0);
|
||||
}
|
||||
} else {
|
||||
range.setStart(element, 0);
|
||||
range.setEnd(element, 0);
|
||||
}
|
||||
|
||||
const cfi = fromRange(range);
|
||||
return this.adjustSpineIndex(cfi);
|
||||
range.setStart(node, offset);
|
||||
range.setEnd(node, offset);
|
||||
return this.adjustSpineIndex(fromRange(range));
|
||||
}
|
||||
|
||||
private convertRangeXPointerToCFI(startXPointer: string, endXPointer: string): string {
|
||||
const startInfo = this.parseXPointer(startXPointer);
|
||||
const endInfo = this.parseXPointer(endXPointer);
|
||||
|
||||
const start = this.anchorPoint(this.parseXPointer(startXPointer));
|
||||
const end = this.anchorPoint(this.parseXPointer(endXPointer));
|
||||
const range = this.document.createRange();
|
||||
if (startInfo.textOffset !== undefined) {
|
||||
const startTextNode = this.findTextNodeAtOffset(startInfo.element, startInfo.textOffset);
|
||||
if (startTextNode) {
|
||||
range.setStart(startTextNode.node, startTextNode.offset);
|
||||
} else {
|
||||
range.setStart(startInfo.element, 0);
|
||||
}
|
||||
} else {
|
||||
range.setStart(startInfo.element, 0);
|
||||
}
|
||||
range.setStart(start.node, start.offset);
|
||||
range.setEnd(end.node, end.offset);
|
||||
return this.adjustSpineIndex(fromRange(range));
|
||||
}
|
||||
|
||||
if (endInfo.textOffset !== undefined) {
|
||||
const endTextNode = this.findTextNodeAtOffset(endInfo.element, endInfo.textOffset);
|
||||
if (endTextNode) {
|
||||
range.setEnd(endTextNode.node, endTextNode.offset);
|
||||
} else {
|
||||
range.setEnd(endInfo.element, 0);
|
||||
}
|
||||
} else {
|
||||
range.setEnd(endInfo.element, 0);
|
||||
/** DOM position for a parsed XPointer, falling back to the element start. */
|
||||
private anchorPoint(target: XPointerTarget): { node: Node; offset: number } {
|
||||
if (target.point) return target.point;
|
||||
if (target.textOffset !== undefined) {
|
||||
const found = this.findTextNodeAtOffset(target.element, target.textOffset);
|
||||
if (found) return found;
|
||||
}
|
||||
|
||||
const cfi = fromRange(range);
|
||||
return this.adjustSpineIndex(cfi);
|
||||
return { node: target.element, offset: 0 };
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse XPointer string to extract element and text offset
|
||||
*
|
||||
* Supports three KOReader text reference formats:
|
||||
* - `/text().N` — cumulative character offset across all text in the element
|
||||
* - `/text().N` — offset N in the element's only text child (crengine
|
||||
* omits the `[K]` when there is a single text child)
|
||||
* - `/text()[K].N` — Kth direct text node child (1-based), offset N within that node
|
||||
* - `/tag[idx].N` — offset N directly on the last path element (no explicit
|
||||
* `text()` step); CREngine emits this when the target
|
||||
|
|
@ -197,7 +180,7 @@ export class XCFI {
|
|||
* content, e.g. `div[1].0`. Semantically equivalent to
|
||||
* `/tag[idx]/text().N`.
|
||||
*/
|
||||
private parseXPointer(xpointer: string): { element: Element; textOffset?: number } {
|
||||
private parseXPointer(xpointer: string): XPointerTarget {
|
||||
// Format: /text()[K].N — indexed text node with offset
|
||||
const indexedTextMatch = xpointer.match(/\/text\(\)\[(\d+)\]\.(\d+)$/);
|
||||
if (indexedTextMatch) {
|
||||
|
|
@ -210,15 +193,13 @@ export class XCFI {
|
|||
throw new Error(`Cannot resolve XPointer path: ${elementPath}`);
|
||||
}
|
||||
|
||||
// Find the Kth direct text node child and compute cumulative offset
|
||||
const textOffset = this.resolveIndexedTextNode(element, textNodeIndex, offsetInNode);
|
||||
return { element, textOffset };
|
||||
return { element, point: this.textChildPoint(element, textNodeIndex, offsetInNode) };
|
||||
}
|
||||
|
||||
// Format: /text().N — cumulative character offset
|
||||
// Format: /text().N — the element's only text child, offset N inside it
|
||||
const textOffsetMatch = xpointer.match(/\/text\(\)\.(\d+)$/);
|
||||
if (textOffsetMatch) {
|
||||
const textOffset = parseInt(textOffsetMatch[1]!, 10);
|
||||
const offsetInNode = parseInt(textOffsetMatch[1]!, 10);
|
||||
const elementPath = xpointer.replace(/\/text\(\)\.\d+$/, '');
|
||||
|
||||
const element = this.resolveXPointerPath(elementPath);
|
||||
|
|
@ -226,7 +207,11 @@ export class XCFI {
|
|||
throw new Error(`Cannot resolve XPointer path: ${elementPath}`);
|
||||
}
|
||||
|
||||
return { element, textOffset };
|
||||
const children = XCFI.crengineTextChildren(element);
|
||||
const node = children.find((t) => t.data.trim().length > 0) ?? children[0];
|
||||
// No direct text at all: keep the cumulative reading as a best effort.
|
||||
if (!node) return { element, textOffset: offsetInNode };
|
||||
return { element, point: { node, offset: XCFI.nodeOffset(node, offsetInNode) } };
|
||||
}
|
||||
|
||||
// Format: /tag[idx].N — offset directly on the last element segment, with
|
||||
|
|
@ -254,40 +239,146 @@ export class XCFI {
|
|||
return { element };
|
||||
}
|
||||
|
||||
private static readonly WHITESPACE = /[ \t\n\r\f]/;
|
||||
/** Tags whose built-in crengine style is `white-space: pre` (fb2def.h). */
|
||||
private static readonly PRE_TAGS = new Set([
|
||||
'pre',
|
||||
'code',
|
||||
'listing',
|
||||
'plaintext',
|
||||
'xmp',
|
||||
'textarea',
|
||||
]);
|
||||
/** Block containers, where crengine drops leading whitespace-only text. */
|
||||
private static readonly BLOCK_TAGS = new Set([
|
||||
'body',
|
||||
'div',
|
||||
'p',
|
||||
'h1',
|
||||
'h2',
|
||||
'h3',
|
||||
'h4',
|
||||
'h5',
|
||||
'h6',
|
||||
'ul',
|
||||
'ol',
|
||||
'li',
|
||||
'dl',
|
||||
'dt',
|
||||
'dd',
|
||||
'blockquote',
|
||||
'pre',
|
||||
'section',
|
||||
'article',
|
||||
'aside',
|
||||
'header',
|
||||
'footer',
|
||||
'nav',
|
||||
'main',
|
||||
'figure',
|
||||
'figcaption',
|
||||
'table',
|
||||
'thead',
|
||||
'tbody',
|
||||
'tfoot',
|
||||
'tr',
|
||||
'td',
|
||||
'th',
|
||||
'caption',
|
||||
'address',
|
||||
'details',
|
||||
'summary',
|
||||
'form',
|
||||
'fieldset',
|
||||
'center',
|
||||
]);
|
||||
|
||||
/** Inside `pre`/`code` and friends crengine keeps whitespace verbatim. */
|
||||
private static preservesWhitespace(node: Node): boolean {
|
||||
for (let el = node.parentElement; el; el = el.parentElement) {
|
||||
if (XCFI.PRE_TAGS.has(el.tagName.toLowerCase())) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve text()[K].N to a cumulative character offset within the element.
|
||||
* K is the 1-based index of direct text node children of the element
|
||||
* (counting only Text nodes that are immediate children, skipping element children).
|
||||
* N is the character offset within that specific text node.
|
||||
* Direct text children as crengine keeps them (ldomDocumentWriter::OnText):
|
||||
* a whitespace-only text node that opens a block is dropped, whitespace
|
||||
* between or after inline children survives as a single space. Pinned
|
||||
* against the real engine with apps/readest.koplugin/scripts/xpointer-oracle.lua.
|
||||
*/
|
||||
private resolveIndexedTextNode(
|
||||
private static crengineTextChildren(element: Element): Text[] {
|
||||
const dropsLeadingBlank =
|
||||
XCFI.BLOCK_TAGS.has(element.tagName.toLowerCase()) &&
|
||||
!XCFI.preservesWhitespace(element.firstChild ?? element);
|
||||
return Array.from(element.childNodes).filter((node, i): node is Text => {
|
||||
if (node.nodeType !== Node.TEXT_NODE) return false;
|
||||
const text = (node as Text).data;
|
||||
if (text.length === 0) return false;
|
||||
return !(dropsLeadingBlank && i === 0 && text.trim() === '');
|
||||
});
|
||||
}
|
||||
|
||||
/** crengine offset -> raw DOM offset inside `node` (see toRawOffset). */
|
||||
private static nodeOffset(node: Text, crengineOffset: number): number {
|
||||
if (XCFI.preservesWhitespace(node)) return Math.min(crengineOffset, node.data.length);
|
||||
return XCFI.toRawOffset(node.data, crengineOffset);
|
||||
}
|
||||
|
||||
/** raw DOM offset inside `node` -> crengine offset (see toCollapsedOffset). */
|
||||
private static crengineOffset(node: Text, rawOffset: number): number {
|
||||
if (XCFI.preservesWhitespace(node)) return rawOffset;
|
||||
return XCFI.toCollapsedOffset(node.data, rawOffset);
|
||||
}
|
||||
|
||||
/**
|
||||
* crengine collapses each whitespace run inside a text node to one space and
|
||||
* counts offsets in that collapsed text; the DOM counts the raw source.
|
||||
*/
|
||||
private static toCollapsedOffset(text: string, rawOffset: number): number {
|
||||
let collapsed = 0;
|
||||
for (let i = 0; i < rawOffset && i < text.length; i++) {
|
||||
if (!XCFI.isRunContinuation(text, i)) collapsed++;
|
||||
}
|
||||
return collapsed + Math.max(0, rawOffset - text.length);
|
||||
}
|
||||
|
||||
private static toRawOffset(text: string, collapsedOffset: number): number {
|
||||
let collapsed = 0;
|
||||
for (let i = 0; i < text.length; i++) {
|
||||
if (XCFI.isRunContinuation(text, i)) continue;
|
||||
if (collapsed === collapsedOffset) return i;
|
||||
collapsed++;
|
||||
}
|
||||
return text.length;
|
||||
}
|
||||
|
||||
private static isRunContinuation(text: string, i: number): boolean {
|
||||
return i > 0 && XCFI.WHITESPACE.test(text[i]!) && XCFI.WHITESPACE.test(text[i - 1]!);
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve text()[K].N to a DOM position. K is the 1-based index among the
|
||||
* direct text children crengine keeps and N is the offset within that text
|
||||
* node's whitespace-collapsed text.
|
||||
*/
|
||||
private textChildPoint(
|
||||
element: Element,
|
||||
textNodeIndex: number,
|
||||
offsetInNode: number,
|
||||
): number {
|
||||
let directTextCount = 0;
|
||||
let cumulativeOffset = 0;
|
||||
|
||||
for (const child of Array.from(element.childNodes)) {
|
||||
if (child.nodeType === Node.TEXT_NODE) {
|
||||
directTextCount++;
|
||||
if (directTextCount === textNodeIndex) {
|
||||
return cumulativeOffset + offsetInNode;
|
||||
}
|
||||
cumulativeOffset += (child.textContent || '').length;
|
||||
} else if (child.nodeType === Node.ELEMENT_NODE) {
|
||||
// Count text length inside child elements for cumulative offset
|
||||
cumulativeOffset += (child.textContent || '').length;
|
||||
}
|
||||
): { node: Text; offset: number } {
|
||||
const children = XCFI.crengineTextChildren(element);
|
||||
const node = children[textNodeIndex - 1];
|
||||
if (!node) {
|
||||
throw new Error(
|
||||
`Text node index ${textNodeIndex} out of bounds (found ${children.length} direct text nodes)`,
|
||||
);
|
||||
}
|
||||
|
||||
throw new Error(
|
||||
`Text node index ${textNodeIndex} out of bounds (found ${directTextCount} direct text nodes)`,
|
||||
);
|
||||
return { node, offset: XCFI.nodeOffset(node, offsetInNode) };
|
||||
}
|
||||
|
||||
private resolveXPointerPath(path: string): Element | null {
|
||||
const pathMatch = path.match(/^\/body\/DocFragment\[\d+\]\/body(.*)$/);
|
||||
const pathMatch = path.match(/^\/body\/DocFragment(?:\[\d+\])?\/body(.*)$/);
|
||||
if (!pathMatch) {
|
||||
throw new Error(`Invalid XPointer format: ${path}`);
|
||||
}
|
||||
|
|
@ -400,9 +491,7 @@ export class XCFI {
|
|||
*/
|
||||
private rangePointToXPointer(container: Node, offset: number): string {
|
||||
if (container.nodeType === Node.TEXT_NODE) {
|
||||
// For text nodes, find the containing element
|
||||
const element = container.parentElement || this.document.documentElement;
|
||||
return this.handleTextOffsetInElement(element, container as Text, offset);
|
||||
return this.textNodeXPointer(container as Text, offset);
|
||||
} else if (container.nodeType === Node.ELEMENT_NODE) {
|
||||
const element = container as Element;
|
||||
if (offset === 0) {
|
||||
|
|
@ -421,11 +510,7 @@ export class XCFI {
|
|||
if (targetChild?.nodeType === Node.ELEMENT_NODE) {
|
||||
return this.buildXPointerPath(targetChild as Element);
|
||||
} else if (targetChild?.nodeType === Node.TEXT_NODE) {
|
||||
return this.handleTextOffsetInElement(
|
||||
element,
|
||||
targetChild as Text,
|
||||
(targetChild as Text).textContent?.length || 0,
|
||||
);
|
||||
return this.textNodeXPointer(targetChild as Text, (targetChild as Text).data.length);
|
||||
} else {
|
||||
return this.buildXPointerPath(element);
|
||||
}
|
||||
|
|
@ -565,49 +650,29 @@ export class XCFI {
|
|||
return this.buildXPointerPath(element);
|
||||
}
|
||||
|
||||
// Use the text node's direct parent for both path and indexing.
|
||||
// This produces correct XPointers even when inline elements (like <a>)
|
||||
// split text into multiple direct text node children.
|
||||
const textParent = targetTextNode.parentElement || element;
|
||||
const basePath = this.buildXPointerPath(textParent);
|
||||
|
||||
// Count direct text node children and find which one the target is
|
||||
let directTextCount = 0;
|
||||
let directTextIndex = 0;
|
||||
for (const child of Array.from(textParent.childNodes)) {
|
||||
if (child.nodeType === Node.TEXT_NODE && (child.textContent || '').length > 0) {
|
||||
directTextCount++;
|
||||
if (child === targetTextNode) {
|
||||
directTextIndex = directTextCount;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Omit [1] when there is only one direct text node (matches KOReader format)
|
||||
if (directTextCount <= 1) {
|
||||
return `${basePath}/text().${offsetInNode}`;
|
||||
}
|
||||
return `${basePath}/text()[${directTextIndex || 1}].${offsetInNode}`;
|
||||
return this.textNodeXPointer(targetTextNode, offsetInNode);
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle text offset for a specific text node within an element
|
||||
* XPointer of a position inside a specific text node, in KOReader's form:
|
||||
* the node's direct parent gives the path (correct even when inline
|
||||
* elements split the text), the index counts the text children crengine
|
||||
* keeps and the offset counts its whitespace-collapsed text.
|
||||
*/
|
||||
private handleTextOffsetInElement(element: Element, textNode: Text, offset: number): string {
|
||||
// Find all text nodes in the element to calculate cumulative offset
|
||||
const textNodes: Text[] = [];
|
||||
this.collectTextNodes(element, textNodes);
|
||||
private textNodeXPointer(textNode: Text, offsetInNode: number): string {
|
||||
const textParent = textNode.parentElement || this.document.documentElement;
|
||||
const basePath = this.buildXPointerPath(textParent);
|
||||
const siblings = XCFI.crengineTextChildren(textParent);
|
||||
const index = siblings.indexOf(textNode);
|
||||
// Dropped whitespace belongs at the block start, before any inline content.
|
||||
if (index < 0) return basePath;
|
||||
const collapsed = XCFI.crengineOffset(textNode, offsetInNode);
|
||||
|
||||
let cumulativeOffset = 0;
|
||||
for (const node of textNodes) {
|
||||
if (node === textNode) {
|
||||
cumulativeOffset += offset;
|
||||
break;
|
||||
}
|
||||
cumulativeOffset += (node.textContent || '').length;
|
||||
// Omit [1] when there is only one direct text node (matches KOReader format)
|
||||
if (siblings.length <= 1) {
|
||||
return `${basePath}/text().${collapsed}`;
|
||||
}
|
||||
|
||||
return this.handleTextOffset(element, cumulativeOffset);
|
||||
return `${basePath}/text()[${index + 1}].${collapsed}`;
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
74
apps/readest.koplugin/scripts/xpointer-oracle.lua
Normal file
74
apps/readest.koplugin/scripts/xpointer-oracle.lua
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
-- XPointer oracle: dump crengine's own XPointer for every visible word of an
|
||||
-- EPUB, so Readest's CFI <-> XPointer conversion (src/utils/xcfi.ts) can be
|
||||
-- validated against the real KOReader engine instead of assumptions about it.
|
||||
--
|
||||
-- Runs headlessly inside a KOReader emulator build (no window, no sync server):
|
||||
--
|
||||
-- cd <koreader-emulator>/koreader
|
||||
-- KO_HOME=/tmp/ko-oracle ./luajit \
|
||||
-- /path/to/readest/apps/readest.koplugin/scripts/xpointer-oracle.lua \
|
||||
-- /path/to/book.epub /path/to/book.crengine.json [every]
|
||||
--
|
||||
-- `every` keeps one word in N (default 1) to keep committed fixtures small.
|
||||
-- KO_HOME keeps the harness settings and cache out of the emulator directory.
|
||||
--
|
||||
-- Consume the JSON with apps/readest-app/src/__tests__/utils/xcfi.crengine-oracle.test.ts,
|
||||
-- either as a committed fixture under src/__tests__/fixtures/crengine/ or ad hoc:
|
||||
-- XPOINTER_ORACLE=/path/to/book.crengine.json pnpm test src/__tests__/utils/xcfi.crengine-oracle.test.ts
|
||||
require("setupkoenv")
|
||||
package.path = "spec/front/unit/?.lua;" .. package.path
|
||||
require("commonrequire") -- the spec harness: dummy framebuffer, no SDL window
|
||||
local JSON = require("json")
|
||||
local InitArray = require("json.util").InitArray -- keep empty lists as [] not {}
|
||||
local DocumentRegistry = require("document/documentregistry")
|
||||
|
||||
local epub, outpath, every = arg[1], arg[2], tonumber(arg[3] or "1")
|
||||
assert(epub and outpath, "usage: xpointer-oracle.lua <book.epub> <out.json> [every]")
|
||||
|
||||
local doc = assert(DocumentRegistry:openDocument(epub), "cannot open " .. epub)
|
||||
-- A book KOReader has never seen before gets the latest DOM version
|
||||
-- (ReaderRolling:onReadSettings); match that so XPointers are normalized.
|
||||
doc:requestDomVersion(doc:getLatestDomVersion())
|
||||
doc:render()
|
||||
|
||||
local result = {
|
||||
epub = epub:match("([^/]+)$"),
|
||||
dom_version = doc:getLatestDomVersion(),
|
||||
every = every,
|
||||
fragments = InitArray({}),
|
||||
}
|
||||
|
||||
-- DocFragment[N] is spine item N-1; crengine creates one per <itemref>. With a
|
||||
-- single spine item crengine omits the index and writes /body/DocFragment/...
|
||||
local single = not doc:isXPointerInDocument("/body/DocFragment[2]")
|
||||
local n = 1
|
||||
while doc:isXPointerInDocument("/body/DocFragment[" .. n .. "]") do
|
||||
local prefix = single and "/body/DocFragment/" or ("/body/DocFragment[" .. n .. "]/")
|
||||
local fragment = { index = n - 1, docfragment = n, words = InitArray({}) }
|
||||
local xp = doc:getNextVisibleWordStart(prefix .. "body")
|
||||
local seen = 0
|
||||
while xp and xp:sub(1, #prefix) == prefix do
|
||||
local xp_end = doc:getNextVisibleWordEnd(xp)
|
||||
if xp_end and xp_end:sub(1, #prefix) == prefix then
|
||||
if seen % every == 0 then
|
||||
table.insert(fragment.words, {
|
||||
xp = xp,
|
||||
xp_end = xp_end,
|
||||
text = doc:getTextFromXPointers(xp, xp_end),
|
||||
})
|
||||
end
|
||||
seen = seen + 1
|
||||
end
|
||||
xp = doc:getNextVisibleWordStart(xp)
|
||||
end
|
||||
fragment.word_count = seen
|
||||
table.insert(result.fragments, fragment)
|
||||
n = n + 1
|
||||
end
|
||||
doc:close()
|
||||
|
||||
local out = assert(io.open(outpath, "w"))
|
||||
-- luajson escapes "/" as "\/", which is valid JSON but hard to read in fixtures
|
||||
out:write((JSON.encode(result):gsub("\\/", "/")))
|
||||
out:close()
|
||||
print(string.format("wrote %s: %d fragments", outpath, #result.fragments))
|
||||
|
|
@ -1 +1 @@
|
|||
Subproject commit 8ffd2cffc36c4be84490c5a1415d11668f37c88f
|
||||
Subproject commit cb3896fd52c243f1dc6bfaf09dbf7d96318a4183
|
||||
Loading…
Add table
Add a link
Reference in a new issue