mirror of
https://github.com/readest/readest.git
synced 2026-10-19 10:13:29 +00:00
35450e965a
* feat(tts): play a book's own recorded narration (EPUB 3 Media Overlays)
Read Aloud always synthesized speech, so a professionally narrated read-along
EPUB was read by a robot voice over a book that came with a human one. Play the
recording instead, keeping the entire existing Read Aloud experience: moving
highlight, page-following, sentence/paragraph skip, scrubber and seek, speed,
sleep timer, lock-screen/CarPlay, and background sessions.
This is the Kindle Immersion Reading / Audible Read & Listen experience built on
the open EPUB standard rather than a store-side pairing of two purchases: no
account, no matching entitlements, any narrated EPUB you own or generate.
The stack already had the two seams this needs. `TTSClient` abstracts where audio
comes from; foliate's `TTS` abstracts how text is cut into marks. Recorded
narration is exactly "a different audio source with a different segmentation", so
`src/services/tts/mediaOverlay/` swaps both and leaves the other ~25 files of the
subsystem untouched:
- `parseSmil.ts` parses SMIL clock values and walks <body>/<seq>/<par>.
- `MediaOverlaySection.ts` resolves each par's text fragment to a DOM Range,
groups pars into blocks by nearest block ancestor, and emits the SSML.
- `MediaOverlayTTS.ts` stands in for foliate's TTS with the same navigation
surface, driven by the par list.
- `MediaOverlayClient.ts` plays clips off one HTMLMediaElement, emitting a
boundary as each par becomes audible.
Marks are section-global par ordinals, so marks and audio clips are 1:1 by
construction and there is no text-to-audio matching anywhere in the feature. A
block plays as one continuous span, so a narrated sentence has no seam mid-way.
Selection reuses the Voice picker: the narrator leads the list for books that
carry overlays, and is preselected. `ttsUseNarration` on TTSConfig records the
per-book opt-out, separate from `ttsVoice` because that inherits the global
default. Offline Audio is hidden during narration, since the audio ships inside
the book. Narrated books get a headphones badge in the library, from a
`Book.hasNarration` flag cached at import time.
Highlighting follows the recording exactly rather than interpolating within a
clip, so it cannot drift. Sections the publisher left unnarrated are skipped.
Alignment stays out of scope: a bare MP3 carries no timings and generating them
would mean shipping a speech-recognition model. docs/read-along-narration.md
points at Storyteller for turning an ebook plus its audiobook into a Media
Overlays EPUB, and is honest about the evidence on reading-while-listening: the
comprehension benefit is trivial overall (g=0.18) and only appears when reading
is externally paced, which is why this is a preference and not a prescription.
Two supporting changes:
- `TimelineSentence.duration` carries the exact clip length and outranks the
measured/estimated tiers in SectionTimeline, so a narrated chapter's scrubber
reports the recording's real duration. Deliberately not routed through the
text-keyed cache in ttsDuration.ts, where two identical sentences collide.
- ensureTimeline/supportsPlaybackInfo/getPlaybackInfo now gate on
`TTSCapabilities.mediaClock` instead of comparing against the Edge client.
Verified end to end in a browser against the W3C Moby-Dick MO sample and a
330 MB Storyteller-generated novel: narration selected automatically, unnarrated
front matter skipped, real audio decoded and played at the user's rate with pitch
preserved, boundaries firing per par across block boundaries, the narrated
sentence visibly highlighted, and the scrubber showing the recording's exact
elapsed/remaining time. That pass found four defects no jsdom test could reach:
element-anchored ranges producing a degenerate CFI so the highlight never
painted, typeless container blobs failing to decode, five concurrent loads of the
same audio file revoking each other's URLs, and a WrongDocumentError that killed
the session whenever the first sections were unnarrated. All four are fixed and
covered by regression tests.
It also surfaced a pre-existing hang: WebSpeechClient.init() waited forever for a
'voiceschanged' event on a platform reporting zero speech voices, wedging
TTSController.init() and so blocking every engine, including narration, which
needs no system voice. It now gives up after a timeout.
* fix(tts): let recorded narration play on instead of re-seeking each paragraph
Media Overlay clips are contiguous and in document order, so sequential
narration needs no seeking at all: the element can simply keep rolling while
boundaries are reported as the clock passes each clip.
It was instead seeking to clipBegin at the start of every block, because the
controller drives one speak() per paragraph. Detecting a clip's end by polling
the media clock always lands a few milliseconds PAST clipEnd, so those
milliseconds of the next clip had already been heard; seeking back replayed
them, audible as a stutter on the paragraph's first word ("Me? ... Me?").
Now the playhead only moves for a genuine discontinuity (session start, a
sentence skip, a scrub, a new audio file), decided from the element's own
position rather than from bookkeeping. Measured over a minute of The Player of
Games: one seek, at startup, and none across four paragraph boundaries.
Also stop padding paragraph transitions for a continuous timeline. The new
TTSCapabilities.continuousTimeline says consecutive blocks are one recording,
so the controller's inter-paragraph delay would add silence the narrator never
left and push the highlight behind the voice.
* fix(tts): stop putting a gap between paragraphs of recorded narration
Every paragraph boundary was audible as a hiccup, most obviously on the short
"Me?" that opens a paragraph early in The Player of Games. The recording was
being silenced and restarted at each boundary: the client paused at its block's
last clip, and the controller's pre-speak stop() paused again, ~50ms before the
next block called play(). On Android that also costs a buffer flush.
Silencing the engine between utterances is right for synthesis, where each
utterance is separately rendered audio, and wrong for one continuous recording,
where the next utterance is the same audio playing on. So stop() now takes a
`handover` flag for the stop the controller performs between two utterances of
the same session: a continuous-timeline engine stays rolling through it, and
engines that synthesize per utterance ignore it and stop as before.
The narration client therefore leaves the element playing when a block ends and
lets the next block join it in progress. A watchdog covers the case where no
next block comes (a one-off selection read, or a session torn down without
stopping), so narration can never be left running unattended.
Measured over 80s of The Player of Games, sixteen sentences across five
paragraph boundaries: no seeks, no pauses, and no stall in the media clock.
* fix(tts): read a selection aloud from the recording when a book is narrated
Selecting text and pressing Read Aloud played for about a second and then
stopped the whole session. The selection path builds SSML from the raw selected
text, which is right for a synthesizer but meaningless to the narration client:
its marks are SMIL par ordinals, so raw text matched no clip, the utterance
ended immediately, and the one-shot callback stopped the session.
A recording has no audio for arbitrary text, only the clips the publisher timed,
so reading a selection from one means starting the narration where that passage
is narrated, via the same tts.from(range) used for any other jump. It is an
ordinary session from that point, not a one-shot utterance, so the one-time
callback no longer applies.
* docs(tts): update the narration notes for the playback fixes
The playback fixes changed how the client drives the media element, so the
"how it works" notes described behaviour that no longer exists: it no longer
seeks at the start of every block, and the capability object it reports gained
continuousTimeline. Record both, plus the handover seam on stop(), the two
defects that surfaced only by listening, and the selection-read fix.
* docs(tts): trim the narration notes to what the code needs
Drop two sections that had grown past the point of earning their place in a
feature doc. The reading-while-listening literature review was an argument about
whether the feature is worth having rather than documentation of how it works,
and the defect postmortem duplicated detail the commit messages already carry.
Keeps the paragraph describing the Media Overlays format itself, which is
reference material the rest of the document depends on.
* fix(tts): follow the reading position across a page break inside a sentence
A sentence laid out across a page break left the reader behind: its mark fires
once, on the page it starts on, and a phrase-timed recording reports no words in
between, so the view stayed put while the voice read the tail rendered on the
next page. Measured at 6.5 seconds on one sentence of The Player of Games.
Synthesized engines hide this by following each word.
What a recording does give is how far through the current phrase its audio is,
now exposed as TTSClient.getChunkProgress(). Where the page stops showing the
sentence cannot be assumed — the same sentence breaks at a different word on
another screen, at a larger font size, or in a narrower column — so it is
measured from the live layout: pageBreakFraction bisects the sentence, probing
characters through getTextSubRange, and returns the break as a fraction of the
sentence's text, which is the unit that tracks how long the narrator spends on
it. Bisection is not tidiness — every probe forces a reflow.
The page turns once audio progress passes that fraction, and the fraction is
re-measured after each turn, so a sentence spanning three pages advances one page
at a time. No word position is invented, so the highlight still follows the
recording exactly. Engines without a chunk clock report no progress and keep
their existing behaviour.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(tts): keep the back-to-position prompt down when the view follows the voice
Following the voice onto the next page moves the view past the *start* of the
sentence being read, and that start is what ttsLocation records. Judged on it
alone the reader looks like they navigated away, so the prompt appeared on every
page turn — while they were in fact looking at the words being spoken. Engines
that report words dodge this because their highlight cfi moves with the voice; a
phrase-timed recording has no such cfi.
So ask the layout instead: isSoundingSentenceOnScreen() reports whether any part
of the sentence now sounding is still on the visible page, and the prompt stays
down while it is. Paging away from the sentence entirely still raises it.
Also rename the prompt from "Back to TTS Location" to "Back to Read Aloud",
after the name the feature actually carries in the UI, and drop the "TTS" jargon
from all 33 locales — several of which had been carrying the acronym untranslated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Sacha Lastow <sacha.lastow@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
16 KiB
16 KiB
Readest
Readest is an open-source ebook reader designed for immersive and deep reading experiences. Built as a modern rewrite of Foliate, it leverages Next.js 16 and Tauri v2 to deliver a smooth, cross-platform experience across macOS, Windows, Linux, Android, iOS, and the Web.
Features • Planned Features • Screenshots • Downloads • Documentation • Getting Started • Troubleshooting • Support • License
Features
✅ Implemented
| Feature | Description | Status |
|---|---|---|
| Multi-Format Support | Support EPUB, PDF, MOBI, KF8 (AZW3), FB2, CBZ, TXT, MD (Markdown) | ✅ |
| Scroll/Page View Modes | Switch between scrolling or paginated reading modes. | ✅ |
| Full-Text Search | Search inside a book or across the current library shelf to find relevant sections. | ✅ |
| Annotations and Highlighting | Add highlights, bookmarks, and notes to enhance your reading experience and use instant mode for quicker interactions. | ✅ |
| Dictionary/Wikipedia Lookup | Instantly look up words and terms when reading. | ✅ |
| Parallel Read | Read two books or documents simultaneously in a split-screen view. | ✅ |
| Customize Font and Layout | Adjust font, layout, theme mode, and theme colors for a personalized experience. | ✅ |
| Code Syntax Highlighting | Read software manuals with rich coloring of code examples. | ✅ |
| File Association and Open With | Quickly open files in Readest in your file browser with one-click. | ✅ |
| Library Management | Organize, sort, and manage your entire ebook library. | ✅ |
| OPDS/Calibre Integration | Integrate OPDS/Calibre to access online libraries and catalogs. | ✅ |
| Translate with DeepL and Yandex | From a single sentence to the entire book—translate instantly. | ✅ |
| Text-to-Speech (TTS) Support | Enjoy smooth, multilingual narration—even within a single book. | ✅ |
| Read-Along Narration | Play a book's own recorded narration with the text highlighted in step — Kindle Immersion Reading / Audible Read & Listen, on the open EPUB standard. Reads EPUB 3 Media Overlays; pair an ebook with its audiobook using Storyteller. | ✅ |
| Sync across Platforms | Synchronize book files, reading progress, notes, and bookmarks across all supported platforms. | ✅ |
| Sync with Koreader | Synchronize reading progress, notes, and bookmarks with Koreader devices. | ✅ |
| Accessibility | Provides full keyboard navigation and support for screen readers such as VoiceOver, TalkBack, NVDA, and Orca. | ✅ |
| Visual & Focus Aids | Reading ruler, paragraph-by-paragraph reading mode, and speed reading features. | ✅ |
Planned Features
🛠 Building
🔄 Planned
| Feature | Description | Priority |
|---|---|---|
| AI-Powered Summarization | Generate summaries of books or chapters using AI for quick insights. | 🛠 |
| Advanced Reading Stats | Track reading time, pages read, and more for detailed insights. | 🛠 |
| Audiobook Support | Extend functionality to play and manage audiobooks. | 🔄 |
| Handwriting Annotations | Add support for handwriting annotations using a pen on compatible devices. | 🔄 |
Stay tuned for continuous improvements and updates! Contributions and suggestions are always welcome—let's build the ultimate reading experience together. 😊
Screenshots
Downloads
Mobile Apps
Platform-Specific Downloads
- macOS / iOS / iPadOS : Search and install Readest on the App Store, also available on TestFlight for beta test (send your Apple ID to readestapp@gmail.com to request access).
- Windows / Linux / Android: Visit and download Readest at https://readest.com or the Releases on GitHub.
- Linux users can also install Readest on Flathub.
- Web: Visit and use Readest for Web at https://web.readest.com.
Documentation
Guides, tutorials, and FAQs for installing and using Readest live in the official documentation:
Building from Source
To build Readest from the latest commit, see Getting Started.
Troubleshooting
1. Readest Won’t Launch on Windows (Missing Edge WebView2 Runtime)
Symptom
- When you double-click readest.exe, nothing happens. No window appears, and Task Manager does not show the process.
- This can affect both the standard installer and the portable version.
Cause
- Microsoft Edge WebView2 Runtime is either missing, outdated, or improperly installed on your system. Readest depends on WebView2 to render the interface on Windows.
How to Fix
- Check if WebView2 is installed
- Open “Add or Remove Programs” (a.k.a. Apps & features) on Windows. Look for “Microsoft Edge WebView2 Runtime.”
- Install or Update WebView2
- Download the WebView2 Runtime directly from Microsoft: link.
- If you prefer an offline installer, download the offline package and run it as an Administrator.
- Re-run Readest
- After installing/updating WebView2, launch readest.exe again.
- If you still encounter problems, reboot your PC and try again.
Additional Tips
- If reinstalling once doesn’t work, uninstall Edge WebView2 completely, then reinstall it with Administrator privileges.
- Verify your Windows installation has the latest updates from Microsoft.
Still Stuck?
- See Issue readest/readest#358 for further details, or head over to our Discord server and open a support discussion with detailed logs of your environment and the steps you’ve taken.
2. AppImage Launches but Only Shows a Taskbar Icon
On some Arch Linux systems—especially those using Wayland—the Readest AppImage may briefly show an icon in the taskbar and then exit without opening a window.
You might see logs such as:
Could not create default EGL display: EGL_BAD_PARAMETER. Aborting...
This behavior is usually caused by compatibility issues between the bundled AppImage libraries and the system’s EGL / Wayland environment.
Workaround 1: Launch with LD_PRELOAD (recommended)
You can preload the system Wayland client library before launching the AppImage:
LD_PRELOAD=/usr/lib/libwayland-client.so /path/to/Readest.AppImage
This workaround has been confirmed to resolve the issue on affected systems.
Workaround 2: Use the Flatpak Version
If you prefer a more reliable out-of-the-box experience on Arch Linux, consider using the Flatpak build on Flathub instead. The Flatpak runtime helps avoid system library mismatches and tends to behave more consistently across different Wayland and X11 setups.
Contributors
Readest is open-source, and contributions are welcome! Feel free to open issues, suggest features, or submit pull requests. Please review our contributing guidelines before you start. We also welcome you to join our Discord community for either support or contributing guidance.
Support
If Readest has been useful to you, consider supporting its development at donate.readest.com, where you'll find all available donation methods, including GitHub Sponsors, card payments, and crypto. Your contribution helps us fix bugs faster, improve performance, and keep building great features.
License
Readest is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. See the LICENSE file for details.
The following libraries and frameworks are used in this software:
- foliate-js, which is MIT licensed.
- zip.js, which is licensed under the BSD-3-Clause license.
- fflate, which is MIT licensed.
- PDF.js, which is licensed under Apache License 2.0.
- daisyUI, which is MIT licensed.
- marked, which is MIT licensed.
- next.js, which is MIT licensed.
- react-icons, which has various open-source licenses.
- react, which is MIT licensed.
- tauri, which is MIT licensed.
The following fonts are utilized in this software, either bundled within the application or provided through web fonts:
Bitter, Fira Code, Inter, Literata, Merriweather, Noto Sans, Roboto, LXGW WenKai, MiSans, Source Han, WenQuanYi Micro Hei
We would also like to thank the Web Chinese Fonts Plan for offering open-source tools that enable the use of Chinese fonts on the web.
Happy reading with Readest!