perf(android): cut the arm64 apk from 90MB to 57MB (#6368)
Some checks are pending
CodeQL Advanced / Analyze (actions) (push) Waiting to run
CodeQL Advanced / Analyze (javascript-typescript) (push) Waiting to run
CodeQL Advanced / Analyze (rust) (push) Waiting to run
Publish Docker image / build (linux/amd64, ubuntu-latest) (push) Waiting to run
Publish Docker image / build (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Publish Docker image / merge (push) Blocked by required conditions
Build with Nix / tests (push) Waiting to run
PR checks / rust_lint (push) Waiting to run
PR checks / koplugin_rust_lint (push) Waiting to run
PR checks / nix_flake_check (push) Waiting to run
PR checks / build_web_app (push) Waiting to run
PR checks / test_web_app (1) (push) Waiting to run
PR checks / test_web_app (2) (push) Waiting to run
PR checks / test_extensions (push) Waiting to run
PR checks / build_tauri_app (push) Waiting to run
Scorecard supply-chain security / Scorecard analysis (push) Waiting to run
Deploy to vercel on merge / build_and_deploy (push) Waiting to run

* perf(android): cut the arm64 apk from 90MB to 57MB

The Rust library is 95% of the APK, and most of what it carried was not
code. Measured on an aarch64 release build: 89.7MB -> 57.5MB.

- Build `release` with fat LTO and one codegen unit. Cross-crate inlining
  cuts .text by 5.2MB and the unwind/exception tables by 2.1MB, at the
  cost of a longer final link.
- Strip the packaged library. AGP's own strip step only drops the
  .debug_* sections, so 22MB of symbol tables (10MB after LTO) reached
  every download. The strip runs on AGP's output, so `target/` keeps the
  unstripped copy, which the release and nightly workflows now upload to
  Sentry as debug files; Rust frames stay symbolicated. Android builds
  also link with a build id, which is how Sentry matches them, and the
  `debug-images` feature attaches the loaded modules to each event.
- Stop shipping jieba's WASM twice (2.7MB). The wasm-bindgen glue
  defaults to the file the bundler already emits; pointing `init()` at a
  public/vendor copy embedded both.
- Keep the browser-only turso WASM out of native builds (3.3MB). The
  alias that excluded it still named `@tursodatabase/database-wasm`,
  which the web database service stopped importing in #4086.

simplecc is still embedded twice (0.4MB): its glue is imported through
the `@simplecc/*` path alias out of public/vendor, so deduping it means
moving that directory.

Verified: the built APK's library carries no symbol tables and a build
id; `target/` keeps its; the bundle requests exactly the one embedded
jieba asset key and no /vendor/jieba path; the turso WASM is absent from
`out/`. The APK installs and runs on a Xiaomi 2211133C, library and a
Chinese book render, and RSVP runs on it. Whether jieba's WASM is
actually fetched at runtime is NOT verified: release webviews are not
inspectable and the fallback to Intl.Segmenter is silent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* perf(android): drop fat LTO, codegen-units=1 is the whole win

The previous commit credited the code-size saving to fat LTO. That was
wrong. Measured on aarch64-linux-android with the flags rustc actually
received (`cargo build -v`):

  codegen-units=16, lto=false   .text 29.66MB  .eh_frame 3.69  .except 2.51
  codegen-units=1,  lto=off     .text 24.43MB  .eh_frame 2.28  .except 1.84
  codegen-units=1,  lto=thin    .text 24.43MB  .eh_frame 2.28  .except 1.84
  codegen-units=1,  lto=fat     .text 24.44MB  .eh_frame 2.28  .except 1.84

One codegen unit accounts for all of it; LTO changes nothing once it is
set, so fat LTO was buying only link time and peak memory on every
release target. The rebuilt APK confirms it: 57.49MB without LTO against
57.50MB with it.

What `codegen-units = 1` costs, from a clean aarch64 build on an
M-series: 5m54s against 4m15s, and 3.3GB peak RSS against 2.9GB. That
answers the review's question about the 75-minute Android and 45-minute
desktop step limits in nightly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* perf: stop shipping vendored modules the bundler already embeds

Tauri embeds every file under `out/`, and `public/` is copied there whole. So
a module imported through a path alias that points into `public/` ships twice:
once bundled into a chunk, once as a published file nobody fetches. jieba cost
2.7MB that way; `@pdfjs/*` and `@simplecc/*` had the same shape.

Split the vendored assets by how they are loaded. What the reader fetches by
URL at runtime stays in `public/vendor/pdfjs` — the worker, the WASM decoders,
cmaps, standard fonts, the layer CSS, all built as runtime URLs in
foliate-js/pdf.js, which is why the bundler never emitted copies of them.
What the bundler imports moves to `vendor/` (outside `public/`):
`pdf.min.mjs` + its types, and the simplecc glue and WASM.

Measured on the embedded frontend, brotli as tauri compresses it: 23.7MB ->
16.96MB across this and the earlier jieba/turso fixes. That comes out of every
Tauri target, not just the APK.

The new test states the invariant that was broken rather than the three
symptoms: no tsconfig path alias may resolve into `public/`. It failed on
`@pdfjs/*` and `@simplecc/*` before this change.

Desktop stripping was measured and deliberately NOT done. On macOS it saves
4.76MB in the binary but only 1.44MB of DMG (36.59 -> 35.15MB), because a DMG
compresses symbol strings while an APK stores the .so uncompressed (verified:
`Stored`, 0%) — which is why Android got the full 22MB. Buying 1.4MB would
mean either a profile-level `strip`, which deletes the unstripped artifact
Sentry symbolicates Rust panics from, or pulling the release job apart from
tauri-action, which builds, bundles, signs and notarizes in one step. deb, rpm
and AppImage are compressed too, and MSVC keeps symbols in the PDB.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: init simplecc from the emitted WASM, not a URL that no longer exists

Moving the simplecc glue out of `public/` left `initSimpleCC` still asking for
`/vendor/simplecc/simplecc_wasm_bg.wasm`, a URL nothing publishes any more, so
Chinese variant conversion would fail to initialize at runtime. Caught in
review of #6368.

`init()` with no path uses the glue's default
`new URL('simplecc_wasm_bg.wasm', import.meta.url)` — the copy the bundler
emits and Tauri embeds — the same shape as the jieba fix.

The existing WASM-init test covered jieba only, which is why this slipped
through; it now covers both. Verified against a fresh build: the chunk
references `static/media/simplecc_wasm_bg.<hash>.wasm` and no runtime
`/vendor/simplecc` reference remains (the two textual hits left are the
bundled package.json script strings and a source-map path, and maps are
stripped before packaging).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Huang Xin 2026-09-23 16:45:14 +08:00 • committed by GitHub
parent ed6dc77d0c
commit 8750c31043
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
25 changed files with 270 additions and 40 deletions

View file

@ -245,6 +245,7 @@ jobs:
NDK_HOME: ${{ env.ANDROID_HOME }}/ndk/28.2.13676358
TAURI_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
run: |
version="${{ needs.compute-version.outputs.nightly_version }}"
cd apps/readest-app/
@ -269,6 +270,16 @@ jobs:
cp ${apk_path}/app-universal-release.apk $universal_apk
pnpm tauri android build -t aarch64
cp ${apk_path}/app-universal-release.apk $arm64_apk
# Sentry symbolicates Rust panics from these: the APK ships the
# library stripped (see gen/android/app/build.gradle.kts), while
# `target/` keeps the unstripped copy. Crash reporting must never
# fail a build, and forks have no token.
if [ -n "$SENTRY_AUTH_TOKEN" ]; then
pnpm exec sentry-cli debug-files upload --org readest --project readest \
../../target/*/release/libreadestlib.so || echo "::warning::Sentry debug file upload failed"
fi
pnpm tauri signer sign $universal_apk
pnpm tauri signer sign $arm64_apk

View file

@ -453,6 +453,7 @@ jobs:
NDK_HOME: ${{ env.ANDROID_HOME }}/ndk/28.2.13676358
TAURI_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
run: |
cd apps/readest-app/
rm -rf src-tauri/gen/android
@ -476,6 +477,15 @@ jobs:
pnpm tauri android build -t aarch64
cp ${apk_path}/app-universal-release.apk $arm64_apk
# Sentry symbolicates Rust panics from these: the APK ships the
# library stripped (see gen/android/app/build.gradle.kts), while
# `target/` keeps the unstripped copy. Crash reporting must never
# fail a build, and forks have no token.
if [ -n "$SENTRY_AUTH_TOKEN" ]; then
pnpm exec sentry-cli debug-files upload --org readest --project readest \
../../target/*/release/libreadestlib.so || echo "::warning::Sentry debug file upload failed"
fi
echo "Uploading $universial_apk to GitHub release"
gh release upload ${{ needs.get-release.outputs.release_tag }} $universial_apk --clobber
echo "Uploading $arm64_apk to GitHub release"

23
Cargo.lock generated
View file

@ -2570,6 +2570,18 @@ version = "0.1.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3e0f1c7c3a72c66fd80abe965175f7523475c0489a87d3ff9d6e8c87d87a9d2d"
[[package]]
name = "findshlibs"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40b9e59cd0f7e0806cca4be089683ecb6434e602038df21fe6bf6711b2f07f64"
dependencies = [
"cc",
"lazy_static",
"libc",
"winapi",
]
[[package]]
name = "fixedbitset"
version = "0.5.7"
@ -6757,6 +6769,7 @@ dependencies = [
"sentry-backtrace",
"sentry-contexts",
"sentry-core",
"sentry-debug-images",
"sentry-panic",
"sentry-tracing",
"tokio",
@ -6801,6 +6814,16 @@ dependencies = [
"url",
]
[[package]]
name = "sentry-debug-images"
version = "0.42.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "00950648aa0d371c7f57057434ad5671bd4c106390df7e7284739330786a01b6"
dependencies = [
"findshlibs",
"sentry-core",
]
[[package]]
name = "sentry-panic"
version = "0.42.0"

View file

@ -54,3 +54,13 @@ tauri = { path = "packages/tauri/crates/tauri" }
# without that cargo cannot unify swift-rs and silently drops the tauri patch
# above in favour of crates.io tauri. See packages/swift-rs.
swift-rs = { path = "packages/swift-rs" }
[profile.release]
# The Android library is 95% of the APK, and one codegen unit is the one knob
# that shrinks the code itself: on aarch64 it cuts .text 29.7MB -> 24.4MB and
# the unwind/exception tables 6.2MB -> 4.1MB. It costs ~1.5min on a clean
# build and ~0.5GB of peak memory (measured aarch64-linux-android, M-series).
#
# LTO is deliberately NOT enabled: fat, thin and off all produce the same
# .text once codegen-units is 1, so it would buy nothing but link time.
codegen-units = 1

View file

@ -65,7 +65,10 @@ src-tauri/gen
src-tauri/tauri.*.local.json
# vendor
# `public/vendor` holds only what the reader fetches by URL; `vendor/` holds
# the copies the bundler imports (see vendor-aliases-outside-public.test.ts).
/public/vendor
/vendor
# docs
docs/superpowers

View file

@ -240,10 +240,13 @@ the adapters under `src/services/annotation`, `src/services/nav`,
features (annotations sync, navigation, content transforms, vertical/Warichu
support, classic mode overlays, etc.).
PDF rendering goes through `pdfjs-dist`, which is copied into
`public/vendor/pdfjs` at build time (`pnpm setup-pdfjs`). Chinese conversion
uses `simplecc-wasm` (`public/vendor/simplecc`), and Chinese segmentation uses
`jieba-wasm` (`public/vendor/jieba`).
PDF rendering goes through `pdfjs-dist` (`pnpm setup-pdfjs`), split by how each
file is loaded: what the reader fetches by URL at runtime — the worker, the
WASM, cmaps, standard fonts, the layer CSS — goes to `public/vendor/pdfjs`,
while `pdf.min.mjs`, which the bundler imports through the `@pdfjs` alias, goes
to `vendor/pdfjs` (outside `public/`, so Tauri does not embed a second copy).
Chinese conversion uses `simplecc-wasm` (`vendor/simplecc`, imported the same
way), and Chinese segmentation uses `jieba-wasm` (the file the bundler emits).
### 3.4 Service worker and offline

View file

@ -83,7 +83,7 @@ const nextConfig = {
// Without an alias, webpack walks up from that source location and
// can't find fflate (only installed in this app's node_modules).
fflate: path.resolve(__dirname, 'node_modules/fflate'),
...(appPlatform !== 'web' ? { '@tursodatabase/database-wasm': false } : {}),
...(appPlatform !== 'web' ? { '@readest/turso-database-wasm/webpack': false } : {}),
...(isServer && appPlatform === 'web'
? { '@readest/turso-database-wasm/webpack': false, 'jieba-wasm': false }
: {}),
@ -96,7 +96,12 @@ const nextConfig = {
// Turbopack rejects absolute paths in resolveAlias ("server relative
// imports not implemented") — use a project-relative path.
fflate: './node_modules/fflate',
...(appPlatform !== 'web' ? { '@tursodatabase/database-wasm': './src/utils/stub.ts' } : {}),
// Only the web build stores its database in turso's WASM build; Tauri
// embeds every file in `out/`, so emitting the unused WASM there would
// ship it inside the app.
...(appPlatform !== 'web'
? { '@readest/turso-database-wasm/webpack': './src/utils/stub.ts' }
: {}),
},
},
transpilePackages: [

View file

@ -52,20 +52,19 @@
"test:rust": "cargo test -p Readest --lib",
"format": "pnpm -w format",
"format:check": "pnpm -w format:check",
"prepare-public-vendor": "mkdirp ./public/vendor/pdfjs ./public/vendor/simplecc ./public/vendor/jieba",
"copy-pdfjs-js": "cpx \"../../packages/foliate-js/node_modules/pdfjs-dist/legacy/build/{pdf.worker.min.mjs,pdf.min.mjs,pdf.d.mts}\" ./public/vendor/pdfjs",
"prepare-vendor": "mkdirp ./public/vendor/pdfjs ./vendor/pdfjs ./vendor/simplecc",
"copy-pdfjs-worker": "cpx \"../../packages/foliate-js/node_modules/pdfjs-dist/legacy/build/pdf.worker.min.mjs\" ./public/vendor/pdfjs",
"copy-pdfjs-lib": "cpx \"../../packages/foliate-js/node_modules/pdfjs-dist/legacy/build/{pdf.min.mjs,pdf.d.mts}\" ./vendor/pdfjs",
"copy-pdfjs-wasm": "cpx \"../../packages/foliate-js/node_modules/pdfjs-dist/wasm/*\" ./public/vendor/pdfjs",
"copy-pdfjs-fonts": "cpx \"../../packages/foliate-js/node_modules/pdfjs-dist/{cmaps,standard_fonts}/*\" ./public/vendor/pdfjs",
"copy-flatten-pdfjs-annotation-layer-css": "npx postcss \"../../packages/foliate-js/vendor/pdfjs/annotation_layer_builder.css\" --no-map -u postcss-nested > ./public/vendor/pdfjs/annotation_layer_builder.css",
"copy-flatten-pdfjs-text-layer-css": "npx postcss \"../../packages/foliate-js/vendor/pdfjs/text_layer_builder.css\" --no-map -u postcss-nested > ./public/vendor/pdfjs/text_layer_builder.css",
"copy-flatten-pdfjs-css": "pnpm copy-flatten-pdfjs-annotation-layer-css && pnpm copy-flatten-pdfjs-text-layer-css",
"copy-pdfjs": "pnpm copy-pdfjs-js && pnpm copy-pdfjs-wasm && pnpm copy-pdfjs-fonts && pnpm copy-flatten-pdfjs-css",
"copy-simplecc": "cpx \"../../packages/simplecc-wasm/dist/web/*\" ./public/vendor/simplecc",
"copy-jieba": "cpx \"./node_modules/jieba-wasm/pkg/web/*\" ./public/vendor/jieba",
"setup-pdfjs": "pnpm prepare-public-vendor && pnpm copy-pdfjs",
"setup-simplecc": "pnpm prepare-public-vendor && pnpm copy-simplecc",
"setup-jieba": "pnpm prepare-public-vendor && pnpm copy-jieba",
"setup-vendors": "pnpm setup-pdfjs && pnpm setup-simplecc && pnpm setup-jieba",
"copy-pdfjs": "pnpm copy-pdfjs-worker && pnpm copy-pdfjs-lib && pnpm copy-pdfjs-wasm && pnpm copy-pdfjs-fonts && pnpm copy-flatten-pdfjs-css",
"copy-simplecc": "cpx \"../../packages/simplecc-wasm/dist/web/*\" ./vendor/simplecc",
"setup-pdfjs": "pnpm prepare-vendor && pnpm copy-pdfjs",
"setup-simplecc": "pnpm prepare-vendor && pnpm copy-simplecc",
"setup-vendors": "pnpm setup-pdfjs && pnpm setup-simplecc",
"build-win-x64": "dotenv -e .env.tauri.local -- tauri build --target i686-pc-windows-msvc --bundles nsis",
"build-win-arm64": "dotenv -e .env.tauri.local -- tauri build --target aarch64-pc-windows-msvc --bundles nsis",
"build-linux-x64": "dotenv -e .env.tauri.local -- pnpm tauri build --bundles appimage",

View file

@ -283,12 +283,16 @@ for (const sub of ['schemas', 'android/keystore.properties']) {
}
}
// Copy public/vendor to the new worktree (built assets not in git)
const srcVendor = path.join(srcAppDir, 'public', 'vendor');
const dstVendor = path.join(dstAppDir, 'public', 'vendor');
if (fs.existsSync(srcVendor) && !fs.existsSync(dstVendor)) {
console.error('\n--- Copying public/vendor ---');
fs.cpSync(srcVendor, dstVendor, { recursive: true });
// Copy the vendored assets to the new worktree (built, not in git):
// `public/vendor` is fetched by URL at runtime, `vendor` is imported by the
// bundler through the @pdfjs / @simplecc aliases.
for (const sub of [['public', 'vendor'], ['vendor']]) {
const srcVendor = path.join(srcAppDir, ...sub);
const dstVendor = path.join(dstAppDir, ...sub);
if (fs.existsSync(srcVendor) && !fs.existsSync(dstVendor)) {
console.error(`\n--- Copying ${sub.join('/')} ---`);
fs.cpSync(srcVendor, dstVendor, { recursive: true });
}
}
// Print path to stdout -- allows: cd $(pnpm worktree:new <arg>)

View file

@ -175,9 +175,15 @@ flate2 = "1"
# never re-exec itself so it aborted in dyld (#5053), and its crash-handler
# `pthread_create` interposer panics on 32-bit ARM, aborting every armeabi-v7a
# device (#5070). See `minidump_feature_is_enabled_on_no_target`.
#
# `debug-images` attaches the loaded libraries and their build ids to each
# event. The Android library ships stripped, so its frames carry addresses
# only, and this is what lets Sentry resolve them against the debug files CI
# uploads.
sentry = { version = "0.42", default-features = false, features = [
"backtrace",
"contexts",
"debug-images",
"panic",
"reqwest",
"rustls",

View file

@ -10,6 +10,13 @@ fn main() {
if target_os == "windows" {
build_windows_thumbnail();
}
if target_os == "android" {
// The APK ships the library stripped (see gen/android/app/build.gradle.kts),
// so Sentry symbolicates Rust panics from the debug files CI uploads. It
// matches them to the crashing library by build id, and the NDK linker
// emits none by default.
println!("cargo:rustc-link-arg=-Wl,--build-id=sha1");
}
propagate_sentry_dsn();
propagate_app_version();

View file

@ -1,5 +1,6 @@
import java.util.Properties
import java.io.FileInputStream
import org.apache.tools.ant.taskdefs.condition.Os
plugins {
id("com.android.application")
@ -110,6 +111,44 @@ rust {
rootDirRel = "../../../"
}
// AGP's own strip step only drops the .debug_* sections, so the Rust library
// reached the APK carrying 22MB of symbol tables (.symtab + .strtab) — a
// quarter of the download. Strip those too, after AGP has run and on the copy
// it packages: `target/` keeps the unstripped library, which is what the
// release workflow uploads to Sentry so Rust panics still symbolicate.
//
// The NDK is located by hand because this module declares no `ndkVersion`, so
// `android.ndkDirectory` throws "NDK is not installed". The Tauri CLI sets
// NDK_HOME for the cargo linkers; the SDK scan covers a reused Gradle daemon
// that started without it in its environment.
val llvmStripExecutable = if (Os.isFamily(Os.FAMILY_WINDOWS)) "llvm-strip.exe" else "llvm-strip"
fun findLlvmStrip(): File {
val ndkRoots = listOfNotNull(
System.getenv("NDK_HOME"),
System.getenv("ANDROID_NDK_HOME"),
System.getenv("ANDROID_NDK_ROOT"),
).map { File(it) } +
File(android.sdkDirectory, "ndk").listFiles().orEmpty().sortedDescending()
return ndkRoots
.flatMap { File(it, "toolchains/llvm/prebuilt").listFiles().orEmpty().toList() }
.map { File(it, "bin/$llvmStripExecutable") }
.firstOrNull { it.isFile }
?: throw GradleException("$llvmStripExecutable not found; looked in $ndkRoots")
}
tasks.configureEach {
if (!name.startsWith("strip") || !name.endsWith("ReleaseDebugSymbols")) return@configureEach
doLast {
val llvmStrip = findLlvmStrip()
outputs.files.asFileTree.matching { include("**/*.so") }.forEach { lib ->
providers.exec {
commandLine(llvmStrip.absolutePath, "--strip-all", lib.absolutePath)
}.result.get().assertNormalExitValue()
}
}
}
dependencies {
implementation("androidx.webkit:webkit:1.14.0")
implementation("androidx.appcompat:appcompat:1.7.1")

View file

@ -58,8 +58,12 @@ describe('pdfjs vendor wasm assets', () => {
it('copies every wasm decoder the bundled pdf.js references', () => {
// Files the worker/main bundle are copied from (source of truth for CI).
const jsGlob = sourceGlobOf(pkg.scripts['copy-pdfjs-js']!);
const { dir: jsDir, files: jsFiles } = resolveCpxGlob(jsGlob);
// The worker is fetched by URL so it lives in `public/vendor/pdfjs`, while
// the library is imported through the @pdfjs alias and so lives outside
// `public/` — scan both, either can carry the wasm references.
const jsSources = (['copy-pdfjs-worker', 'copy-pdfjs-lib'] as const).map((script) =>
resolveCpxGlob(sourceGlobOf(pkg.scripts[script]!)),
);
// The wasm modules available in pdfjs-dist and what the copy script ships.
const wasmGlob = sourceGlobOf(pkg.scripts['copy-pdfjs-wasm']!);
@ -70,12 +74,14 @@ describe('pdfjs vendor wasm assets', () => {
// Scan the bundled JS for `*.wasm` references, keeping only ones that map to
// a real file (drops minified false positives like `e.wasm`/`t.wasm`).
const referenced = new Set<string>();
for (const file of jsFiles) {
const full = resolve(jsDir, file);
if (!existsSync(full)) continue;
const text = readFileSync(full, 'utf8');
for (const m of text.matchAll(/[A-Za-z0-9_-]+\.wasm/g)) {
if (availableWasm.has(m[0])) referenced.add(m[0]);
for (const { dir: jsDir, files: jsFiles } of jsSources) {
for (const file of jsFiles) {
const full = resolve(jsDir, file);
if (!existsSync(full)) continue;
const text = readFileSync(full, 'utf8');
for (const m of text.matchAll(/[A-Za-z0-9_-]+\.wasm/g)) {
if (availableWasm.has(m[0])) referenced.add(m[0]);
}
}
}

View file

@ -64,3 +64,31 @@ describe('Proxy request body clone limit', () => {
);
});
});
describe('Browser-only database WASM', () => {
// The web build stores its database in turso's WASM build; Tauri uses the
// native turso crate. Tauri embeds every file in `out/` into the binary, so
// the WASM chunk must not be emitted there (a 12.7MB file, 3.3MB in the APK).
afterEach(() => {
vi.unstubAllEnvs();
vi.resetModules();
});
const loadConfig = async (platform: string) => {
vi.stubEnv('NEXT_PUBLIC_APP_PLATFORM', platform);
vi.resetModules();
return (await import('../../next.config.mjs')).default;
};
test('the Tauri build stubs the package the web database imports', async () => {
const tauri = await loadConfig('tauri');
expect(tauri.turbopack?.resolveAlias?.['@readest/turso-database-wasm/webpack']).toBe(
'./src/utils/stub.ts',
);
});
test('the web build keeps it', async () => {
const web = await loadConfig('web');
expect(web.turbopack?.resolveAlias?.['@readest/turso-database-wasm/webpack']).toBeUndefined();
});
});

View file

@ -9,7 +9,7 @@ import { join } from 'path';
describe.concurrent('jieba-wasm', () => {
beforeAll(async () => {
const wasmPath = join(process.cwd(), 'public/vendor/jieba/jieba_rs_wasm_bg.wasm');
const wasmPath = join(process.cwd(), 'node_modules/jieba-wasm/pkg/web/jieba_rs_wasm_bg.wasm');
const wasmBuffer = await readFile(wasmPath);
await init({ module_or_path: wasmBuffer });
});

View file

@ -5,7 +5,7 @@ import { join } from 'path';
describe.concurrent('suite', () => {
beforeAll(async () => {
const wasmPath = join(process.cwd(), 'public/vendor/simplecc/simplecc_wasm_bg.wasm');
const wasmPath = join(process.cwd(), 'vendor/simplecc/simplecc_wasm_bg.wasm');
const wasmBuffer = await readFile(wasmPath);
await init({ module_or_path: wasmBuffer });
});

View file

@ -0,0 +1,30 @@
import { beforeEach, describe, expect, test, vi } from 'vitest';
// Both wasm-bindgen glues default to `new URL('<name>_bg.wasm', import.meta.url)`,
// which the bundler always emits and Tauri embeds. Passing a `/vendor/...` URL
// instead needs a published copy: it either ships the WASM twice (#6368) or,
// once the published copy is gone, 404s at runtime.
const jiebaInit = vi.fn();
const simpleccInit = vi.fn();
vi.mock('jieba-wasm', () => ({ default: jiebaInit, cut: vi.fn() }));
vi.mock('@simplecc/simplecc_wasm', () => ({ default: simpleccInit, simplecc: vi.fn() }));
describe('WASM init uses the bundler-emitted file', () => {
beforeEach(() => {
vi.resetModules();
jiebaInit.mockReset();
simpleccInit.mockReset();
});
test('jieba', async () => {
const { initJieba } = await import('@/utils/jieba');
await initJieba();
expect(jiebaInit).toHaveBeenCalledWith();
});
test('simplecc', async () => {
const { initSimpleCC } = await import('@/utils/simplecc');
await initSimpleCC();
expect(simpleccInit).toHaveBeenCalledWith();
});
});

View file

@ -0,0 +1,40 @@
import { describe, expect, test } from 'vitest';
import { existsSync, readFileSync } from 'node:fs';
import path from 'node:path';
// Tauri embeds every file under `out/` into the app binary, and `public/` is
// copied there wholesale. So a module that is ALSO imported through a path
// alias ships twice: once bundled into a chunk, once as a published file
// nobody fetches. jieba cost 2.7MB that way (#6368).
//
// The rule: an alias used at build time resolves outside `public/`. Only files
// fetched by URL at runtime belong in `public/vendor`.
const appDir = path.resolve(import.meta.dirname, '../..');
const readJson = (file: string) =>
JSON.parse(readFileSync(path.join(appDir, file), 'utf8').replace(/^\s*\/\/.*$/gm, ''));
describe('vendored build-time modules live outside public/', () => {
test('no tsconfig path alias resolves into public/', () => {
const paths: Record<string, string[]> = readJson('tsconfig.json').compilerOptions.paths;
const inPublic = Object.entries(paths)
.filter(([, targets]) => targets.some((t) => t.replace(/^\.\//, '').startsWith('public/')))
.map(([alias]) => alias);
expect(inPublic).toEqual([]);
});
test('pdf.js and simplecc are vendored for the bundler, not published', () => {
expect(existsSync(path.join(appDir, 'vendor/pdfjs/pdf.min.mjs'))).toBe(true);
expect(existsSync(path.join(appDir, 'public/vendor/pdfjs/pdf.min.mjs'))).toBe(false);
expect(existsSync(path.join(appDir, 'vendor/simplecc/simplecc_wasm_bg.wasm'))).toBe(true);
expect(existsSync(path.join(appDir, 'public/vendor/simplecc'))).toBe(false);
});
test('what the reader fetches by URL stays published', () => {
// foliate-js/pdf.js builds these URLs at runtime, so the bundler never
// sees a specifier for them and cannot emit a copy.
for (const f of ['pdf.worker.min.mjs', 'openjpeg.wasm', 'text_layer_builder.css']) {
expect(existsSync(path.join(appDir, 'public/vendor/pdfjs', f))).toBe(true);
}
});
});

View file

@ -8,7 +8,10 @@ const initJieba = async (): Promise<void> => {
if (!initPromise) {
initPromise = (async () => {
try {
await init('/vendor/jieba/jieba_rs_wasm_bg.wasm');
// No path: the glue falls back to `new URL('jieba_rs_wasm_bg.wasm',
// import.meta.url)`, the copy the bundler already emits. Pointing it at
// a public/vendor copy instead shipped the 4MB WASM twice.
await init();
initialized = true;
} catch (e) {
initPromise = null;

View file

@ -6,7 +6,10 @@ let initialized = false;
const initSimpleCC = async () => {
if (initialized) return;
await init('/vendor/simplecc/simplecc_wasm_bg.wasm');
// No path: the glue falls back to `new URL('simplecc_wasm_bg.wasm',
// import.meta.url)`, the copy the bundler emits. The public/vendor copy this
// used to fetch was the second, embedded-twice copy (#6368).
await init();
initialized = true;
};

View file

@ -24,8 +24,8 @@
"paths": {
"@/*": ["./src/*"],
"@/components/ui/*": ["./src/components/primitives/*"],
"@pdfjs/*": ["./public/vendor/pdfjs/*"],
"@simplecc/*": ["./public/vendor/simplecc/*"],
"@pdfjs/*": ["./vendor/pdfjs/*"],
"@simplecc/*": ["./vendor/simplecc/*"],
"js-mdict": ["../../packages/js-mdict/src/index.ts"],
"fflate": ["./node_modules/fflate"],
"tauri-plugin-turso": ["./src-tauri/plugins/tauri-plugin-turso/guest-js"]

View file

@ -6,8 +6,8 @@ export default defineConfig({
plugins: [vinext()],
resolve: {
alias: {
'@pdfjs': path.resolve('public/vendor/pdfjs'),
'@simplecc': path.resolve('public/vendor/simplecc'),
'@pdfjs': path.resolve('vendor/pdfjs'),
'@simplecc': path.resolve('vendor/simplecc'),
},
},
build: {

View file

@ -24,7 +24,7 @@ export default defineConfig({
// The @pdfjs alias from tsconfig only resolves within the app's own
// source files. foliate-js/pdf.js lives outside that scope, so Vite
// needs an explicit alias to find the vendored pdfjs build.
'@pdfjs': resolve(import.meta.dirname, 'public/vendor/pdfjs'),
'@pdfjs': resolve(import.meta.dirname, 'vendor/pdfjs'),
},
},
optimizeDeps: {

View file

@ -10,7 +10,7 @@ export default defineConfig({
// The @pdfjs alias from tsconfig only resolves within the app's own
// source files. foliate-js/pdf.js lives outside that scope, so Vite
// needs an explicit alias to find the vendored pdfjs build.
'@pdfjs': path.resolve(__dirname, 'public/vendor/pdfjs'),
'@pdfjs': path.resolve(__dirname, 'vendor/pdfjs'),
// `js-mdict` is consumed via tsconfig paths from `packages/js-mdict/src/`.
// Its sources `import 'fflate'` directly — without an alias, vite's
// import-analysis walks up from the redirected file location and fails

View file

@ -14,7 +14,7 @@ export default defineConfig({
},
resolve: {
alias: {
'@pdfjs': path.resolve(__dirname, 'public/vendor/pdfjs'),
'@pdfjs': path.resolve(__dirname, 'vendor/pdfjs'),
},
conditions: ['development'],
},