mirror of
https://github.com/readest/readest.git
synced 2026-10-21 14:12:43 +00:00
perf(android): cut the arm64 apk from 90MB to 57MB (#6368)
Some checks are pending
CodeQL Advanced / Analyze (actions) (push) Waiting to run
CodeQL Advanced / Analyze (javascript-typescript) (push) Waiting to run
CodeQL Advanced / Analyze (rust) (push) Waiting to run
Publish Docker image / build (linux/amd64, ubuntu-latest) (push) Waiting to run
Publish Docker image / build (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Publish Docker image / merge (push) Blocked by required conditions
Build with Nix / tests (push) Waiting to run
PR checks / rust_lint (push) Waiting to run
PR checks / koplugin_rust_lint (push) Waiting to run
PR checks / nix_flake_check (push) Waiting to run
PR checks / build_web_app (push) Waiting to run
PR checks / test_web_app (1) (push) Waiting to run
PR checks / test_web_app (2) (push) Waiting to run
PR checks / test_extensions (push) Waiting to run
PR checks / build_tauri_app (push) Waiting to run
Scorecard supply-chain security / Scorecard analysis (push) Waiting to run
Deploy to vercel on merge / build_and_deploy (push) Waiting to run
Some checks are pending
CodeQL Advanced / Analyze (actions) (push) Waiting to run
CodeQL Advanced / Analyze (javascript-typescript) (push) Waiting to run
CodeQL Advanced / Analyze (rust) (push) Waiting to run
Publish Docker image / build (linux/amd64, ubuntu-latest) (push) Waiting to run
Publish Docker image / build (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Publish Docker image / merge (push) Blocked by required conditions
Build with Nix / tests (push) Waiting to run
PR checks / rust_lint (push) Waiting to run
PR checks / koplugin_rust_lint (push) Waiting to run
PR checks / nix_flake_check (push) Waiting to run
PR checks / build_web_app (push) Waiting to run
PR checks / test_web_app (1) (push) Waiting to run
PR checks / test_web_app (2) (push) Waiting to run
PR checks / test_extensions (push) Waiting to run
PR checks / build_tauri_app (push) Waiting to run
Scorecard supply-chain security / Scorecard analysis (push) Waiting to run
Deploy to vercel on merge / build_and_deploy (push) Waiting to run
* perf(android): cut the arm64 apk from 90MB to 57MB The Rust library is 95% of the APK, and most of what it carried was not code. Measured on an aarch64 release build: 89.7MB -> 57.5MB. - Build `release` with fat LTO and one codegen unit. Cross-crate inlining cuts .text by 5.2MB and the unwind/exception tables by 2.1MB, at the cost of a longer final link. - Strip the packaged library. AGP's own strip step only drops the .debug_* sections, so 22MB of symbol tables (10MB after LTO) reached every download. The strip runs on AGP's output, so `target/` keeps the unstripped copy, which the release and nightly workflows now upload to Sentry as debug files; Rust frames stay symbolicated. Android builds also link with a build id, which is how Sentry matches them, and the `debug-images` feature attaches the loaded modules to each event. - Stop shipping jieba's WASM twice (2.7MB). The wasm-bindgen glue defaults to the file the bundler already emits; pointing `init()` at a public/vendor copy embedded both. - Keep the browser-only turso WASM out of native builds (3.3MB). The alias that excluded it still named `@tursodatabase/database-wasm`, which the web database service stopped importing in #4086. simplecc is still embedded twice (0.4MB): its glue is imported through the `@simplecc/*` path alias out of public/vendor, so deduping it means moving that directory. Verified: the built APK's library carries no symbol tables and a build id; `target/` keeps its; the bundle requests exactly the one embedded jieba asset key and no /vendor/jieba path; the turso WASM is absent from `out/`. The APK installs and runs on a Xiaomi 2211133C, library and a Chinese book render, and RSVP runs on it. Whether jieba's WASM is actually fetched at runtime is NOT verified: release webviews are not inspectable and the fallback to Intl.Segmenter is silent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(android): drop fat LTO, codegen-units=1 is the whole win The previous commit credited the code-size saving to fat LTO. That was wrong. Measured on aarch64-linux-android with the flags rustc actually received (`cargo build -v`): codegen-units=16, lto=false .text 29.66MB .eh_frame 3.69 .except 2.51 codegen-units=1, lto=off .text 24.43MB .eh_frame 2.28 .except 1.84 codegen-units=1, lto=thin .text 24.43MB .eh_frame 2.28 .except 1.84 codegen-units=1, lto=fat .text 24.44MB .eh_frame 2.28 .except 1.84 One codegen unit accounts for all of it; LTO changes nothing once it is set, so fat LTO was buying only link time and peak memory on every release target. The rebuilt APK confirms it: 57.49MB without LTO against 57.50MB with it. What `codegen-units = 1` costs, from a clean aarch64 build on an M-series: 5m54s against 4m15s, and 3.3GB peak RSS against 2.9GB. That answers the review's question about the 75-minute Android and 45-minute desktop step limits in nightly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf: stop shipping vendored modules the bundler already embeds Tauri embeds every file under `out/`, and `public/` is copied there whole. So a module imported through a path alias that points into `public/` ships twice: once bundled into a chunk, once as a published file nobody fetches. jieba cost 2.7MB that way; `@pdfjs/*` and `@simplecc/*` had the same shape. Split the vendored assets by how they are loaded. What the reader fetches by URL at runtime stays in `public/vendor/pdfjs` — the worker, the WASM decoders, cmaps, standard fonts, the layer CSS, all built as runtime URLs in foliate-js/pdf.js, which is why the bundler never emitted copies of them. What the bundler imports moves to `vendor/` (outside `public/`): `pdf.min.mjs` + its types, and the simplecc glue and WASM. Measured on the embedded frontend, brotli as tauri compresses it: 23.7MB -> 16.96MB across this and the earlier jieba/turso fixes. That comes out of every Tauri target, not just the APK. The new test states the invariant that was broken rather than the three symptoms: no tsconfig path alias may resolve into `public/`. It failed on `@pdfjs/*` and `@simplecc/*` before this change. Desktop stripping was measured and deliberately NOT done. On macOS it saves 4.76MB in the binary but only 1.44MB of DMG (36.59 -> 35.15MB), because a DMG compresses symbol strings while an APK stores the .so uncompressed (verified: `Stored`, 0%) — which is why Android got the full 22MB. Buying 1.4MB would mean either a profile-level `strip`, which deletes the unstripped artifact Sentry symbolicates Rust panics from, or pulling the release job apart from tauri-action, which builds, bundles, signs and notarizes in one step. deb, rpm and AppImage are compressed too, and MSVC keeps symbols in the PDB. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: init simplecc from the emitted WASM, not a URL that no longer exists Moving the simplecc glue out of `public/` left `initSimpleCC` still asking for `/vendor/simplecc/simplecc_wasm_bg.wasm`, a URL nothing publishes any more, so Chinese variant conversion would fail to initialize at runtime. Caught in review of #6368. `init()` with no path uses the glue's default `new URL('simplecc_wasm_bg.wasm', import.meta.url)` — the copy the bundler emits and Tauri embeds — the same shape as the jieba fix. The existing WASM-init test covered jieba only, which is why this slipped through; it now covers both. Verified against a fresh build: the chunk references `static/media/simplecc_wasm_bg.<hash>.wasm` and no runtime `/vendor/simplecc` reference remains (the two textual hits left are the bundled package.json script strings and a source-map path, and maps are stripped before packaging). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
ed6dc77d0c
commit
8750c31043
25 changed files with 270 additions and 40 deletions
11
.github/workflows/nightly.yml
vendored
11
.github/workflows/nightly.yml
vendored
|
|
@ -245,6 +245,7 @@ jobs:
|
|||
NDK_HOME: ${{ env.ANDROID_HOME }}/ndk/28.2.13676358
|
||||
TAURI_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
|
||||
TAURI_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
|
||||
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
|
||||
run: |
|
||||
version="${{ needs.compute-version.outputs.nightly_version }}"
|
||||
cd apps/readest-app/
|
||||
|
|
@ -269,6 +270,16 @@ jobs:
|
|||
cp ${apk_path}/app-universal-release.apk $universal_apk
|
||||
pnpm tauri android build -t aarch64
|
||||
cp ${apk_path}/app-universal-release.apk $arm64_apk
|
||||
|
||||
# Sentry symbolicates Rust panics from these: the APK ships the
|
||||
# library stripped (see gen/android/app/build.gradle.kts), while
|
||||
# `target/` keeps the unstripped copy. Crash reporting must never
|
||||
# fail a build, and forks have no token.
|
||||
if [ -n "$SENTRY_AUTH_TOKEN" ]; then
|
||||
pnpm exec sentry-cli debug-files upload --org readest --project readest \
|
||||
../../target/*/release/libreadestlib.so || echo "::warning::Sentry debug file upload failed"
|
||||
fi
|
||||
|
||||
pnpm tauri signer sign $universal_apk
|
||||
pnpm tauri signer sign $arm64_apk
|
||||
|
||||
|
|
|
|||
10
.github/workflows/release.yml
vendored
10
.github/workflows/release.yml
vendored
|
|
@ -453,6 +453,7 @@ jobs:
|
|||
NDK_HOME: ${{ env.ANDROID_HOME }}/ndk/28.2.13676358
|
||||
TAURI_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
|
||||
TAURI_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
|
||||
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
|
||||
run: |
|
||||
cd apps/readest-app/
|
||||
rm -rf src-tauri/gen/android
|
||||
|
|
@ -476,6 +477,15 @@ jobs:
|
|||
pnpm tauri android build -t aarch64
|
||||
cp ${apk_path}/app-universal-release.apk $arm64_apk
|
||||
|
||||
# Sentry symbolicates Rust panics from these: the APK ships the
|
||||
# library stripped (see gen/android/app/build.gradle.kts), while
|
||||
# `target/` keeps the unstripped copy. Crash reporting must never
|
||||
# fail a build, and forks have no token.
|
||||
if [ -n "$SENTRY_AUTH_TOKEN" ]; then
|
||||
pnpm exec sentry-cli debug-files upload --org readest --project readest \
|
||||
../../target/*/release/libreadestlib.so || echo "::warning::Sentry debug file upload failed"
|
||||
fi
|
||||
|
||||
echo "Uploading $universial_apk to GitHub release"
|
||||
gh release upload ${{ needs.get-release.outputs.release_tag }} $universial_apk --clobber
|
||||
echo "Uploading $arm64_apk to GitHub release"
|
||||
|
|
|
|||
23
Cargo.lock
generated
23
Cargo.lock
generated
|
|
@ -2570,6 +2570,18 @@ version = "0.1.12"
|
|||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3e0f1c7c3a72c66fd80abe965175f7523475c0489a87d3ff9d6e8c87d87a9d2d"
|
||||
|
||||
[[package]]
|
||||
name = "findshlibs"
|
||||
version = "0.10.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "40b9e59cd0f7e0806cca4be089683ecb6434e602038df21fe6bf6711b2f07f64"
|
||||
dependencies = [
|
||||
"cc",
|
||||
"lazy_static",
|
||||
"libc",
|
||||
"winapi",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "fixedbitset"
|
||||
version = "0.5.7"
|
||||
|
|
@ -6757,6 +6769,7 @@ dependencies = [
|
|||
"sentry-backtrace",
|
||||
"sentry-contexts",
|
||||
"sentry-core",
|
||||
"sentry-debug-images",
|
||||
"sentry-panic",
|
||||
"sentry-tracing",
|
||||
"tokio",
|
||||
|
|
@ -6801,6 +6814,16 @@ dependencies = [
|
|||
"url",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sentry-debug-images"
|
||||
version = "0.42.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "00950648aa0d371c7f57057434ad5671bd4c106390df7e7284739330786a01b6"
|
||||
dependencies = [
|
||||
"findshlibs",
|
||||
"sentry-core",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sentry-panic"
|
||||
version = "0.42.0"
|
||||
|
|
|
|||
10
Cargo.toml
10
Cargo.toml
|
|
@ -54,3 +54,13 @@ tauri = { path = "packages/tauri/crates/tauri" }
|
|||
# without that cargo cannot unify swift-rs and silently drops the tauri patch
|
||||
# above in favour of crates.io tauri. See packages/swift-rs.
|
||||
swift-rs = { path = "packages/swift-rs" }
|
||||
|
||||
[profile.release]
|
||||
# The Android library is 95% of the APK, and one codegen unit is the one knob
|
||||
# that shrinks the code itself: on aarch64 it cuts .text 29.7MB -> 24.4MB and
|
||||
# the unwind/exception tables 6.2MB -> 4.1MB. It costs ~1.5min on a clean
|
||||
# build and ~0.5GB of peak memory (measured aarch64-linux-android, M-series).
|
||||
#
|
||||
# LTO is deliberately NOT enabled: fat, thin and off all produce the same
|
||||
# .text once codegen-units is 1, so it would buy nothing but link time.
|
||||
codegen-units = 1
|
||||
|
|
|
|||
3
apps/readest-app/.gitignore
vendored
3
apps/readest-app/.gitignore
vendored
|
|
@ -65,7 +65,10 @@ src-tauri/gen
|
|||
src-tauri/tauri.*.local.json
|
||||
|
||||
# vendor
|
||||
# `public/vendor` holds only what the reader fetches by URL; `vendor/` holds
|
||||
# the copies the bundler imports (see vendor-aliases-outside-public.test.ts).
|
||||
/public/vendor
|
||||
/vendor
|
||||
|
||||
# docs
|
||||
docs/superpowers
|
||||
|
|
|
|||
|
|
@ -240,10 +240,13 @@ the adapters under `src/services/annotation`, `src/services/nav`,
|
|||
features (annotations sync, navigation, content transforms, vertical/Warichu
|
||||
support, classic mode overlays, etc.).
|
||||
|
||||
PDF rendering goes through `pdfjs-dist`, which is copied into
|
||||
`public/vendor/pdfjs` at build time (`pnpm setup-pdfjs`). Chinese conversion
|
||||
uses `simplecc-wasm` (`public/vendor/simplecc`), and Chinese segmentation uses
|
||||
`jieba-wasm` (`public/vendor/jieba`).
|
||||
PDF rendering goes through `pdfjs-dist` (`pnpm setup-pdfjs`), split by how each
|
||||
file is loaded: what the reader fetches by URL at runtime — the worker, the
|
||||
WASM, cmaps, standard fonts, the layer CSS — goes to `public/vendor/pdfjs`,
|
||||
while `pdf.min.mjs`, which the bundler imports through the `@pdfjs` alias, goes
|
||||
to `vendor/pdfjs` (outside `public/`, so Tauri does not embed a second copy).
|
||||
Chinese conversion uses `simplecc-wasm` (`vendor/simplecc`, imported the same
|
||||
way), and Chinese segmentation uses `jieba-wasm` (the file the bundler emits).
|
||||
|
||||
### 3.4 Service worker and offline
|
||||
|
||||
|
|
|
|||
|
|
@ -83,7 +83,7 @@ const nextConfig = {
|
|||
// Without an alias, webpack walks up from that source location and
|
||||
// can't find fflate (only installed in this app's node_modules).
|
||||
fflate: path.resolve(__dirname, 'node_modules/fflate'),
|
||||
...(appPlatform !== 'web' ? { '@tursodatabase/database-wasm': false } : {}),
|
||||
...(appPlatform !== 'web' ? { '@readest/turso-database-wasm/webpack': false } : {}),
|
||||
...(isServer && appPlatform === 'web'
|
||||
? { '@readest/turso-database-wasm/webpack': false, 'jieba-wasm': false }
|
||||
: {}),
|
||||
|
|
@ -96,7 +96,12 @@ const nextConfig = {
|
|||
// Turbopack rejects absolute paths in resolveAlias ("server relative
|
||||
// imports not implemented") — use a project-relative path.
|
||||
fflate: './node_modules/fflate',
|
||||
...(appPlatform !== 'web' ? { '@tursodatabase/database-wasm': './src/utils/stub.ts' } : {}),
|
||||
// Only the web build stores its database in turso's WASM build; Tauri
|
||||
// embeds every file in `out/`, so emitting the unused WASM there would
|
||||
// ship it inside the app.
|
||||
...(appPlatform !== 'web'
|
||||
? { '@readest/turso-database-wasm/webpack': './src/utils/stub.ts' }
|
||||
: {}),
|
||||
},
|
||||
},
|
||||
transpilePackages: [
|
||||
|
|
|
|||
|
|
@ -52,20 +52,19 @@
|
|||
"test:rust": "cargo test -p Readest --lib",
|
||||
"format": "pnpm -w format",
|
||||
"format:check": "pnpm -w format:check",
|
||||
"prepare-public-vendor": "mkdirp ./public/vendor/pdfjs ./public/vendor/simplecc ./public/vendor/jieba",
|
||||
"copy-pdfjs-js": "cpx \"../../packages/foliate-js/node_modules/pdfjs-dist/legacy/build/{pdf.worker.min.mjs,pdf.min.mjs,pdf.d.mts}\" ./public/vendor/pdfjs",
|
||||
"prepare-vendor": "mkdirp ./public/vendor/pdfjs ./vendor/pdfjs ./vendor/simplecc",
|
||||
"copy-pdfjs-worker": "cpx \"../../packages/foliate-js/node_modules/pdfjs-dist/legacy/build/pdf.worker.min.mjs\" ./public/vendor/pdfjs",
|
||||
"copy-pdfjs-lib": "cpx \"../../packages/foliate-js/node_modules/pdfjs-dist/legacy/build/{pdf.min.mjs,pdf.d.mts}\" ./vendor/pdfjs",
|
||||
"copy-pdfjs-wasm": "cpx \"../../packages/foliate-js/node_modules/pdfjs-dist/wasm/*\" ./public/vendor/pdfjs",
|
||||
"copy-pdfjs-fonts": "cpx \"../../packages/foliate-js/node_modules/pdfjs-dist/{cmaps,standard_fonts}/*\" ./public/vendor/pdfjs",
|
||||
"copy-flatten-pdfjs-annotation-layer-css": "npx postcss \"../../packages/foliate-js/vendor/pdfjs/annotation_layer_builder.css\" --no-map -u postcss-nested > ./public/vendor/pdfjs/annotation_layer_builder.css",
|
||||
"copy-flatten-pdfjs-text-layer-css": "npx postcss \"../../packages/foliate-js/vendor/pdfjs/text_layer_builder.css\" --no-map -u postcss-nested > ./public/vendor/pdfjs/text_layer_builder.css",
|
||||
"copy-flatten-pdfjs-css": "pnpm copy-flatten-pdfjs-annotation-layer-css && pnpm copy-flatten-pdfjs-text-layer-css",
|
||||
"copy-pdfjs": "pnpm copy-pdfjs-js && pnpm copy-pdfjs-wasm && pnpm copy-pdfjs-fonts && pnpm copy-flatten-pdfjs-css",
|
||||
"copy-simplecc": "cpx \"../../packages/simplecc-wasm/dist/web/*\" ./public/vendor/simplecc",
|
||||
"copy-jieba": "cpx \"./node_modules/jieba-wasm/pkg/web/*\" ./public/vendor/jieba",
|
||||
"setup-pdfjs": "pnpm prepare-public-vendor && pnpm copy-pdfjs",
|
||||
"setup-simplecc": "pnpm prepare-public-vendor && pnpm copy-simplecc",
|
||||
"setup-jieba": "pnpm prepare-public-vendor && pnpm copy-jieba",
|
||||
"setup-vendors": "pnpm setup-pdfjs && pnpm setup-simplecc && pnpm setup-jieba",
|
||||
"copy-pdfjs": "pnpm copy-pdfjs-worker && pnpm copy-pdfjs-lib && pnpm copy-pdfjs-wasm && pnpm copy-pdfjs-fonts && pnpm copy-flatten-pdfjs-css",
|
||||
"copy-simplecc": "cpx \"../../packages/simplecc-wasm/dist/web/*\" ./vendor/simplecc",
|
||||
"setup-pdfjs": "pnpm prepare-vendor && pnpm copy-pdfjs",
|
||||
"setup-simplecc": "pnpm prepare-vendor && pnpm copy-simplecc",
|
||||
"setup-vendors": "pnpm setup-pdfjs && pnpm setup-simplecc",
|
||||
"build-win-x64": "dotenv -e .env.tauri.local -- tauri build --target i686-pc-windows-msvc --bundles nsis",
|
||||
"build-win-arm64": "dotenv -e .env.tauri.local -- tauri build --target aarch64-pc-windows-msvc --bundles nsis",
|
||||
"build-linux-x64": "dotenv -e .env.tauri.local -- pnpm tauri build --bundles appimage",
|
||||
|
|
|
|||
|
|
@ -283,12 +283,16 @@ for (const sub of ['schemas', 'android/keystore.properties']) {
|
|||
}
|
||||
}
|
||||
|
||||
// Copy public/vendor to the new worktree (built assets not in git)
|
||||
const srcVendor = path.join(srcAppDir, 'public', 'vendor');
|
||||
const dstVendor = path.join(dstAppDir, 'public', 'vendor');
|
||||
if (fs.existsSync(srcVendor) && !fs.existsSync(dstVendor)) {
|
||||
console.error('\n--- Copying public/vendor ---');
|
||||
fs.cpSync(srcVendor, dstVendor, { recursive: true });
|
||||
// Copy the vendored assets to the new worktree (built, not in git):
|
||||
// `public/vendor` is fetched by URL at runtime, `vendor` is imported by the
|
||||
// bundler through the @pdfjs / @simplecc aliases.
|
||||
for (const sub of [['public', 'vendor'], ['vendor']]) {
|
||||
const srcVendor = path.join(srcAppDir, ...sub);
|
||||
const dstVendor = path.join(dstAppDir, ...sub);
|
||||
if (fs.existsSync(srcVendor) && !fs.existsSync(dstVendor)) {
|
||||
console.error(`\n--- Copying ${sub.join('/')} ---`);
|
||||
fs.cpSync(srcVendor, dstVendor, { recursive: true });
|
||||
}
|
||||
}
|
||||
|
||||
// Print path to stdout -- allows: cd $(pnpm worktree:new <arg>)
|
||||
|
|
|
|||
|
|
@ -175,9 +175,15 @@ flate2 = "1"
|
|||
# never re-exec itself so it aborted in dyld (#5053), and its crash-handler
|
||||
# `pthread_create` interposer panics on 32-bit ARM, aborting every armeabi-v7a
|
||||
# device (#5070). See `minidump_feature_is_enabled_on_no_target`.
|
||||
#
|
||||
# `debug-images` attaches the loaded libraries and their build ids to each
|
||||
# event. The Android library ships stripped, so its frames carry addresses
|
||||
# only, and this is what lets Sentry resolve them against the debug files CI
|
||||
# uploads.
|
||||
sentry = { version = "0.42", default-features = false, features = [
|
||||
"backtrace",
|
||||
"contexts",
|
||||
"debug-images",
|
||||
"panic",
|
||||
"reqwest",
|
||||
"rustls",
|
||||
|
|
|
|||
|
|
@ -10,6 +10,13 @@ fn main() {
|
|||
if target_os == "windows" {
|
||||
build_windows_thumbnail();
|
||||
}
|
||||
if target_os == "android" {
|
||||
// The APK ships the library stripped (see gen/android/app/build.gradle.kts),
|
||||
// so Sentry symbolicates Rust panics from the debug files CI uploads. It
|
||||
// matches them to the crashing library by build id, and the NDK linker
|
||||
// emits none by default.
|
||||
println!("cargo:rustc-link-arg=-Wl,--build-id=sha1");
|
||||
}
|
||||
|
||||
propagate_sentry_dsn();
|
||||
propagate_app_version();
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
import java.util.Properties
|
||||
import java.io.FileInputStream
|
||||
import org.apache.tools.ant.taskdefs.condition.Os
|
||||
|
||||
plugins {
|
||||
id("com.android.application")
|
||||
|
|
@ -110,6 +111,44 @@ rust {
|
|||
rootDirRel = "../../../"
|
||||
}
|
||||
|
||||
// AGP's own strip step only drops the .debug_* sections, so the Rust library
|
||||
// reached the APK carrying 22MB of symbol tables (.symtab + .strtab) — a
|
||||
// quarter of the download. Strip those too, after AGP has run and on the copy
|
||||
// it packages: `target/` keeps the unstripped library, which is what the
|
||||
// release workflow uploads to Sentry so Rust panics still symbolicate.
|
||||
//
|
||||
// The NDK is located by hand because this module declares no `ndkVersion`, so
|
||||
// `android.ndkDirectory` throws "NDK is not installed". The Tauri CLI sets
|
||||
// NDK_HOME for the cargo linkers; the SDK scan covers a reused Gradle daemon
|
||||
// that started without it in its environment.
|
||||
val llvmStripExecutable = if (Os.isFamily(Os.FAMILY_WINDOWS)) "llvm-strip.exe" else "llvm-strip"
|
||||
|
||||
fun findLlvmStrip(): File {
|
||||
val ndkRoots = listOfNotNull(
|
||||
System.getenv("NDK_HOME"),
|
||||
System.getenv("ANDROID_NDK_HOME"),
|
||||
System.getenv("ANDROID_NDK_ROOT"),
|
||||
).map { File(it) } +
|
||||
File(android.sdkDirectory, "ndk").listFiles().orEmpty().sortedDescending()
|
||||
return ndkRoots
|
||||
.flatMap { File(it, "toolchains/llvm/prebuilt").listFiles().orEmpty().toList() }
|
||||
.map { File(it, "bin/$llvmStripExecutable") }
|
||||
.firstOrNull { it.isFile }
|
||||
?: throw GradleException("$llvmStripExecutable not found; looked in $ndkRoots")
|
||||
}
|
||||
|
||||
tasks.configureEach {
|
||||
if (!name.startsWith("strip") || !name.endsWith("ReleaseDebugSymbols")) return@configureEach
|
||||
doLast {
|
||||
val llvmStrip = findLlvmStrip()
|
||||
outputs.files.asFileTree.matching { include("**/*.so") }.forEach { lib ->
|
||||
providers.exec {
|
||||
commandLine(llvmStrip.absolutePath, "--strip-all", lib.absolutePath)
|
||||
}.result.get().assertNormalExitValue()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
dependencies {
|
||||
implementation("androidx.webkit:webkit:1.14.0")
|
||||
implementation("androidx.appcompat:appcompat:1.7.1")
|
||||
|
|
|
|||
|
|
@ -58,8 +58,12 @@ describe('pdfjs vendor wasm assets', () => {
|
|||
|
||||
it('copies every wasm decoder the bundled pdf.js references', () => {
|
||||
// Files the worker/main bundle are copied from (source of truth for CI).
|
||||
const jsGlob = sourceGlobOf(pkg.scripts['copy-pdfjs-js']!);
|
||||
const { dir: jsDir, files: jsFiles } = resolveCpxGlob(jsGlob);
|
||||
// The worker is fetched by URL so it lives in `public/vendor/pdfjs`, while
|
||||
// the library is imported through the @pdfjs alias and so lives outside
|
||||
// `public/` — scan both, either can carry the wasm references.
|
||||
const jsSources = (['copy-pdfjs-worker', 'copy-pdfjs-lib'] as const).map((script) =>
|
||||
resolveCpxGlob(sourceGlobOf(pkg.scripts[script]!)),
|
||||
);
|
||||
|
||||
// The wasm modules available in pdfjs-dist and what the copy script ships.
|
||||
const wasmGlob = sourceGlobOf(pkg.scripts['copy-pdfjs-wasm']!);
|
||||
|
|
@ -70,12 +74,14 @@ describe('pdfjs vendor wasm assets', () => {
|
|||
// Scan the bundled JS for `*.wasm` references, keeping only ones that map to
|
||||
// a real file (drops minified false positives like `e.wasm`/`t.wasm`).
|
||||
const referenced = new Set<string>();
|
||||
for (const file of jsFiles) {
|
||||
const full = resolve(jsDir, file);
|
||||
if (!existsSync(full)) continue;
|
||||
const text = readFileSync(full, 'utf8');
|
||||
for (const m of text.matchAll(/[A-Za-z0-9_-]+\.wasm/g)) {
|
||||
if (availableWasm.has(m[0])) referenced.add(m[0]);
|
||||
for (const { dir: jsDir, files: jsFiles } of jsSources) {
|
||||
for (const file of jsFiles) {
|
||||
const full = resolve(jsDir, file);
|
||||
if (!existsSync(full)) continue;
|
||||
const text = readFileSync(full, 'utf8');
|
||||
for (const m of text.matchAll(/[A-Za-z0-9_-]+\.wasm/g)) {
|
||||
if (availableWasm.has(m[0])) referenced.add(m[0]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -64,3 +64,31 @@ describe('Proxy request body clone limit', () => {
|
|||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Browser-only database WASM', () => {
|
||||
// The web build stores its database in turso's WASM build; Tauri uses the
|
||||
// native turso crate. Tauri embeds every file in `out/` into the binary, so
|
||||
// the WASM chunk must not be emitted there (a 12.7MB file, 3.3MB in the APK).
|
||||
afterEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
vi.resetModules();
|
||||
});
|
||||
|
||||
const loadConfig = async (platform: string) => {
|
||||
vi.stubEnv('NEXT_PUBLIC_APP_PLATFORM', platform);
|
||||
vi.resetModules();
|
||||
return (await import('../../next.config.mjs')).default;
|
||||
};
|
||||
|
||||
test('the Tauri build stubs the package the web database imports', async () => {
|
||||
const tauri = await loadConfig('tauri');
|
||||
expect(tauri.turbopack?.resolveAlias?.['@readest/turso-database-wasm/webpack']).toBe(
|
||||
'./src/utils/stub.ts',
|
||||
);
|
||||
});
|
||||
|
||||
test('the web build keeps it', async () => {
|
||||
const web = await loadConfig('web');
|
||||
expect(web.turbopack?.resolveAlias?.['@readest/turso-database-wasm/webpack']).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ import { join } from 'path';
|
|||
|
||||
describe.concurrent('jieba-wasm', () => {
|
||||
beforeAll(async () => {
|
||||
const wasmPath = join(process.cwd(), 'public/vendor/jieba/jieba_rs_wasm_bg.wasm');
|
||||
const wasmPath = join(process.cwd(), 'node_modules/jieba-wasm/pkg/web/jieba_rs_wasm_bg.wasm');
|
||||
const wasmBuffer = await readFile(wasmPath);
|
||||
await init({ module_or_path: wasmBuffer });
|
||||
});
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ import { join } from 'path';
|
|||
|
||||
describe.concurrent('suite', () => {
|
||||
beforeAll(async () => {
|
||||
const wasmPath = join(process.cwd(), 'public/vendor/simplecc/simplecc_wasm_bg.wasm');
|
||||
const wasmPath = join(process.cwd(), 'vendor/simplecc/simplecc_wasm_bg.wasm');
|
||||
const wasmBuffer = await readFile(wasmPath);
|
||||
await init({ module_or_path: wasmBuffer });
|
||||
});
|
||||
|
|
|
|||
30
apps/readest-app/src/__tests__/utils/wasm-init-url.test.ts
Normal file
30
apps/readest-app/src/__tests__/utils/wasm-init-url.test.ts
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
import { beforeEach, describe, expect, test, vi } from 'vitest';
|
||||
|
||||
// Both wasm-bindgen glues default to `new URL('<name>_bg.wasm', import.meta.url)`,
|
||||
// which the bundler always emits and Tauri embeds. Passing a `/vendor/...` URL
|
||||
// instead needs a published copy: it either ships the WASM twice (#6368) or,
|
||||
// once the published copy is gone, 404s at runtime.
|
||||
const jiebaInit = vi.fn();
|
||||
const simpleccInit = vi.fn();
|
||||
vi.mock('jieba-wasm', () => ({ default: jiebaInit, cut: vi.fn() }));
|
||||
vi.mock('@simplecc/simplecc_wasm', () => ({ default: simpleccInit, simplecc: vi.fn() }));
|
||||
|
||||
describe('WASM init uses the bundler-emitted file', () => {
|
||||
beforeEach(() => {
|
||||
vi.resetModules();
|
||||
jiebaInit.mockReset();
|
||||
simpleccInit.mockReset();
|
||||
});
|
||||
|
||||
test('jieba', async () => {
|
||||
const { initJieba } = await import('@/utils/jieba');
|
||||
await initJieba();
|
||||
expect(jiebaInit).toHaveBeenCalledWith();
|
||||
});
|
||||
|
||||
test('simplecc', async () => {
|
||||
const { initSimpleCC } = await import('@/utils/simplecc');
|
||||
await initSimpleCC();
|
||||
expect(simpleccInit).toHaveBeenCalledWith();
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
import { describe, expect, test } from 'vitest';
|
||||
import { existsSync, readFileSync } from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
// Tauri embeds every file under `out/` into the app binary, and `public/` is
|
||||
// copied there wholesale. So a module that is ALSO imported through a path
|
||||
// alias ships twice: once bundled into a chunk, once as a published file
|
||||
// nobody fetches. jieba cost 2.7MB that way (#6368).
|
||||
//
|
||||
// The rule: an alias used at build time resolves outside `public/`. Only files
|
||||
// fetched by URL at runtime belong in `public/vendor`.
|
||||
const appDir = path.resolve(import.meta.dirname, '../..');
|
||||
const readJson = (file: string) =>
|
||||
JSON.parse(readFileSync(path.join(appDir, file), 'utf8').replace(/^\s*\/\/.*$/gm, ''));
|
||||
|
||||
describe('vendored build-time modules live outside public/', () => {
|
||||
test('no tsconfig path alias resolves into public/', () => {
|
||||
const paths: Record<string, string[]> = readJson('tsconfig.json').compilerOptions.paths;
|
||||
const inPublic = Object.entries(paths)
|
||||
.filter(([, targets]) => targets.some((t) => t.replace(/^\.\//, '').startsWith('public/')))
|
||||
.map(([alias]) => alias);
|
||||
|
||||
expect(inPublic).toEqual([]);
|
||||
});
|
||||
|
||||
test('pdf.js and simplecc are vendored for the bundler, not published', () => {
|
||||
expect(existsSync(path.join(appDir, 'vendor/pdfjs/pdf.min.mjs'))).toBe(true);
|
||||
expect(existsSync(path.join(appDir, 'public/vendor/pdfjs/pdf.min.mjs'))).toBe(false);
|
||||
expect(existsSync(path.join(appDir, 'vendor/simplecc/simplecc_wasm_bg.wasm'))).toBe(true);
|
||||
expect(existsSync(path.join(appDir, 'public/vendor/simplecc'))).toBe(false);
|
||||
});
|
||||
|
||||
test('what the reader fetches by URL stays published', () => {
|
||||
// foliate-js/pdf.js builds these URLs at runtime, so the bundler never
|
||||
// sees a specifier for them and cannot emit a copy.
|
||||
for (const f of ['pdf.worker.min.mjs', 'openjpeg.wasm', 'text_layer_builder.css']) {
|
||||
expect(existsSync(path.join(appDir, 'public/vendor/pdfjs', f))).toBe(true);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
|
@ -8,7 +8,10 @@ const initJieba = async (): Promise<void> => {
|
|||
if (!initPromise) {
|
||||
initPromise = (async () => {
|
||||
try {
|
||||
await init('/vendor/jieba/jieba_rs_wasm_bg.wasm');
|
||||
// No path: the glue falls back to `new URL('jieba_rs_wasm_bg.wasm',
|
||||
// import.meta.url)`, the copy the bundler already emits. Pointing it at
|
||||
// a public/vendor copy instead shipped the 4MB WASM twice.
|
||||
await init();
|
||||
initialized = true;
|
||||
} catch (e) {
|
||||
initPromise = null;
|
||||
|
|
|
|||
|
|
@ -6,7 +6,10 @@ let initialized = false;
|
|||
const initSimpleCC = async () => {
|
||||
if (initialized) return;
|
||||
|
||||
await init('/vendor/simplecc/simplecc_wasm_bg.wasm');
|
||||
// No path: the glue falls back to `new URL('simplecc_wasm_bg.wasm',
|
||||
// import.meta.url)`, the copy the bundler emits. The public/vendor copy this
|
||||
// used to fetch was the second, embedded-twice copy (#6368).
|
||||
await init();
|
||||
initialized = true;
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -24,8 +24,8 @@
|
|||
"paths": {
|
||||
"@/*": ["./src/*"],
|
||||
"@/components/ui/*": ["./src/components/primitives/*"],
|
||||
"@pdfjs/*": ["./public/vendor/pdfjs/*"],
|
||||
"@simplecc/*": ["./public/vendor/simplecc/*"],
|
||||
"@pdfjs/*": ["./vendor/pdfjs/*"],
|
||||
"@simplecc/*": ["./vendor/simplecc/*"],
|
||||
"js-mdict": ["../../packages/js-mdict/src/index.ts"],
|
||||
"fflate": ["./node_modules/fflate"],
|
||||
"tauri-plugin-turso": ["./src-tauri/plugins/tauri-plugin-turso/guest-js"]
|
||||
|
|
|
|||
|
|
@ -6,8 +6,8 @@ export default defineConfig({
|
|||
plugins: [vinext()],
|
||||
resolve: {
|
||||
alias: {
|
||||
'@pdfjs': path.resolve('public/vendor/pdfjs'),
|
||||
'@simplecc': path.resolve('public/vendor/simplecc'),
|
||||
'@pdfjs': path.resolve('vendor/pdfjs'),
|
||||
'@simplecc': path.resolve('vendor/simplecc'),
|
||||
},
|
||||
},
|
||||
build: {
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ export default defineConfig({
|
|||
// The @pdfjs alias from tsconfig only resolves within the app's own
|
||||
// source files. foliate-js/pdf.js lives outside that scope, so Vite
|
||||
// needs an explicit alias to find the vendored pdfjs build.
|
||||
'@pdfjs': resolve(import.meta.dirname, 'public/vendor/pdfjs'),
|
||||
'@pdfjs': resolve(import.meta.dirname, 'vendor/pdfjs'),
|
||||
},
|
||||
},
|
||||
optimizeDeps: {
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ export default defineConfig({
|
|||
// The @pdfjs alias from tsconfig only resolves within the app's own
|
||||
// source files. foliate-js/pdf.js lives outside that scope, so Vite
|
||||
// needs an explicit alias to find the vendored pdfjs build.
|
||||
'@pdfjs': path.resolve(__dirname, 'public/vendor/pdfjs'),
|
||||
'@pdfjs': path.resolve(__dirname, 'vendor/pdfjs'),
|
||||
// `js-mdict` is consumed via tsconfig paths from `packages/js-mdict/src/`.
|
||||
// Its sources `import 'fflate'` directly — without an alias, vite's
|
||||
// import-analysis walks up from the redirected file location and fails
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ export default defineConfig({
|
|||
},
|
||||
resolve: {
|
||||
alias: {
|
||||
'@pdfjs': path.resolve(__dirname, 'public/vendor/pdfjs'),
|
||||
'@pdfjs': path.resolve(__dirname, 'vendor/pdfjs'),
|
||||
},
|
||||
conditions: ['development'],
|
||||
},
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue