mirror of
https://github.com/Helldez/BigMoeOnEdge.git
synced 2026-10-21 04:43:39 +00:00
5bf70c8b08
* refactor(plan): hold the dense set on every platform, and read what a reclaim costs from the kernel The dense policy left the dense weights mapped wherever exceeding the memory limit ends the process, because clean file pages are not charged to that limit. They are not charged because the kernel may drop them, and every decoded token reads the whole dense set: on a 12 GB iOS phone the 35B-A3B decoded at 1.2 tok/s with 25581 major faults a token, where the same model held in memory makes 5 on the Android test phone (#304). That branch is removed: a fatal limit widens the margin the ledger sizes against, and the dense set is held like anywhere else. The facts behind the policy were constants per OS on Apple and Windows. They are now read from the kernel (pio::reclaim_facts: the per-process share left, compressions since boot, a swap or pagefile), a syscall each, so the probe derives Kill / Compress / Swap with no platform branch of its own. The Apple-only buffers_locked constant goes too, since unknown is charged the same. HardwareProfile::file_pages_counted is removed. * fix(moe): the dense residency sensor reads each tensor's own address, on every platform Under mmap/warm the sensor found the model's mapping through the process's region list (/proc/self/maps, a walk of the address space on macOS). An iOS app is not given that list and Windows has none, so dense_resident_frac read -1 exactly where #304 found the dense set refaulted every token. llama.cpp loads memory-mapped, so before anything is rebound a dense tensor's data is its address in the mapping: the sensor, the drop of the mapped copies after an anon read and the random-access advice on held-back tables now look addresses up there. Measured in the iOS simulator: 0.85 under mmap, 1.00 under ahwb, where the same build read -1 before. Windows gains the instruments it was missing: the residency sample is QueryWorkingSetEx and the CPU time GetProcessTimes, so the compute line is printed there. No per-process hard-fault count is published, so pio::major_faults_counted() says so and the summary's majflt_tok is -1 rather than a 0 that reads as no faults. * build(ios): ask for the memory entitlements one at a time, and run a device report in one command BMOE_IOS_MEMORY_ENTITLEMENTS asked for the increased memory limit and the extended address space together; a free Personal Team is refused the second, so the build did not sign, while the first alone was enough for a streamed 35B on a 12 GB phone (#235). It now asks for the memory limit only, and BMOE_IOS_EXTENDED_ADDRESSING adds the address space for a team that has it. scripts/ios-runs.sh runs the plan, a run of it and the same run with the dense set left mapped, each in a fresh process of the harness, and brings back the console output and per-token CSV of each with a summary (tok/s, decode split, major faults, dense residency). device drives a signed harness through devicectl, sim the booted simulator. The harness README and docs/platforms.md say what has run on a device. |
||
|---|---|---|
| .. | ||
| CMakeLists.txt | ||
| main.cpp | ||