mirror of
https://github.com/Helldez/BigMoeOnEdge.git
synced 2026-10-20 02:33:01 +00:00
88c6c46236
Every probe that touches a device does so by using it, and a device is only as sound as its driver. The planner is meant to be handed a binary that carries every backend and to find out for itself what the machine has, which only works if finding out cannot end the program. It could: on a phone from 2020 a build carrying the Vulkan backend loaded it, listed the GPU, and died at the first allocation, in a function that driver does not have and the backend calls without checking. Each device beyond the host is now asked in a second process first: this same executable started again with the device's name, which runs every probe that touches that device and prints its facts as one line. A line that comes back is applied whole or not at all. A child that dies or does not answer in two minutes marks the device unusable, with the reason. Either way the probes in this process leave that device alone. An unusable device is weighed by nothing and kept out of the run: the capacity fitter is not asked about it, no layer is placed on it, and the host carries the run unless another device was armed for the prefill. The plan prints it as UNUSABLE with what it did. Starting a second process is the front-end's business, so the probe takes a runner. bmoe-cli gives it one (--probe-device is the child's mode, --no-isolated-probes turns it off). Where a platform has no way to start one the runner is empty and the probes run in-process as before: Windows today, and the desktop server. Also: a build that carries no accelerator backend no longer reports "no device" as a measured fact about the machine in the prefill line. The same phone and build: the plan completes in 23 s, reports the GPU unusable (signal 11) and the run decodes at 1.87 and 2.02 tok/s on the cores. On a laptop whose device works the plan is unchanged and the second process costs about 3.5 s. An iOS harness (examples/ios, scripts/build-ios.sh). An app on iOS may not start a process, so examples/ios compiles cli/main.cpp into an app and calls it on a thread. On iOS the planner reads the process's own memory share and probes devices in-process. CI compiles the harness for the simulator. Not run on a device. |
||
|---|---|---|
| .. | ||
| android | ||
| ios | ||