mirror of
https://github.com/Helldez/BigMoeOnEdge.git
synced 2026-10-19 15:33:46 +00:00
88c6c46236
Every probe that touches a device does so by using it, and a device is only as sound as its driver. The planner is meant to be handed a binary that carries every backend and to find out for itself what the machine has, which only works if finding out cannot end the program. It could: on a phone from 2020 a build carrying the Vulkan backend loaded it, listed the GPU, and died at the first allocation, in a function that driver does not have and the backend calls without checking. Each device beyond the host is now asked in a second process first: this same executable started again with the device's name, which runs every probe that touches that device and prints its facts as one line. A line that comes back is applied whole or not at all. A child that dies or does not answer in two minutes marks the device unusable, with the reason. Either way the probes in this process leave that device alone. An unusable device is weighed by nothing and kept out of the run: the capacity fitter is not asked about it, no layer is placed on it, and the host carries the run unless another device was armed for the prefill. The plan prints it as UNUSABLE with what it did. Starting a second process is the front-end's business, so the probe takes a runner. bmoe-cli gives it one (--probe-device is the child's mode, --no-isolated-probes turns it off). Where a platform has no way to start one the runner is empty and the probes run in-process as before: Windows today, and the desktop server. Also: a build that carries no accelerator backend no longer reports "no device" as a measured fact about the machine in the prefill line. The same phone and build: the plan completes in 23 s, reports the GPU unusable (signal 11) and the run decodes at 1.87 and 2.02 tok/s on the cores. On a laptop whose device works the plan is unchanged and the second process costs about 3.5 s. An iOS harness (examples/ios, scripts/build-ios.sh). An app on iOS may not start a process, so examples/ios compiles cli/main.cpp into an app and calls it on a thread. On iOS the planner reads the process's own memory share and probes devices in-process. CI compiles the harness for the simulator. Not run on a device.
81 lines
3 KiB
Bash
Executable file
81 lines
3 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# Configure and build the iOS harness (examples/ios). Needs Xcode, not only the command line tools.
|
|
#
|
|
# scripts/build-ios.sh the simulator, unsigned
|
|
# scripts/build-ios.sh device a device, signed by BMOE_IOS_TEAM
|
|
#
|
|
# Env overrides:
|
|
# BUILD_DIR (default build-ios-sim / build-ios-device)
|
|
# BUILD_TYPE (Release)
|
|
# BMOE_IOS_TEAM Apple development team id; required for a device
|
|
# BMOE_IOS_BUNDLE_ID (io.bigmoeonedge.harness) must be unique to the team on a device
|
|
# BMOE_IOS_MEMORY_ENTITLEMENTS=ON ask for the larger memory limit and address space
|
|
# IOS_MIN_VERSION (16.4, the oldest llama.cpp builds for)
|
|
set -euo pipefail
|
|
|
|
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
TARGET="${1:-sim}"
|
|
BUILD_TYPE="${BUILD_TYPE:-Release}"
|
|
IOS_MIN_VERSION="${IOS_MIN_VERSION:-16.4}"
|
|
|
|
cd "$ROOT"
|
|
if [ ! -f third_party/llama.cpp/CMakeLists.txt ]; then
|
|
echo "llama.cpp submodule missing — run: git submodule update --init --recursive"
|
|
exit 1
|
|
fi
|
|
if ! xcrun --sdk iphoneos --show-sdk-path >/dev/null 2>&1; then
|
|
echo "no iOS SDK: install Xcode, then: sudo xcode-select -s /Applications/Xcode.app"
|
|
exit 1
|
|
fi
|
|
|
|
case "$TARGET" in
|
|
sim)
|
|
BUILD_DIR="${BUILD_DIR:-build-ios-sim}"
|
|
SYSROOT=iphonesimulator
|
|
# The architecture of this machine: the simulator runs the host's own instruction set.
|
|
ARCHS="$(uname -m)"
|
|
SIGN=(-DCMAKE_XCODE_ATTRIBUTE_CODE_SIGNING_ALLOWED=NO)
|
|
;;
|
|
device)
|
|
BUILD_DIR="${BUILD_DIR:-build-ios-device}"
|
|
SYSROOT=iphoneos
|
|
ARCHS=arm64
|
|
if [ -z "${BMOE_IOS_TEAM:-}" ]; then
|
|
echo "a device build is signed: set BMOE_IOS_TEAM to your Apple development team id"
|
|
exit 1
|
|
fi
|
|
SIGN=(-DBMOE_IOS_TEAM="$BMOE_IOS_TEAM")
|
|
;;
|
|
*)
|
|
echo "usage: $0 [sim|device]"
|
|
exit 1
|
|
;;
|
|
esac
|
|
|
|
# Static libraries, so the app is one executable with nothing to embed and sign beside it; the
|
|
# Metal shaders compiled in for the same reason. The rest is what the host builds already say.
|
|
cmake -S . -B "$BUILD_DIR" -G Xcode \
|
|
-DCMAKE_SYSTEM_NAME=iOS \
|
|
-DCMAKE_OSX_SYSROOT="$SYSROOT" \
|
|
-DCMAKE_OSX_ARCHITECTURES="$ARCHS" \
|
|
-DCMAKE_OSX_DEPLOYMENT_TARGET="$IOS_MIN_VERSION" \
|
|
-DBUILD_SHARED_LIBS=OFF \
|
|
-DBMOE_BUILD_TESTS=OFF \
|
|
-DGGML_NATIVE=OFF \
|
|
-DGGML_OPENMP=OFF \
|
|
-DGGML_METAL_EMBED_LIBRARY=ON \
|
|
-DBMOE_IOS_BUNDLE_ID="${BMOE_IOS_BUNDLE_ID:-io.bigmoeonedge.harness}" \
|
|
-DBMOE_IOS_MEMORY_ENTITLEMENTS="${BMOE_IOS_MEMORY_ENTITLEMENTS:-OFF}" \
|
|
"${SIGN[@]}"
|
|
if [ "$TARGET" = device ]; then
|
|
cmake --build "$BUILD_DIR" --config "$BUILD_TYPE" --target bmoe-harness -- -allowProvisioningUpdates
|
|
else
|
|
cmake --build "$BUILD_DIR" --config "$BUILD_TYPE" --target bmoe-harness
|
|
fi
|
|
|
|
APP="$(find "$BUILD_DIR" -maxdepth 4 -name bmoe-harness.app -type d | head -n 1)"
|
|
echo "built: $APP"
|
|
if [ "$TARGET" = sim ]; then
|
|
echo "run: xcrun simctl boot <device>; xcrun simctl install booted \"$APP\""
|
|
echo " see examples/ios/README.md"
|
|
fi
|