/network
You pick the model. We pick the Macs.
The scheduler places each request on hardware that can serve it well — one Mac when it fits, several when it doesn't.
[01]
Two ways to run
Single Mac
One eligible Mac serves the whole model when it can comfortably handle it.
MODEL ▮▮▮▮▮▮▮▮
Split model
Layers are spread across several Macs when one doesn't have the memory.
L1–26
L27–52
L53–80
Note: splitting gives access to bigger models. It doesn't automatically make them faster.
[02]
What the scheduler weighs
01
Memory
02
Model fit
03
Measured speed
04
Network quality
05
Current load
06
Reliability
Starting scope: Apple Silicon Macs and a small catalog of supported models.