Phi-3: a language model small enough for a phone

Microsoft’s Phi-3-mini is interesting because of where it can run. The 3.8-billion-parameter model is small enough to deploy on a phone. In the researchers’ tests, it performed competitively with much larger models on several benchmarks. That is a benchmark comparison, rather than a guarantee of equivalent performance on every task. Technical report.

Local inference gives developers more options. Some applications could work offline and keep prompts on the device. They could also reduce dependence on a hosted service, although memory, battery use and response time still matter.

The training approach is worth attention too. Microsoft emphasises the quality and selection of training data. A smaller model that does a defined job well can be more useful than a larger one that is impractical to deploy.

For me, the next question is which tasks work well enough on actual devices. That is where the promise of a pocket-sized model becomes a product decision.