Apple

Apple VP of AI Retiring

Apple’s Senior VP of Machine Learning and AI Strategy, John Giannandrea, is retiring, and being replaced with Amar Subramanya who recently worked at Microsoft, but has been with Google for 16 years and worked on the Gemini assistant.

Apple is in a weird place with AI. They recognized early on the value of machine learning, and their strategy has been to bake it into the products to produce valuable features, without really identifying it as AI.

Dramatic improvement in Time to First Token on M5 iPad

Apple’s unified memory architecture is great for LLMS (and starting to show up in the PC world), but Apple had an achilles heel: the time it took to generate the first output token with a long prompt.

This was due to the lack of an accelerated matmul instruction, and that was allegedly addressed in the M5. I say allegedly, because the software support to expose wasn’t there until now.