Macos

Detailed post of Apple M-series chips with LLM performance.

The M5 is listed in one of the comments as 153Gb/s. I don’t think this takes into account the improvement in time-to-first-token from the improved matmul that they added to the A19.

The Best Local LLMs To Run On Every Mac

Good research, and a good list of models, sorted by the amount of memory you need to run them. A bit dated for the very large models with so many good ones being released in the latter part of this year, but for people with average an average Mac, it’s still relevant.