The tool Simon Willison highlights is valuable in its simplicity — it bridges the gap between technical jargon and user comprehension. But it's a niche solution for a niche audience. Token speed matters, but it's just one piece of the LLM puzzle; latency, accuracy, and context retention are equally critical and harder to visualize. The post highlights a recurring challenge in AI: making complex metrics accessible without oversimplifying. The next step would be tools that contextualize multiple performance factors, not just speed.
While this simulation is a neat demo, it's a reminder that AI adoption hinges on better user education. Vendors touting token speeds often assume users understand the implications — they don't. The industry needs more efforts like this, but broader in scope.