Testing Qwen on MacBook Pro

I've been testing a local LLM for documents and queries I don't want sent to a third-party API, especially when "we don't train on your data" isn't enough.

Setup: MacBook Pro with M5 Pro, 48GB RAM, LM Studio, Qwen 3.5 9B.

It's surprisingly capable for summaries, drafts and straightforward Q&A. The 9B model also supports a very large context window, although in practice its ability to reliably reason over long documents and handle multi-step tasks is weaker than larger models. It's still nowhere near the current free models from frontier labs in overall quality, and speed varies a lot depending on workload and configuration.

That's not really the point, though.

For work that needs to stay on the device, a capable local model is useful enough to be practical. It's also interesting to experiment with how these models actually behave under the hood. #OpenCode and Continue make it more useful directly from VSCode as well.

Next experiment: Qwen 3.8 27B. It should give better output, and a 4-bit version can fit within 48GB, although large context windows, other applications and runtime overhead could create enough memory pressure to cause swapping.

Not a daily driver yet, but a useful option for sensitive work where keeping documents off third-party inference APIs and Cloud matters.