In a test summarizing documents with Gemma 4 E4B, turning thinking off boosted speed five-fold and result quality was about the same. For Gemma 4 12B, the speed improvement was eight-fold.
How thinking affects coding will depend on the complexity of the task. You may want to test some real-world work to see whether time spent “thinking” pays off with better results.
Running models outside the Ollama catalog
One of the largest sources of open-source LLMs is Hugging Face, and many models there have a “use this model” button that gives the command you need to download and run it on Ollama. Just be aware that not all of those models are as plug-and-play as the Ollama-optimized versions on Ollama.com, especially new ones.
For example, I tried running the Unsloth project’s 2-bit quantized version of Meta’s new Muse Glimmer. The command was
ollama run hf.co/unsloth/Muse-Glimmer-30B-GGUF:UD-Q2_K_XL
However, this errored out of the box. ChatGPT helped me fix the issue (with a Modelfile to create a modified version of the LLM) after I told it the error messages.
Hugging Face model pages often show the command needed to run the LLM on Ollama.
Foundry
The bottom line
Local LLMs have come a long way, but a model that can fit inside a consumer desktop or laptop won’t be able to do the kind of work that top cloud-based models can. Gemma 4 12B may write good basic functions and document summaries, but don’t expect it to do complicated jobs from a single prompt like “create an R package to use the National Weather Service API.” As with frontier LLMs of old, you’ll need to go step by step, be specific, and have some patience. If you never want to go back to those days, stay with the state-of-the-art models in the cloud.
Gemma works well for basic data analysis, “importing, tidying, and visualizing data,” and “information retrieval and short factual questions,” according to Posit AI’s documentation. “It is less reliable than the Claude models on longer, multi-step tasks. You might see a strong response on one turn and a misread on the next. We do not recommend Gemma 4 for long-running agentic coding or complex package development.”
That was pretty much the status of paid cloud-based LLMs a couple of years ago.
Even if limited compared to current state-of-the-art models, though, free coding help that’s 100% local and private can still be compelling for use cases like exploring sensitive data, summarizing personal documents, doing simple tasks, working on a plane with bad Internet, and saving paid tokens for when they matter. Plus it can feel empowering to have everything on your own system.
And, local model capabilities will likely keep getting better.
