Running Gemma locally via Ollama for command-line and API access
The second path to local AI is Ollama. Install it, then run `ollama pull gemma4` followed by `ollama run gemma4:E4B`. You now have Gemma running locally from the command line. Ollama also exposes a local API port — port 11434 — so you can connect your own app or script directly to it. If you want to push further, you can try the 12B, 26B, or 31B versions later, assuming your hardware can handle it. You can ask an LLM whether your hardware is up to the task, or just try it yourself and suffer through the slowness — both are valid options.
Go deeper with Greg Isenberg
Ask anything about this, and Greg Isenberg answers from everything they've shared.
