OpenAI-compatible
Drop-in for any OpenAI SDK code
AI & MLFrom $99/app/month
Managed Ollama Hosting
Run large language models locally
What is Ollama on ManageStacks?
Ollama on ManageStacks is the open-source LLM runtime deployed to a GPU instance in your own AWS, Azure, or GCP region — priced flat starting at $99 per instance per month (GPU + platform), with CUDA drivers, model persistence, an OpenAI-compatible API, and unlimited inference calls. Run Llama, Mistral, Qwen, Gemma, DeepSeek, or any GGUF model without per-token pricing, and prompts + responses never leave your cloud region. Ideal for regulated industries, high-volume inference, or teams that want OpenAI's API shape without OpenAI's data-sharing policy.
Ollama on ManageStacks is the open-source LLM runtime deployed to a GPU instance in your own AWS, Azure, or GCP region — priced flat starting at $99 per instance per month (GPU + platform), with CUDA drivers, model persistence, an OpenAI-compatible API, and unlimited inference calls. Run Llama, Mistral, Qwen, Gemma, DeepSeek, or any GGUF model without per-token pricing, and prompts + responses never leave your cloud region. Ideal for regulated industries, high-volume inference, or teams that want OpenAI's API shape without OpenAI's data-sharing policy.
Last updated Official siteSource on GitHubModel library
About Ollama
What Ollama does, and why teams deploy it.
Ollama makes it easy to run large language models on your own infrastructure. It bundles model weights, quantization, tokenization, and runtime into a single binary and exposes an OpenAI-compatible REST API — meaning any application built for OpenAI's API works against Ollama with a single base-URL change.
Models supported include Llama 3.x, Mistral, Qwen 2.5, Gemma 2, DeepSeek, Phi, Codestral, and any GGUF-format model from Hugging Face. Multi-model serving from one instance is native — swap between a chat model and a code model based on the request.
Self-hosting Ollama at production quality means running the Ollama binary on a GPU-enabled host, installing the right CUDA driver version for your GPU generation (H100, L40S, A10G, RTX 4090 each have preferences), sizing GPU memory correctly for the model you want (a 70B model in Q4 quantization needs ~40 GB VRAM), keeping downloaded model weights on persistent storage (they're often 4-40 GB each), and integrating with monitoring for token throughput and GPU utilisation. ManageStacks handles all of that.
DIY vs ManageStacks
What running Ollama yourself looks like — and what it looks like with us.
DIY self-hosting
- Provision a GPU instance; install correct CUDA driver for the GPU generation
- Install Ollama binary and configure model storage on persistent disk
- Download 4-40 GB model weights repeatedly on ephemeral instances
- Wire OpenAI-SDK-based apps to your endpoint; test tool-calling compatibility
- Monitor GPU utilisation, VRAM pressure, tokens/sec via your own tooling
On ManageStacks
- Subscribe through your AWS, Azure, or GCP marketplace
- Ollama comes up on GPU infrastructure with CUDA + model storage ready
- OpenAI-compatible endpoint available immediately; drop your `base_url` in
- Grafana dashboards ship for tokens/sec, GPU util, VRAM, latency percentiles
- Pair with Open WebUI (chat UI) or LiteLLM (multi-model router) on the same account
Ollama on ManageStacks — key numbers
Unlimited tokens
Flat monthly price — no per-token metering
L4 → H100
GPU tiers scale with model size
Multi-model
Hot-swap between chat, code, embed models
Key features
Everything Ollama ships with, running on our stack.
- OpenAI-compatible REST API — drop-in for any OpenAI SDK code
- Run Llama 3.x, Mistral, Qwen 2.5, Gemma 2, DeepSeek, Phi, Codestral
- Any GGUF model from Hugging Face via `ollama pull`
- GPU acceleration with CUDA + optimised memory management
- Multi-model serving from one instance — hot-swap models per request
- Modelfile for custom model configuration + system-prompt embedding
- Streaming responses via SSE for real-time output
- Persistent model storage — downloaded weights survive restarts
- Prometheus-exporter metrics for tokens/sec, GPU util, VRAM usage
- Bring-your-own fine-tuned models via LoRA adapters
How it deploys
From subscribe to live in minutes.
Subscribe
Subscribe to ManageStacks through your AWS, Azure, or GCP marketplace. Pick a GPU tier appropriate for your target model size.
Provision
Ollama spins up with CUDA drivers, persistent model storage, and Grafana monitoring — typically 5-10 minutes.
Pull models
`ollama pull llama3.3`, `ollama pull qwen2.5-coder`, or any GGUF from Hugging Face. Weights persist across restarts.
Integrate
Point your OpenAI-SDK code at the Ollama endpoint by setting `base_url`. Add Open WebUI or LiteLLM for a UI or multi-model routing.
Who this is for
Built for teams that want Ollama to just work.
High-volume LLM API consumers
You're spending $5-50k+/month on OpenAI or Anthropic and per-token pricing is unpredictable. Self-hosted Llama 3.3 70B on ManageStacks is flat-priced and covers 80% of production use cases at 10-100x cheaper.
Regulated industries
Healthcare, financial, government — data can't leave your cloud region. Ollama on ManageStacks in your VPC is the only option that satisfies both compliance and modern-LLM capability.
Product teams building AI features
You want to iterate on prompts + models without a per-request meter running. Flat-priced inference lets you test, measure, and ship without cost anxiety.
Compliance & compatibility
What we handle, what Ollama runs on.
Compliance & operations
- TLS-encrypted inference API + persistent model storage encrypted at rest
- Prompts + responses stay in your cloud region — no third-party data flow
- GDPR + HIPAA data-residency — deployment in your chosen cloud region
- GPU driver security patches applied during your maintenance window
- Optional API-key authentication + rate limiting via LiteLLM proxy
- Model weights auditable — SHA-256 hashes provided per model version
Compatibility
- Version
- Latest Ollama stable (validated before release)
- Runtime
- Ollama binary on GPU-enabled containerised infrastructure
- Dependencies
- CUDA drivers, persistent storage for model weights
- Min. resources
- 1 GPU (L4 or better) / 4 vCPU / 16 GB RAM (standard tier)
How ManageStacks helps
We handle the parts you shouldn't be writing yourself.
ManageStacks deploys Ollama on GPU-enabled infrastructure (L4/L40S/H100 depending on plan) with CUDA drivers, persistent model storage, an OpenAI-compatible endpoint, and Prometheus metrics. We handle GPU driver management, model caching, throughput monitoring, and integration with Open WebUI or LiteLLM for the full self-hosted LLM stack.
How it compares
Ollama on ManageStacks vs the alternatives.
How Ollama on ManageStacks compares to the two dominant hosted-LLM APIs and running Ollama yourself.
| Property | Ollama on ManageStacksUs | OpenAI API | Anthropic API | Self-hosted GPU + Ollama |
|---|---|---|---|---|
| Deployment | Managed GPU on your AWS, Azure, or GCP | Vendor-hosted (multi-region) | Vendor-hosted (multi-region) | You provision + operate |
| Data residency | Your cloud region | Vendor infrastructure | Vendor infrastructure | Your cloud region |
| Pricing basis | Flat per instance + GPU | Per input/output token | Per input/output token | Your GPU compute cost |
| Model choice | Any GGUF (Llama, Qwen, etc.) | GPT-4o, o1, o3 (closed) | Claude Sonnet, Opus, Haiku (closed) | Any GGUF |
| Open source | Yes (MIT + open weights) | No (proprietary) | No (proprietary) | Yes |
| Unlimited inference | Yes | No (metered) | No (metered) | Yes |
Comparison focuses on architectural properties (deployment model, pricing basis, open-source status) that don't change with vendor pricing pages. Verify current pricing on each vendor's own site.
FAQ
Common questions about Ollama on ManageStacks.
How does this compare to OpenAI or Anthropic APIs?
OpenAI, Anthropic, and Google are priced per input+output token, which grows with usage. Ollama on ManageStacks is a flat instance-plus-GPU price ($99/mo standard). For high-volume inference (embeddings pipelines, chatbot backends serving 100k+ requests/day, agentic workflows with many LLM calls per user action), self-hosted Ollama is often 10-100x cheaper. Frontier-model quality is still with OpenAI/Anthropic/Google — Llama 3.3 70B and Qwen 2.5 72B are competitive for many tasks but not all.
Which models are worth running on ManageStacks Ollama?
Depends on your GPU: L40S/A10G (48 GB) runs Llama 3.3 70B Q4, Qwen 2.5 72B Q4, DeepSeek V3 (Q3), most Mistral variants; H100 (80 GB) runs any of those unquantized. Smaller instances (RTX 4090 24 GB) run Llama 3.1 8B, Qwen 2.5 32B Q4, Gemma 2 27B Q4. For coding, Codestral 22B and Qwen 2.5 Coder 32B are excellent; for embedding, nomic-embed-text is the default.
How do I add new models?
`ollama pull <model>` fetches from the Ollama library. `ollama create` with a Modelfile builds custom models with baked-in system prompts. Or download GGUF weights from Hugging Face and register them directly. All downloaded weights persist across restarts on ManageStacks — no re-downloading multi-GB files after every deploy.
Can I use Ollama as a drop-in OpenAI replacement?
Yes. Ollama exposes `/v1/chat/completions`, `/v1/embeddings`, and `/v1/completions` endpoints matching OpenAI's API shape. Point any OpenAI SDK (Python, TypeScript, LangChain, LlamaIndex) at your Ollama endpoint by setting `base_url` — no code changes beyond that. Note: OpenAI-specific features (tool-calling variations, JSON mode) work on models that support them (Llama 3.1+, Qwen 2.5+ do).
What GPU instance types does ManageStacks support?
AWS: g5 (A10G), g6 (L4/L40S), p4/p5 (A100/H100). Azure: NC/ND series. GCP: G2 (L4), A2 (A100), A3 (H100). Standard tier includes a moderate GPU (L4 or A10G class) suitable for models up to 32B parameters at Q4 quantization. Business/Enterprise for H100/L40S for 70B+ models unquantized.
How is fine-tuning handled?
Ollama doesn't do fine-tuning itself — but it serves LoRA adapters produced by Axolotl, Unsloth, or Hugging Face's TRL. Fine-tune elsewhere (or on a separate ManageStacks GPU deployment), then load the LoRA adapter into Ollama for inference. Custom system prompts embed via Modelfile without fine-tuning.
Can I put a UI in front of Ollama?
Yes. Open WebUI (also on ManageStacks) is the standard chat interface for Ollama. Deploy both together for a private ChatGPT-like experience. LiteLLM (also on ManageStacks) gives you routing between Ollama and cloud LLMs for multi-model apps.
What happens to my prompts and data?
Everything stays in your cloud region. Prompts, responses, embeddings, and downloaded model weights all live on infrastructure you own. No data sent to Ollama Inc. or any third party. Ideal for regulated industries (healthcare, finance, government), IP-sensitive use cases, or organisations with strict data-residency requirements.
Deploy Ollama in under 5 minutes.
Subscribe through your AWS, Azure, or GCP marketplace. We handle provisioning, SSL, monitoring, backups, updates, and security. From $99/app/month.