Ollama
One install command and one command to pull and run a model. The simplest way to a working endpoint for yourself or a small team.
GPU servers · LLM hosting
Price list updated
LLM hosting on a RyzenRDP GPU server means running an open-weight language model on your own Linux server and serving it through an OpenAI-compatible API. One NVIDIA RTX PRO 6000 Blackwell SE gives the model 96 GB of VRAM, and plans go up to 8 GPUs with 768 GB in total. You install the runtime yourself, usually Ollama or vLLM, on a server in Frankfurt, Germany that is billed monthly.
An LLM needs enough VRAM for its weights, its context cache and the runtime's own overhead. The weights take roughly the parameter count times the bytes stored per parameter: half a byte at 4-bit, one byte at 8-bit, two bytes at 16-bit.
| Model size | Weights at 4-bit | Weights at 8-bit | Weights at 16-bit | Smallest plan that holds the 16-bit weights |
|---|---|---|---|---|
| 8 billion parameters | about 4 GB | about 8 GB | about 16 GB | 1 × NVIDIA RTX PRO 6000 Blackwell SE (96 GB) |
| 32 billion parameters | about 16 GB | about 32 GB | about 64 GB | 1 × NVIDIA RTX PRO 6000 Blackwell SE (96 GB) |
| 70 billion parameters | about 35 GB | about 70 GB | about 140 GB | 2 × NVIDIA RTX PRO 6000 Blackwell SE (192 GB) |
| 120 billion parameters | about 60 GB | about 120 GB | about 240 GB | 4 × NVIDIA RTX PRO 6000 Blackwell SE (384 GB) |
| 400 billion parameters | about 200 GB | about 400 GB | about 800 GB | None: run it at 8-bit on 8 × NVIDIA RTX PRO 6000 Blackwell SE (768 GB) |
This is arithmetic on the weights, not a measurement, and the weights are not the whole bill:
One install command and one command to pull and run a model. The simplest way to a working endpoint for yourself or a small team.
A serving engine that batches many simultaneous requests and splits a model across several GPUs. The usual choice for an API that an application depends on.
A chat interface in the browser for your team, connected to Ollama or to any OpenAI-compatible endpoint.
Both runtimes expose an OpenAI-compatible API, so client code written for a hosted model usually needs only a new base URL and model name. By default Ollama processes one request per model at a time and queues the rest; vLLM is built to batch requests from many users. The guide vLLM vs Ollama compares them, and how to self-host an LLM walks through the setup step by step.
Use the sizing table, then pick the server in the plan table.
Connect with ssh root@YOUR_SERVER_IP and run nvidia-smi. Each GPU of the plan is listed with its memory.
For Ollama: curl -fsSL https://ollama.com/install.sh | sh. For vLLM, follow its quickstart, which installs it into a Python environment.
With Ollama, ollama run MODEL downloads the model and opens a chat. With vLLM, vllm serve MODEL downloads it and starts the API on port 8000.
Set your client's base URL to the server's endpoint, such as http://127.0.0.1:8000/v1 for vLLM through an SSH tunnel, and send the same chat requests you would send to a hosted API.
A model endpoint that anyone can reach is an endpoint anyone can use at your cost. Ollama listens on
127.0.0.1 by default and has no login of its own, so keep it that way and reach it through an SSH
tunnel, a VPN, or a reverse proxy that adds TLS and authentication. Start vLLM with --api-key so it
rejects requests without your key, and still put TLS in front of it. Open only the ports you need in the
server's firewall.
A model that does not fit on one GPU can be split across the GPUs of a larger server. vLLM does this
with the --tensor-parallel-size option, set to the number of GPUs. Ollama loads a model on a single
GPU when it fits there and spreads it across the available GPUs when it does not.
More GPUs are also how you serve more at once: several different models side by side, or several copies of one model behind a load balancer.
Fine-tuning needs more memory than inference of the same model, because training keeps gradients and optimizer state next to the weights. LoRA and similar methods train small adapter matrices and keep the base weights frozen, which lowers the requirement, so adapting a mid-size model is realistic on one or two GPUs.
Every plan has NVIDIA RTX PRO 6000 Blackwell SE GPUs with 96 GB of VRAM each, a 10 Gbps port and 20 TB of outbound transfer a month. The disk holds your model files: the single-GPU plan has 16 vCPU, 128 GB RAM, 500 GB disk and a 10 Gbps port.
| GPUs | VRAM per GPU | Total VRAM | vCPU | RAM | Disk | Monthly | Stock and order |
|---|---|---|---|---|---|---|---|
| 1 × NVIDIA RTX PRO 6000 Blackwell SE | 96 GB | 96 GB | 16 | 128 GB | 500 GB | $1,951.09 | |
| 2 × NVIDIA RTX PRO 6000 Blackwell SE | 96 GB | 192 GB | 32 | 256 GB | 1,000 GB | $3,900.60 | |
| 4 × NVIDIA RTX PRO 6000 Blackwell SE | 96 GB | 384 GB | 64 | 512 GB | 2,000 GB | $7,799.63 | |
| 8 × NVIDIA RTX PRO 6000 Blackwell SE | 96 GB | 768 GB | 128 | 1,024 GB | 4,000 GB | $15,597.68 |
Stock follows free GPU capacity and is shown live: a row marked Out of stock cannot be ordered until capacity returns. If the plan you need is out of stock, or you are not sure which plan you need, ask the sales team: tell us the plan and how many servers you need, or the model and the number of users you expect.
LLM hosting is running a large language model on a server and exposing it through an API so applications can send it prompts. With a self-hosted LLM the server is yours: you choose the model, and the prompts are processed on your server instead of a model provider's.
Yes, on a GPU server. Plans have 1, 2, 4 or 8 NVIDIA RTX PRO 6000 Blackwell SE GPUs with 96 GB of VRAM each, run Ubuntu 24.04 LTS with NVIDIA CUDA, and give you root access over SSH to install Ollama, vLLM or another runtime.
About 35 GB for the weights at 4-bit, about 70 GB at 8-bit and about 140 GB at 16-bit, plus headroom for the context cache. The 4-bit and 8-bit versions fit on one 96 GB GPU; the 16-bit version needs the two-GPU server with 192 GB.
Use Ollama when you want the simplest setup for yourself or a few users. Use vLLM when an application or many users will call the model at once, or when the model has to be split across several GPUs. Both expose an OpenAI-compatible API. See vLLM vs Ollama.
Yes, with both common runtimes. vLLM serves an OpenAI-compatible API, by default on port 8000, and Ollama offers OpenAI-compatible endpoints on its own port. In most client libraries you change the base URL and the model name.
No. The server runs Ubuntu 24.04 LTS with NVIDIA CUDA and gives you root access. You install the runtime and download the model yourself; the apps page has the commands.
Any open-weight model whose license allows your use and whose weights fit in the plan's VRAM. You download models yourself, for example from Hugging Face or the Ollama library, and you are responsible for following each model's license.
Small quantized models run on a CPU, slowly. If your application calls a hosted model through an API, it needs no GPU at all: an AI VPS on CPU is enough. A GPU server is for running the model yourself at useful speed.
By the month, at the price of the GPU server: from $1,951.09 a month. There is no charge per token. The server costs the same whether it is busy or idle.
96 GB to 768 GB of VRAM on NVIDIA RTX PRO 6000 Blackwell SE GPUs in Frankfurt, Germany. From $1,951.09 a month.