Skip to content
Deploy

GPU servers · LLM hosting

LLM hosting: run a self-hosted LLM on a GPU server

Price list updated

LLM hosting on a RyzenRDP GPU server means running an open-weight language model on your own Linux server and serving it through an OpenAI-compatible API. One NVIDIA RTX PRO 6000 Blackwell SE gives the model 96 GB of VRAM, and plans go up to 8 GPUs with 768 GB in total. You install the runtime yourself, usually Ollama or vLLM, on a server in Frankfurt, Germany that is billed monthly.

  • 96 GB to 768 GB of VRAM
  • OpenAI-compatible API with Ollama or vLLM
  • Root access over SSH

How much VRAM does an LLM need?

An LLM needs enough VRAM for its weights, its context cache and the runtime's own overhead. The weights take roughly the parameter count times the bytes stored per parameter: half a byte at 4-bit, one byte at 8-bit, two bytes at 16-bit.

Estimated size of model weights by parameter count and precision
Model sizeWeights at 4-bitWeights at 8-bitWeights at 16-bitSmallest plan that holds the 16-bit weights
8 billion parametersabout 4 GBabout 8 GBabout 16 GB1 × NVIDIA RTX PRO 6000 Blackwell SE (96 GB)
32 billion parametersabout 16 GBabout 32 GBabout 64 GB1 × NVIDIA RTX PRO 6000 Blackwell SE (96 GB)
70 billion parametersabout 35 GBabout 70 GBabout 140 GB2 × NVIDIA RTX PRO 6000 Blackwell SE (192 GB)
120 billion parametersabout 60 GBabout 120 GBabout 240 GB4 × NVIDIA RTX PRO 6000 Blackwell SE (384 GB)
400 billion parametersabout 200 GBabout 400 GBabout 800 GBNone: run it at 8-bit on 8 × NVIDIA RTX PRO 6000 Blackwell SE (768 GB)

This is arithmetic on the weights, not a measurement, and the weights are not the whole bill:

  • Context cache. Every token a request holds in context takes cache memory, per request. Long contexts and many simultaneous users can add tens of gigabytes, so do not plan to fill a GPU with weights alone.
  • Quantized files run a little larger than the bare arithmetic, because some layers are kept at higher precision.
  • Rule of thumb: pick the plan whose total VRAM is comfortably above the weights at the precision you want, then measure with your own context length and load before you depend on it.

Which runtime: Ollama or vLLM?

Ollama

One install command and one command to pull and run a model. The simplest way to a working endpoint for yourself or a small team.

vLLM

A serving engine that batches many simultaneous requests and splits a model across several GPUs. The usual choice for an API that an application depends on.

Open WebUI

A chat interface in the browser for your team, connected to Ollama or to any OpenAI-compatible endpoint.

Both runtimes expose an OpenAI-compatible API, so client code written for a hosted model usually needs only a new base URL and model name. By default Ollama processes one request per model at a time and queues the rest; vLLM is built to batch requests from many users. The guide vLLM vs Ollama compares them, and how to self-host an LLM walks through the setup step by step.

From a new server to a first answer

  1. Order a plan by VRAM

    Use the sizing table, then pick the server in the plan table.

  2. Sign in and check the GPUs

    Connect with ssh root@YOUR_SERVER_IP and run nvidia-smi. Each GPU of the plan is listed with its memory.

  3. Install a runtime

    For Ollama: curl -fsSL https://ollama.com/install.sh | sh. For vLLM, follow its quickstart, which installs it into a Python environment.

  4. Load a model

    With Ollama, ollama run MODEL downloads the model and opens a chat. With vLLM, vllm serve MODEL downloads it and starts the API on port 8000.

  5. Point your application at it

    Set your client's base URL to the server's endpoint, such as http://127.0.0.1:8000/v1 for vLLM through an SSH tunnel, and send the same chat requests you would send to a hosted API.

Do not put a model API on the open internet

A model endpoint that anyone can reach is an endpoint anyone can use at your cost. Ollama listens on 127.0.0.1 by default and has no login of its own, so keep it that way and reach it through an SSH tunnel, a VPN, or a reverse proxy that adds TLS and authentication. Start vLLM with --api-key so it rejects requests without your key, and still put TLS in front of it. Open only the ports you need in the server's firewall.

Running one model on several GPUs

A model that does not fit on one GPU can be split across the GPUs of a larger server. vLLM does this with the --tensor-parallel-size option, set to the number of GPUs. Ollama loads a model on a single GPU when it fits there and spreads it across the available GPUs when it does not.

More GPUs are also how you serve more at once: several different models side by side, or several copies of one model behind a load balancer.

Fine-tuning needs more memory than inference of the same model, because training keeps gradients and optimizer state next to the weights. LoRA and similar methods train small adapter matrices and keep the base weights frozen, which lowers the requirement, so adapting a mid-size model is realistic on one or two GPUs.

LLM server plans

Every plan has NVIDIA RTX PRO 6000 Blackwell SE GPUs with 96 GB of VRAM each, a 10 Gbps port and 20 TB of outbound transfer a month. The disk holds your model files: the single-GPU plan has 16 vCPU, 128 GB RAM, 500 GB disk and a 10 Gbps port.

GPU servers for LLM hosting in Frankfurt, Germany, USD per month
GPUsVRAM per GPUTotal VRAMvCPURAMDiskMonthlyStock and order
1 × NVIDIA RTX PRO 6000 Blackwell SE 96 GB 96 GB 16 128 GB 500 GB $1,951.09
2 × NVIDIA RTX PRO 6000 Blackwell SE 96 GB 192 GB 32 256 GB 1,000 GB $3,900.60
4 × NVIDIA RTX PRO 6000 Blackwell SE 96 GB 384 GB 64 512 GB 2,000 GB $7,799.63
8 × NVIDIA RTX PRO 6000 Blackwell SE 96 GB 768 GB 128 1,024 GB 4,000 GB $15,597.68

Stock follows free GPU capacity and is shown live: a row marked Out of stock cannot be ordered until capacity returns. If the plan you need is out of stock, or you are not sure which plan you need, ask the sales team: tell us the plan and how many servers you need, or the model and the number of users you expect.

LLM hosting FAQ

What is LLM hosting?

LLM hosting is running a large language model on a server and exposing it through an API so applications can send it prompts. With a self-hosted LLM the server is yours: you choose the model, and the prompts are processed on your server instead of a model provider's.

Can I host my own LLM on a RyzenRDP server?

Yes, on a GPU server. Plans have 1, 2, 4 or 8 NVIDIA RTX PRO 6000 Blackwell SE GPUs with 96 GB of VRAM each, run Ubuntu 24.04 LTS with NVIDIA CUDA, and give you root access over SSH to install Ollama, vLLM or another runtime.

How much VRAM do I need for a 70B model?

About 35 GB for the weights at 4-bit, about 70 GB at 8-bit and about 140 GB at 16-bit, plus headroom for the context cache. The 4-bit and 8-bit versions fit on one 96 GB GPU; the 16-bit version needs the two-GPU server with 192 GB.

Should I use Ollama or vLLM?

Use Ollama when you want the simplest setup for yourself or a few users. Use vLLM when an application or many users will call the model at once, or when the model has to be split across several GPUs. Both expose an OpenAI-compatible API. See vLLM vs Ollama.

Is the API compatible with OpenAI client libraries?

Yes, with both common runtimes. vLLM serves an OpenAI-compatible API, by default on port 8000, and Ollama offers OpenAI-compatible endpoints on its own port. In most client libraries you change the base URL and the model name.

Do you install the model or the runtime for me?

No. The server runs Ubuntu 24.04 LTS with NVIDIA CUDA and gives you root access. You install the runtime and download the model yourself; the apps page has the commands.

Which models can I run?

Any open-weight model whose license allows your use and whose weights fit in the plan's VRAM. You download models yourself, for example from Hugging Face or the Ollama library, and you are responsible for following each model's license.

Can I run an LLM without a GPU?

Small quantized models run on a CPU, slowly. If your application calls a hosted model through an API, it needs no GPU at all: an AI VPS on CPU is enough. A GPU server is for running the model yourself at useful speed.

How is LLM hosting billed?

By the month, at the price of the GPU server: from $1,951.09 a month. There is no charge per token. The server costs the same whether it is busy or idle.

Host your own LLM

96 GB to 768 GB of VRAM on NVIDIA RTX PRO 6000 Blackwell SE GPUs in Frankfurt, Germany. From $1,951.09 a month.