An API for your product
vLLM behind a reverse proxy, with an API key.
GPU servers · Apps
Updated
A RyzenRDP GPU server runs Ubuntu 24.04 LTS with NVIDIA CUDA and gives you root access over SSH. You install the AI software you want on it yourself. The most common choices are Ollama and vLLM for language models, Open WebUI for a chat interface, ComfyUI for image and video workflows, and PyTorch with JupyterLab for your own code. This page lists what each one does, the command that installs it, and the port it listens on.
We do not preinstall these apps or install them for you. Each one installs with a few commands, shown
below, and installing it yourself means you choose the version and know exactly what runs on your
server. Start every new server with nvidia-smi, which lists the GPUs and their memory.
Seven tools cover most of what people run on a GPU server. Each row links to the project's own documentation, which is the source to follow when a command changes.
| App | What it does | Listens on | Documentation |
|---|---|---|---|
| Ollama | Downloads and runs language models with one command; offers an OpenAI-compatible API | 127.0.0.1:11434 | Ollama on Linux |
| vLLM | Serving engine for language models that batches many requests and can split a model across GPUs; OpenAI-compatible API | localhost:8000 | vLLM quickstart |
| Open WebUI | Browser chat interface with user accounts, for Ollama or any OpenAI-compatible endpoint | Port 3000 with the documented Docker command | Open WebUI quick start |
| ComfyUI | Node-based interface and API for image, video and audio generation workflows | 127.0.0.1:8188 | ComfyUI repository |
| TextGen | Interface for running and chatting with local language models, formerly known as text-generation-webui | 127.0.0.1:7860 | TextGen repository |
| JupyterLab | Notebooks in the browser for experiments, data work and training code | localhost:8888 | JupyterLab installation |
| PyTorch | The deep learning framework most models and training scripts are written in | No port: a Python library | PyTorch: start locally |
Ollama is the shortest path from a new server to a model that answers. Install it with the script from its documentation, then pull and run a model:
curl -fsSL https://ollama.com/install.sh | sh
ollama run MODEL_NAME
The installer sets Ollama up as a systemd service that listens on 127.0.0.1:11434. ollama ps shows
whether a model is loaded on the GPU. To let another program on the same server use it, point that
program at http://127.0.0.1:11434. The OLLAMA_HOST variable changes the bind address, but Ollama has
no sign-in of its own, so do not expose it to the internet. More on
LLM hosting.
vLLM is a serving engine for an API that many users or an application call at once. Its quickstart
installs it into a Python environment with uv and starts an OpenAI-compatible server:
uv venv --python 3.12 --seed
source .venv/bin/activate
uv pip install vllm --torch-backend=auto
vllm serve MODEL_ID --api-key YOUR_SECRET_KEY
The server starts on http://localhost:8000. --api-key makes it reject requests that do not carry
your key, and --tensor-parallel-size set to the number of GPUs splits one model across them. See
vLLM vs Ollama for when to choose which.
Open WebUI gives a team a chat interface in the browser, with accounts and conversation history, in front of Ollama or any OpenAI-compatible endpoint. It runs as a Docker container; install Docker first, then use the command from the Open WebUI quick start:
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway \
-v open-webui:/app/backend/data -e WEBUI_SECRET_KEY=your-secret-key \
--name open-webui --restart always ghcr.io/open-webui/open-webui:main
Replace your-secret-key with the output of openssl rand -hex 32. The interface is then on port 3000
of the server. The first account you create becomes the administrator, so create it before you let
anyone else reach the port. A private AI server is usually this:
a model runtime with Open WebUI in front.
git clone, a Python environment and
python main.py. The ComfyUI server page has the full steps and the SSH
tunnel to reach it../start_linux.sh from its
repository, which sets up its own environment on the first run.pip install jupyterlab and starts with jupyter lab. It prints a
link with an access token; open it through an SSH tunnel.pip install torch inside a virtual environment. Confirm it sees the
GPUs with python -c "import torch; print(torch.cuda.is_available(), torch.cuda.device_count())".Most of these tools listen on 127.0.0.1 by default. That is the safe setting on a server with a public IP address: keep it.
ssh -L 8188:127.0.0.1:8188 root@YOUR_SERVER_IP forwards a port to your own computer, where you open http://127.0.0.1:8188. Change the number to the app's port.
To share an interface, put a web server such as Caddy or nginx in front of it with a domain name, TLS and sign-in, or connect the team through a VPN such as WireGuard.
Allow SSH and only the ports you actually publish in the server's firewall, for example with ufw.
vLLM behind a reverse proxy, with an API key.
Ollama or vLLM with Open WebUI in front.
ComfyUI, reached through an SSH tunnel.
The NVIDIA RTX PRO 6000 Blackwell SE with 96 GB of VRAM.
No. The server runs Ubuntu 24.04 LTS with NVIDIA CUDA, with no AI apps installed. You install the apps you want over SSH; the commands for the common ones are on this page.
No. Each app installs with a few commands from its own documentation. That takes minutes and leaves you in control of the version.
Run nvidia-smi after you sign in. It lists each GPU of the server with its memory and current use. Inside Python, torch.cuda.is_available() confirms that PyTorch sees the GPUs.
Yes. You have root access, so you can install Docker and run containers. To give a container access to the GPUs, install the NVIDIA Container Toolkit and start the container with --gpus all.
Ollama for the simplest setup and a few users, vLLM for an API under real load or a model split across several GPUs. Both offer an OpenAI-compatible API. See LLM hosting.
No. GPU servers run Linux only. For a Windows desktop over Remote Desktop, see our Windows VPS plans, which have no GPU.
1, 2, 4 or 8 NVIDIA RTX PRO 6000 Blackwell SE GPUs with 96 GB of VRAM each, in Frankfurt, Germany. From $1,951.09 a month.