Table of Contents

🚀 Docker GPU Ollama + Codex Client: Full Architecture Guide

This guide shows how to deploy Ollama on a GPU machine (via Docker) and use netsh portproxy on Windows to forward localhost:11434 to the remote Ollama, so that Codex (on Windows) calls the remote GPU as if it were a local service. It fills in the architecture setup, fixes common typos, and walks through the usual gotchas, so you can paste or publish it directly.

1. Server (GPU machine) — Building Ollama with Docker Compose

I recommend managing the Ollama container with docker compose. Example:

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ./ollama_models:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - capabilities: [gpu]
    runtime: nvidia
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
    healthcheck:
      test: ["CMD", "ollama", "list"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 30s

Key points:

  • Don’t mangle the volume path into a typo like ./ollama_models:/root/.ollamaentrypoint script; the correct value is ./ollama_models:/root/.ollama, which stores models and data.
  • If you use the NVIDIA Container Toolkit, runtime: nvidia and NVIDIA_VISIBLE_DEVICES make the GPU visible to the container.
  • The healthcheck helps Docker determine whether the service is ready.

Start it:

docker compose up -d

Once it’s up, confirm that Ollama can run a model on the GPU (using gemma4:e4b as an example):

ollama run gemma4:e4b

Or list the available models:

curl http://localhost:11434/v1/models

Remember to open port 11434 (or whichever port you configured) in the server firewall.

2. Client (Windows) — Codex Configuration

On Windows, configure Codex to call the local endpoint (localhost), then use portproxy to forward the local connection to the remote GPU machine. Example Codex config:

model = "gemma4:e4b"
model_provider = "ollama"
base_url = "http://localhost:11434"

sandbox = "elevated"

Important concepts:

  • Codex only connects to localhost: this avoids the Ollama adapter’s fallback bug, prevents the remote IP from being ignored, and keeps Codex’s behavior predictable.
  • Windows forwards the local connection to the remote Ollama (via portproxy), transparently to Codex.

3. Client (Windows) — Aider Configuration

The same portproxy architecture works for aider too. Just point aider at localhost:11434, and portproxy will automatically forward to the remote GPU.

Via the command line:

aider --model ollama/gemma4:e4b --openai-api-base http://localhost:11434/v1

Or create a .aider.conf.yml config file in your project directory:

model: ollama/gemma4:e4b
openai-api-base: http://localhost:11434/v1

Key points:

  • aider talks to Ollama through an OpenAI-compatible interface, so --openai-api-base needs the /v1 path.
  • The model name follows the ollama/<model_name> format, which tells aider to use the Ollama provider.
  • Just like Codex, aider only connects to localhost, and portproxy handles the forwarding — no remote configuration changes required.

4. Windows portproxy (the critical step)

On Windows, open PowerShell as Administrator and add a portproxy rule:

netsh interface portproxy add v4tov4 `
  listenaddress=127.0.0.1 `
  listenport=11434 `
  connectaddress=192.168.15.235 `
  connectport=11434

Replace 192.168.15.235 with your GPU server’s IP.

Verify:

netsh interface portproxy show all

You should see something like:

Listen on IPv4:             Connect to IPv4:
Address         Port        Address         Port
--------------- ----------  --------------- ----------
127.0.0.1       11434       192.168.15.235  11434

Then test from the Windows machine itself:

curl http://localhost:11434/v1/models

If it returns the model list, the forwarding works and the Ollama API is responding correctly.

5. Common Issues and Troubleshooting (don’t skip this)

  1. Codex can’t connect to localhost, or there’s no response

    • Check whether the Windows IP Helper service is enabled (portproxy depends on iphlpsvc): 
      Get-Service iphlpsvc
    • Confirm the firewall isn’t blocking local loopback (usually it isn’t), and that no other service is occupying 11434.

  2. portproxy isn’t taking effect

    • Confirm the rule was added: netsh interface portproxy show all
    • If you hit IPv6 issues, you may need to add a v6tov4 rule as well, or make sure the application binds to IPv4.

  3. The GPU server isn’t responding, or Ollama won’t start

    • Confirm the container can see the GPU (check nvidia-smi both on the host and inside the container).
    • If you need Ollama to bind to 0.0.0.0 (externally accessible), use: 
      OLLAMA_HOST=0.0.0.0 ollama serve
    • Check the Docker logs: docker logs ollama

  4. Model fails to load, or out of memory

    • Use a smaller model, or confirm you have enough GPU memory (VRAM); if necessary, use swap, or spread load across multiple smaller models.

6. Full Architecture Diagram

flowchart TD
  A[Codex on Windows] -->|"localhost:11434"| B[Windows portproxy]
  D[Aider on Windows] -->|"localhost:11434"| B
  B -->|"192.168.15.235:11434"| C["GPU Server: Ollama in Docker"]

7. The Core Value (what you want readers to take away)

This workflow solves three real problems:

  • A GPU server is easy to deploy reliably on Linux (containerized with Docker).
  • Windows clients (Codex) often only want to connect to localhost; portproxy lets you use a remote GPU without modifying Codex.
  • It avoids standing up an extra reverse proxy or jump host, reducing both latency and operational complexity.

References

Ask this article

Answers come from this article only. Click any prompt below or open the chat at the bottom right.

🇺🇸 English

Picture this: you've got a beefy GPU box sitting in your house or your lab, and you want your Windows machine to use it for local AI coding — running models through tools like Codex or Aider. The problem? Those clients really, really want to talk to localhost. They get weird and unreliable when you point them at a remote IP. So how do you bridge that gap without spinning up a whole reverse proxy or a jump host? The answer is a clever little Windows trick called portproxy. Let me walk you through the whole setup.

Let's start on the server side — the GPU machine. The cleanest way to run Ollama here is with Docker Compose. You define a single service: pull the official Ollama image, give it a container name, and map port 11434 to the host. Then you mount a local folder into the container so your downloaded models and data persist across restarts — that's important, otherwise you're re-downloading gigabytes every time. The magic for GPU access comes from two settings: you set the runtime to NVIDIA and tell it to make all GPU devices visible. That's what lets the container actually reach the graphics card. And finally, a healthcheck that periodically runs "ollama list" so Docker knows when the service is truly ready.

One quick gotcha worth calling out: be careful with that volume path. It should map your models folder to the dot-ollama directory inside the container, nothing more. It's easy to accidentally smush extra text onto the end of that path and break everything.

Once you bring the stack up in the background, confirm it works. Run a model — something like a small Gemma variant — and make sure it actually loads on the GPU. Or hit the models endpoint with curl to list what's available. And don't forget the obvious: open port 11434 in your server's firewall, or none of this matters.

Now to the Windows side. Here's where the philosophy clicks into place. In your Codex config, you tell it the model name, the provider is Ollama, and — this is the key — the base URL points at localhost, port 11434. Not the GPU machine's IP. Localhost. Why? Because pointing these clients at a remote address triggers fallback bugs in the Ollama adapter; the remote IP gets quietly ignored, behavior becomes unpredictable. By keeping Codex pointed at localhost, everything stays clean and deterministic. The remote-ness is somebody else's job.

And the exact same idea works for Aider. You point Aider at localhost as well, with one small difference: Aider talks to Ollama through an OpenAI-compatible interface, so its base URL needs the "/v1" path on the end. The model name uses the "ollama-slash-modelname" format, which signals to Aider that it's using the Ollama provider. Either pass these on the command line or drop them into a project config file. No remote configuration anywhere.

So if both clients are talking to localhost, but Ollama lives on a different machine — what's doing the actual forwarding? That's the critical step: Windows portproxy. You open PowerShell as Administrator and add a single rule. In plain terms, you're telling Windows: anything that arrives on the loopback address at port 11434, quietly forward it to the GPU server's IP at port 11434. You swap in your own server's address, of course. Verify the rule landed by listing all portproxy entries — you'll see a little table showing the local listen address mapped to the remote connect address. Then the real test: from the Windows box itself, curl localhost and ask for the models list. If you get models back, the tunnel is live and Ollama is answering. That's the whole trick — Windows transparently relays local traffic to the remote GPU, and Codex never knows the difference.

Now, a few things that'll bite you. First, if Codex can't connect or just hangs — check that the Windows IP Helper service is running. Portproxy completely depends on it; if that service is stopped, your rules do nothing. Also make sure nothing else is squatting on port 11434.

Second, if the rule itself doesn't seem to take effect, re-list your portproxy entries to confirm it's actually there. And watch out for IPv6 — sometimes you need an additional v6-to-v4 rule, or you make sure the app binds to IPv4 specifically.

Third, if the GPU server won't respond, verify the container can actually see the card by running nvidia-smi both on the host and inside the container. If they disagree, your GPU passthrough is broken. You can also force Ollama to bind to all interfaces with the OLLAMA_HOST setting, and always check the Docker logs when something's off.

And fourth, the classic one: out of memory. If a model won't load, it's usually VRAM. Drop to a smaller model, or spread your load across a couple of lighter ones.

So let me leave you with the three things that make this approach genuinely worth it. One: your GPU server lives happily on Linux in a Docker container, which is the reliable, reproducible way to run it. Two: portproxy lets your Windows clients keep talking to localhost — the way they want to — while transparently reaching a remote GPU, with zero changes to Codex or Aider themselves. And three: you skip the entire reverse proxy and jump host detour, which means less latency and far less operational complexity. One netsh rule, and your remote GPU feels local. That's the elegance of it.

🇹🇼 中文

這篇要聊的是怎麼把 GPU 跑在一台機器上，然後讓 Windows 上的 Codex 完全感覺不出來——它以為自己在呼叫本機服務，其實背後是一台遠端的 GPU 在幹活。整個關鍵，就是 Windows 內建的 portproxy，不用額外架反向代理，也不用跳板機。

先講 GPU 那台機器，也就是 server 端。建議用 Docker Compose 來管理 Ollama。設定上有幾個重點：第一，要把模型資料的目錄掛載出來，讓模型不會每次重啟就消失，這裡有個常見的手滑，路徑後面別不小心黏到多餘的字，乾乾淨淨對到 Ollama 的資料夾就好。第二，如果你裝了 NVIDIA Container Toolkit，記得在設定裡指定 nvidia runtime，並且把可見的 GPU 設成全部，這樣 container 裡才看得到顯卡。第三，加個 healthcheck，讓 Docker 知道服務到底起來了沒。

跑起來之後，你可以實際 run 一個模型確認 GPU 有在動，或者打一下 models 這個 API 看看模型清單回不回得來。最後一步很容易忘——server 的防火牆要把 11434 這個 port 開出來，不然外面連不進去。

接著是 Windows 客戶端，也就是 Codex 這邊。設定的精神很單純：Codex 的 base url 一律指向 localhost，也就是本機的 11434。為什麼一定要連 localhost？因為這樣可以避開 Ollama adapter 那個 fallback 的 bug，也避免它把你填的遠端 IP 忽略掉，整體行為更可預期。至於連線怎麼飛到遠端 GPU，那是 portproxy 的事，對 Codex 來說完全透明。

順帶一提，同一套架構也適用 Aider。一樣讓它指向 localhost 的 11434，差別只在 Aider 走的是 OpenAI 相容介面，所以 base url 後面要加上 v1 這個路徑，模型名稱也要用 ollama 斜線開頭的格式，告訴它走 Ollama provider。遠端那邊一行都不用改。

好，重頭戲來了——portproxy 本身。你在 Windows 上用管理員權限開 PowerShell，下一條 netsh 的指令，新增一條 v4 對 v4 的轉發規則。意思就是：監聽本機的 127.0.0.1 的 11434，把進來的連線轉到 GPU server 的 IP 的 11434。文章裡用的範例 IP 是 192.168.15.235，你換成自己那台就好。

設完之後用 show all 確認規則有掛上，你會看到一條 listen 在本機 11434、connect 到遠端 11434 的對應。然後在本機打一下 models 的 API，如果模型清單回得來，恭喜，轉發成功，Ollama 也活著。

實務上會踩到的坑，大概這幾類。第一，Codex 連 localhost 沒反應，先檢查 Windows 的 IP Helper 服務有沒有開，因為 portproxy 就是靠它,服務沒跑規則就是死的；順便確認 11434 沒被別的程式占走。第二，規則明明加了卻沒生效，多半是 IPv6 在搗亂，必要時補一條 v6 對 v4 的規則，或確保應用程式是綁在 IPv4 上。第三，GPU 那邊沒回應，先在主機跟 container 裡各跑一次 nvidia-smi，確認顯卡真的看得到；如果要讓 Ollama 對外開放，可以把它綁在 0.0.0.0，再不行就翻 docker logs。第四，模型載不起來通常是 VRAM 不夠，那就換小一點的模型，或者拆成幾個小模型分流。

整條鏈路串起來看就是：Windows 上的 Codex 跟 Aider 都打 localhost 的 11434，連到本機的 portproxy，portproxy 再把流量導到遠端那台跑在 Docker 裡的 Ollama。

最後收個尾，這套做法其實在解三個很真實的問題。第一，GPU server 放在 Linux 上、用 Docker 化部署，最穩、最好維護。第二，Windows 的客戶端通常只認 localhost，靠 portproxy 你完全不用動 Codex 的設定，就能借用遠端 GPU。第三，省掉了額外架反向代理或跳板機的麻煩，延遲更低，運維也更簡單。記住這三點，這套架構你就拿得走了。

Tags

#ollama#codex#aider#docker#gpu#windows#portproxy

Related Articles