Local Ollama URL incorrect and no adjustment. #10

BloodBlight · 2024-12-07T23:04:29Z

When attempting to use a local instance of olamma, I get this error:

Failed to generate step after 3 attempts. **Error: 404 Client Error: Not Found** for url: http://192.168.10.50:11434/api/chat

However manually verifying that the service is up and running works:

curl http://192.168.10.50:11434/api/generate -d '{
  "model": "llama3.1:latest",
  "prompt": "Why is the sky blue?"
}'
{"model":"llama3.1:latest","created_at":"2024-12-07T22:59:12.977900524Z","response":"The","done":false}
{"model":"llama3.1:latest","created_at":"2024-12-07T22:59:12.988189036Z","response":" sky","done":false}
{"model":"llama3.1:latest","created_at":"2024-12-07T22:59:12.998730522Z","response":" appears","done":false}
{"model":"llama3.1:latest","created_at":"2024-12-07T22:59:13.009296528Z","response":" blue","done":false}
{"model":"llama3.1:latest","created_at":"2024-12-07T22:59:13.019687252Z","response":" to","done":false}

It APPEARS that "/api/chat" is not the correct URL and without digging into the source, there doesn't appear to be a way to change it.

I have a fairly generic ollama install, only oddity is the use of ROCM as I have an AMD GPU:

version: '3.8'

services:
  ollama:
    image: ollama/ollama:rocm
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ./Ollama/data:/root/.ollama
    devices:
      - /dev/kfd
      - /dev/dri
    shm_size: '16gb'
    group_add:
      - video
    cap_add:
      - SYS_PTRACE
    security_opt:
      - seccomp=unconfined
    restart: always


  ollama-webui:
    build:
      context: .
      args:
        OLLAMA_API_BASE_URL: '/ollama/api'
      dockerfile: Dockerfile
    image: ghcr.io/ollama-webui/ollama-webui:main
    container_name: ollama-webui
    volumes:
      - ./Ollama/webui:/app/backend/data
    depends_on:
      - ollama
    ports:
      - 8081:8080
    environment:
      - 'OLLAMA_API_BASE_URL=http://ollama:11434/api'
      - 'WEBUI_SECRET_KEY='
    extra_hosts:
      - host.docker.internal:host-gateway
    restart: always

EDIT: Updated test to actually use IP rather than localhost just to confirm. No change.

The text was updated successfully, but these errors were encountered:

BloodBlight · 2024-12-08T21:16:32Z

More from the Ollama logs:

Working request from the Ollama UI:

llm_load_print_meta: EOG token        = 128008 '<|eom_id|>'
llm_load_print_meta: EOG token        = 128009 '<|eot_id|>'
llm_load_print_meta: max token length = 256
llama_model_load: vocab only - skipping tensors
[GIN] 2024/12/08 - 21:08:16 | 200 | 13.403022737s |      172.21.0.3 | POST     "/api/chat"
[GIN] 2024/12/08 - 21:08:55 | 200 |  6.479498014s |      172.21.0.3 | POST     "/api/chat"
[GIN] 2024/12/08 - 21:08:55 | 200 |  138.817915ms |      172.21.0.3 | POST     "/api/generate"

Logs from Streamlit UI:

[GIN] 2024/12/08 - 21:14:17 | 404 |     182.912µs |   192.168.10.50 | POST     "/api/chat"
[GIN] 2024/12/08 - 21:14:18 | 404 |     162.111µs |   192.168.10.50 | POST     "/api/chat"
[GIN] 2024/12/08 - 21:14:19 | 404 |     121.001µs |   192.168.10.50 | POST     "/api/chat"
[GIN] 2024/12/08 - 21:14:19 | 404 |     268.412µs |   192.168.10.50 | POST     "/api/chat"
[GIN] 2024/12/08 - 21:14:20 | 404 |     122.731µs |   192.168.10.50 | POST     "/api/chat"
[GIN] 2024/12/08 - 21:14:21 | 404 |     127.201µs |   192.168.10.50 | POST     "/api/chat"
[GIN] 2024/12/08 - 21:14:21 | 404 |     385.263µs |   192.168.10.50 | POST     "/api/chat"

BloodBlight · 2024-12-13T06:32:49Z

This issue was caused by not having the model already downloaded in ollamma.

This should probably have a check if possible, or maybe include some extra detail in the error? Maybe "check your selected model is available"?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Local Ollama URL incorrect and no adjustment. #10

Local Ollama URL incorrect and no adjustment. #10

Local Ollama URL incorrect and no adjustment. #10

Local Ollama URL incorrect and no adjustment. #10

Comments