You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
16 lines
811 B
16 lines
811 B
##
|
|
## Per-instance fixed host-loopback port for the smol135 model's API server.
|
|
##
|
|
## Fed to vLLM's `--port` flag (see vllm-model@.container: `Exec=... --host
|
|
## 127.0.0.1 --port ${COOKBOOK_VLLM_MODEL_PORT}`) and reused by the unit's
|
|
## HealthCmd. Referenced by llmsnap's per-model `proxy: http://127.0.0.1:5802`
|
|
## in config.yaml (llmsnap runs Network=host and reaches the model here). Each
|
|
## model gets its OWN port so two models can be resident at once (GPU sleep
|
|
## mode). Keep it in sync with config.yaml.
|
|
##
|
|
## NOT `VLLM_PORT`: that variable is for vLLM's INTERNAL use (ZMQ / distributed
|
|
## executor) and does NOT set the API server port — reusing it as the API port
|
|
## collides with vLLM's internal usage. See
|
|
## https://docs.vllm.ai/en/stable/configuration/env_vars/
|
|
##
|
|
COOKBOOK_VLLM_MODEL_PORT=5802
|
|
|