Quick Troubleshooting

Start here if the hit rate never rises

Two rows below have the same root cause — a missing hostIPC: true — and between them they account for most "the cache is not working" reports. Check kubectl get pod <vllm-pod> -o jsonpath='{.spec.hostIPC}' first.

Symptom

Likely cause / fix

KeyError: LcfVllmConnectorV1 at startup

vLLM could not resolve the connector class — check kv_connector_module_path is lcf_vllm_connector.connector in --kv-transfer-config, and that the install step in the pod's command actually succeeded (its output is at the top of the container log).

No buffer space / SDK init fails

Host memlock / ulimit -l too low — ensure it is unlimited for the vLLM pod's node.

SDK connect fails with SHM error: … ENOENT, engine stays on local fallback

The pod is missing hostIPC: true — see Connector Environment and Launch Flags. The SDK reaches LCFC through the host's /dev/shm; without a shared IPC namespace there is nothing to open. If hostIPC is set, check that no volume is mounted at /dev/shm — it hides the host's. If both are right and the error persists, contact Lightbits Support.

Connector loads, no error anywhere, and the cache hit rate never rises

Almost always the same cause as the row above: the SDK fell back to local-only and vLLM served normally. Check kubectl get pod <vllm-pod> -o jsonpath='{.spec.hostIPC}' first.

Cache appears to work, but the tenant you configured is not the one in use

The tenant or logical namespace does not exist, so the session is unbound and LCF routed the KV to namespace 0 / the system tenant instead of rejecting it. kubectl -n lcf logs -l app.kubernetes.io/name=lcfc -c lcfc | grep 'session created' shows UNBOUND instead of (bound). Create the pair — see Connector Environment and Launch Flags and LCF-CLI Deployment; a running engine rebinds on its own once they exist.

Engine log says the connector downgraded cudagraph_mode to PIECEWISE, or decode is unexpectedly slower than vanilla

A connector-forced downgrade is not expected in this release. Check that the connector and pylcf wheels match, and that the serving configuration has not explicitly selected PIECEWISE or eager mode. The connector preserves vLLM's native FULL graphs and holds loads before a replay that would skip layer hooks; compare decode performance under the same model, prompt, output length, and concurrency. See Supported vLLM Versions and Models.

Engine fails at startup with KvLayoutError after enabling speculative decoding

The drafter added a KV layout the connector does not transfer, or the connector and pylcf wheels do not match. Use a matching wheel pair and a supported vLLM version, and check the configuration against the Speculative-Decoding Support Matrix in Supported vLLM Versions and Models. Contact Lightbits Support with the layout named in the error.

Engine fails at startup naming max_model_len and the KV it needs, with LCF_STREAMING_SHIM=1 set

VP streaming was enabled in the environment but cannot run: --max-num-seqs is not 1, the model has sliding-window layers, or streaming_enabled / virtual_paging_enabled are missing from kv_connector_extra_config. Fix the cause, or lower --max-model-len to fit GPU memory (see The Fetch → Install → Run Flow).

Connector loads but no warm-turn cache hits

--disable-hybrid-kv-cache-manager passed on a hybrid model — the engine log then warns refusing external KV reuse to avoid corrupting un-checkpointed GDN state; remove the flag (see Supported vLLM Versions and Models). Otherwise a stale/mismatched wheel pair — the pylcf and lcf-vllm-connector versions must match.

Connector works but no metrics reach the dashboard

The connector pushes plaintext OTLP on port 24317, not the mTLS 4317 the daemons use, and, if lcf-observability.networkPolicy.otlpFromCidrs is set, the collector's NetworkPolicy admits only those CIDRs and the namespaces listed in lcf-observability.networkPolicy.otlpPlaintextFromNamespaces (default vllm). A vLLM pod outside them, or a multi-node rank on a node outside them, is denied with nothing in any log. If Inferra is not installed in the namespace lcf, also set LCF_TELEMETRY__OTLP_ENDPOINT (see The Fetch → Install → Run Flow). See Observability.

Wheels won't install

Check Cloudsmith index URL / token, or that the version x.y.z exists in lightbits/lightbits-inferra.

Still stuck?

Contact Lightbits support with the Inferra version you are running, the model being served, and the relevant pod logs. The per-release known issues are in the Release Notes.