Prove It's Actually Using LCF
The simplest way to show the connector is live is a cold-then-warm test against the vLLM OpenAI-compatible API.
The idea
Four steps. The whole point is the gap between step 2 and step 3.
Expose the vLLM service — for example port-forward it to localhost:8000 — so you can send it requests.
Send a long prompt (tens to hundreds of thousands of tokens) once. Nothing is cached yet, so vLLM must prefill the whole prompt — note the time-to-first-token (TTFT). Stream the response so you can measure TTFT separately from decode time.
Send the identical prompt again. The KV cache for that prompt is now resident on the LCF fabric, so prefill is skipped and TTFT drops sharply.
A large, repeatable warm TTFT ≪ cold TTFT gap is the proof the connector served the KV from LCF instead of recomputing it.
Make it a definitive proof: clear the GPU prefix cache
For a stronger result, clear vLLM's local GPU prefix cache between the cold and warm turns — otherwise a warm hit could be served from the GPU and tell you nothing about LCF. vLLM exposes this as an HTTP call:
/reset_prefix_cache is served only when the engine was started with VLLM_SERVER_DEV_MODE=1. There is no LCF deployment template on a released build, so add it to the pod's env as shown in Connector Environment and Launch Flags and restart before running this test.
With the GPU cache cleared, any warm-turn speed-up can only come from KV reloaded over the LCF fabric, not from the GPU — that is the definitive proof the connector is serving cache.
Resetting the prefix cache empties vLLM's GPU-side cache only; the KV stays in the Inferra fabric, which is the point. If you also want to force the read past the node-local RAM tier and down to the storage tier — useful when establishing both figures — run lcf-cli node clear-ram-tier between the turns as well.
You can drive this with any HTTP/benchmark client you already use (a short streaming script, or a standard LLM load-testing tool). Lightbits Support can provide a ready-made cold/warm benchmark helper on request.
For the cluster-side view of the same test — what the dashboard should show, and the three causes of a flat hit rate — see Post-Deployment Validation.