Use cases¶
Real serving systems and workloads written as serQ programs. Each page puts the source or the traffic next to the program that describes it, says what the program checks, and says what it leaves out.
| Page | What it shows |
|---|---|
| vLLM | The vLLM v1 engine, checked against the scheduler oracle |
| vllm-ascend, vllm-rbln | Vendor plugins' scheduling and KV-cache behavior, as reduced specifications |
| Input shapes and padding | How scheduler output becomes compiled device inputs, per vendor |
| Different workloads | One engine under single-turn, chat and subagent traffic |
| Prefill/decode over NIXL | llm-d's prefill/decode disaggregation with the KV transfer |
Vendor plugins¶
vLLM is the baseline the plugins are compared against. The vendor pages examine the latest version tags checked on 2026-09-30, including release candidates and alpha tags. They cover full attention only and distinguish the plugin version from its upstream vLLM dependency. Features from development branches or older schedulers are not mixed into the comparison.
The examples were verified with IR v9. They are reduced specifications, not vendor scheduler oracles. Their capacities and cost constants are illustrative, not hardware measurements.
Version basis¶
| Repository | Latest version tag | Upstream vLLM basis | Evidence |
|---|---|---|---|
| vLLM | v0.31.0rc2 (RC) |
Same tag | Tag |
| vllm-ascend | v0.27.1rc1 (RC) |
v0.27.1 |
Release |
| vllm-rbln | v0.11.3a21 (alpha) |
0.26.0+cpu |
Dependency |
Vendor models¶
| Repository | Tagged implementation features | Executable serQ model | Remaining refinement |
|---|---|---|---|
| vLLM | Token budget, prefix lookup, priority/FCFS preemption, deferred free | Existing synchronous engine and request library | Content-key sharing, priority victims, asynchronous scheduling |
| vllm-ascend | Short-request classes and aging, job predictors, offload/recompute routing | FCFS classes with selection-time aging | Shared policy history, priority lanes, connector failure |
| vllm-rbln | Native phase isolation, PP decode caps, sub-block prefix copy | Whole-batch phase isolation and full-sequence admission gate | PP/remote-KV guards, copy references and cache units |
Device input constraints¶
Input shapes and padding follows scheduler output into runner inputs: compiled buckets, token/request padding, uniform query lengths and dummy metadata. Each vendor page now includes the specific tagged path. Real progress and padded device work must not be conflated. Padding/shape IR extensions are deferred; the audit is retained as source research, not an implementation commitment.
Run an example¶
From the repository root:
cargo run --release -- check examples/vendors/rbln.sq
cargo run --release -- run examples/vendors/rbln.sq --json
Replace rbln with ascend. Both use a finite batch of six requests and observe completed responses. The pages include these files directly, and the regular check gate links and draws them.
What has been verified¶
The pages separate tagged-source findings, executable approximations, remaining gaps and proposed oracle scenarios. The examples have been checked and run. Vendor SDKs and hardware were not exercised, and no vendor differential tests were run. The existing validation corpus does not validate these vendor tags.
Each vendor page records the limitations of its executable example. The waiting-selection design describes the implemented FCFS class precedence and aging in IR v9. The vendor documentation adds no language semantics.