API Reference

serQ's API is its language: the constructs a program is written with. Each entry gives the signature, what it does, the type of every argument, what it evaluates to or changes, and where it may appear. The language is the specification and argues the design; this section is the lookup.

By what you are modelling

Start from the part of a deployment you want to write; each row names the constructs that say it and a program that uses them.

To model…	Read	In a program
a request queue and a server	stage (fifo, ps, delay), run	examples/single-turn/mg1.sq; tutorial 1
arrivals, turns and think time	arrive, turn, session; think time as a delay stage, tool	the workloads use case
replaying a production trace	trace	examples/replay/vllm_replay.sq (discussed in tutorial 6)
memory a request holds while it runs	pool, hold	tutorial 2
what the scheduler reads at admission	at admission, moments	lib/vllm.sq (known, hit)
a prefix cache across turns	cache, evict, cached (attributes), drop; cachedin for the lookup at admission	tutorial 4; lib/vllm.sq for cachedin
an offload tier for evicted prefixes	spill	no example program yet (tests/pool_semantics.rs)
continuous batching and chunked prefill	step (budget, chunk, cost), prefill, decode, growing	the vLLM use case; tutorial 5
the order requests are admitted and served in	queue by, serve (serve decode first, serve by (…))	examples/multi-turn/replica.sq (serve decode first); queue by in no example yet (tests/pool_semantics.rs)
preemption and recompute	preempt lifo, computed (attributes)	lib/vllm.sq (known = computed …)
the scheduler's waiting loop and its budget	admit via, budget_left	examples/replay/vllm_replay.sq (admit via engine), tutorial 5; lib/vllm.sq (budget_left)
what the scheduler may not read	hidden	every vLLM program hides the output length (o; out in the replay)
prefill/decode disaggregation	lease, release, load, transfer … from … to	the P/D use case
a routing policy	choose, observables	examples/multi-turn/routing.sq
an engine shared by several programs	def, use	lib/vllm.sq and the programs that use it
what to measure	observe	every program; serq run reports it

By construct

Page	Contents
Program	let, pool, stage, workload, session, server, run
Pool	the options of pool
Stage	fifo, ps, delay, step
Workload	arrive, trace, init, turn, hidden
Statements	the kernel: hold, run, grow, branch, …
Serving vocabulary	prefill, transfer, decode, tool
Functions	arithmetic functions and observables
Distributions	~exp, ~det, ~uniform, ~erlang, ~h2, ~bernoulli
Context variables	tokens, age, remaining, … and the moment each exists at
Attributes	the built-in session attributes

Types

Type	Meaning
expr	An expression. Every value is a number (f64); comparisons and ! give 0 or 1, and a non-zero operand is true. inf is a number.
const	An expression that folds at link time: numbers, let constants, arithmetic and the arithmetic functions. It may not read a session attribute, a pool or a stage, or draw.
pool	The name of a pool, with an index (kv[j], any expr) when it is an array.
stage	The name of a stage, indexed the same way.
block	{ stmt* }.
NAME	An identifier. What it names (a session attribute, a constant, a pool, a stage) depends on where it is written.

An expr is evaluated at one moment, and the context variables it may read depend on which. The moment of each argument is stated in its entry.

expr, pool and stage are the three argument kinds a call can take. A function names the kind of every argument, and a pool name where an expr is expected (or the reverse) is a link error.