Workload¶
workload {
arrive poisson(rate); | arrive renewal(gap); | arrive closed(n); | arrive batch(n); | arrive none;
trace "file.csv" [ordered];
init block
turn block
session block
hidden NAME [, NAME]*;
}
How sessions arrive and what each turn brings. init and turn blocks may
only set and observe. Random draws in the workload use their own stream.
arrive¶
| Form | Argument | Sessions arrive |
|---|---|---|
poisson(rate) |
const, arrivals per clock unit |
one at time 0, then at exponential gaps of mean 1/rate |
renewal(gap) |
expr of constants and draws |
after one gap, then at every further gap; gap is evaluated anew for each |
closed(n) |
const, positive integer |
n at time 0, and a new one whenever a session ends |
batch(n) |
const, positive integer |
n at time 0, and no more |
none |
never (the default) |
poisson and renewal are open: they arrive until the horizon, or until
run { arrivals N; } has had its N.
renewal¶
arrive renewal(~h2(2, 4)); // interarrival times of mean 2, CV² 4
arrive renewal(2); // one every 2 clock units
| Argument | Type | Description |
|---|---|---|
gap |
expr |
The interarrival time. It may read numbers, let constants, functions of them and draws, and not a session attribute, a pool, a stage or a context variable. Each gap must be positive and finite, or the run aborts. |
The first arrival is one gap after time 0, where poisson has one at 0: with
the same seed, renewal(~exp(1 / rate)) arrives at the same times as
poisson(rate) after its first arrival, and has no arrival at 0
(design). Gaps draw from the arrival stream.
trace¶
| Argument | Type | Description |
|---|---|---|
| file | string | Path relative to the program. Columns session,turn,new,out,think[,forced], one row per turn. |
ordered |
flag | Session i replays trace session i. Without it, sessions draw turns from the corpus. |
At every turn; the next turn sets new, out, think and forced, and
sets more to 1 while another turn remains (attributes).
init¶
Runs once, when the session arrives. Moment Session.
turn¶
Runs at every turn; statement of the session.
Moment Session.
session¶
The session's side of a two-sided program. See Program.
hidden¶
| Argument | Type | Description |
|---|---|---|
NAME |
session attribute | The scheduler may not read it. |
A hidden attribute is legal at the Session moment only, so it may be read in a
session statement (decode (o - 1)), a run or a hold's cache. It is a link
error in a hold's units, reserve or reuse, a queue or eviction key, a spill
clause, a ps capacity, or a step stage's budget, cost, chunk or serve
keys. An attribute the scheduler itself sets (cached, computed) cannot be
hidden, and a name nothing sets is an error.
The vLLM programs hide o: the scheduler knows max_tokens and learns the
length only at EOS, so a program that reserves prompt + o is one vLLM cannot be.