LLM engines

reading engines…

window

Engines answering

n/a

Decode (sum)

n/atok/s

Prompt (sum)

n/atok/s

Active sessions

n/a

Queued

n/a

Tokens lifetime

n/a

KV cache now

n/a%

Decode throughput

tok/s · every engine, summed slots

Prompt / prefill throughput

tok/s · newly computed tokens

KV / prefix cache pressure

% of cache in use

Sessions & queue

running vs waiting

gufo slots

one row per parallel slot

Slot	State	Tokens	N prompt	N gen	Cache	Speed

Events

engine up / down

loading…