Interface AcquiredState

A request's state, owned by the caller.

interface AcquiredState {
    attachments?: CheckpointAttachment[];
    caches: Cache[];
    continuation?: ContinuationState;
    retain?: () => void;
    reusedTokens: number;
}

Index

Properties

attachments? caches continuation? retain? reusedTokens

Properties

`Optional` `Readonly`attachments

attachments?: CheckpointAttachment[]

Method state stored with a reused prefix.

`Readonly`caches

caches: Cache[]

Restored, a reused prefix (composed as the graph's caches), or fresh.

`Optional` `Readonly`continuation

continuation?: ContinuationState

A resumed generation: caches cover its prompt and generated tokens.

`Optional` `Readonly`retain

retain?: () => void

Releases the backing of reused state; run it after disposing caches.

`Readonly`reusedTokens

reusedTokens: number

Prompt tokens caches already cover; 0 is a miss.