Interface GenerationSession<Metrics>

interface GenerationSession<Metrics> {
    events: AsyncIterable<GenerationEvent>;
    id: string;
    result: Promise<GenerationOutcome<Metrics>>;
    state: SessionState;
    cancel(reason?: CancelReason): Promise<void>;
    close(): Promise<void>;
}

Type Parameters

Metrics

Implemented by

EngineSession

Index

Properties

events id result state

Methods

Properties

`Readonly`events

events: AsyncIterable<GenerationEvent>

Single consumer. Stream mode starts on first next(); collect mode has no events.

`Readonly`id

id: string

`Readonly`result

result: Promise<GenerationOutcome<Metrics>>

Always resolves with one terminal outcome, after native resources are released.

`Readonly`state

state: SessionState

Methods

cancel

cancel(reason?: CancelReason): Promise<void>
Parameters
- Optionalreason: CancelReason
Returns Promise<void>
- Defined in packages/inference/src/contracts/portable/generation.ts:91

close

close(): Promise<void>
Returns Promise<void>
- Defined in packages/inference/src/contracts/portable/generation.ts:92