Interface MethodRun<Metrics>

One attempt's private state, including any pending device work. close must await safe resource release, including on execute failure.

interface MethodRun<Metrics> {
    close(): Promise<void>;
    execute(
        output: GenerationOutput,
        cancellation: Cancellation,
    ): Promise<MethodResult<Metrics>>;
}

Type Parameters

Metrics

Index

Methods

Methods

close

close(): Promise<void>
Returns Promise<void>
- Defined in packages/inference/src/contracts/portable/generation.ts:40

execute

execute(
output: GenerationOutput,
cancellation: Cancellation,
): Promise<MethodResult<Metrics>>
Parameters
- output: GenerationOutput
- cancellation: Cancellation
Returns Promise<MethodResult<Metrics>>
- Defined in packages/inference/src/contracts/portable/generation.ts:39