mlx-bun library API
    Preparing search index...

    Interface MlxGatewayBinding

    Planning consumes the graph's declared capabilities (declaredGraph), the request, and dynamic state; it never inspects a model's class, type, or name, nor how the composition's caches store keys and values: every cache lego serves every method. Request state, prefix reuse and persistence are the cache service's (state/cache-service), handed to createBatchGroup.

    interface MlxGatewayBinding {
        capabilities: GraphCapabilities;
        config: ModelConfig;
        graph: CacheServiceGraph;
        runtime: RuntimeConfig;
        bindAdapterContext?(adapters: string[], key: string): ExecutionContext;
        continuationRequest?(
            execution: ResolvedExecution | undefined,
            options: GenerateOptions,
            prompt: number[],
            onToken: TokenSink,
            interval: number | undefined,
        ):
            | {
                continuation: OrdinaryContinuation;
                dispose: () => void;
                onToken: TokenSink;
                plainGreedy: boolean;
                sample: (logits: MlxArray, step: number) => MlxArray;
            }
            | undefined;
        createBatchGroup(options: MlxBatchExecutionGroupOptions): MlxBatchGroup;
        mediaInput?(input: Vision): MlxPromptInput;
        methodRequest?(
            execution: ResolvedExecution | undefined,
            options: GenerateOptions,
        ): MlxGroupMethodRequest | undefined;
        plan(
            request: ExecutionRequirements,
            options: GenerateOptions,
            scheduling: { checkpoints: boolean },
        ): ResolvedExecution;
        prefixNamespace?(
            execution: ResolvedExecution | undefined,
            options: GenerateOptions,
            base: string,
        ): string | null;
        reusesPrefix(execution: ResolvedExecution | undefined): boolean;
    }
    Index
    capabilities config graph runtime
    bindAdapterContext? continuationRequest? createBatchGroup mediaInput? methodRequest? plan prefixNamespace? reusesPrefix
    capabilities: GraphCapabilities

    What the bound graph declares, as resolved when it was bound.

    config: ModelConfig
    graph: CacheServiceGraph

    What the composition's cache service builds request state from: the bound graph's caches, config and composition.

    runtime: RuntimeConfig

    • A replayable ordinary request when the plan qualified one: its sampler and the key the cache service checkpoints it under every interval tokens (the service's checkpointEveryTokens).

      Parameters

      Returns
          | {
              continuation: OrdinaryContinuation;
              dispose: () => void;
              onToken: TokenSink;
              plainGreedy: boolean;
              sample: (logits: MlxArray, step: number) => MlxArray;
          }
          | undefined

    Settings
    Member Visibility

    On This Page
    Properties
    Methods