mlx-bun library API
    Preparing search index...

    Interface CompiledGemmaTarget

    What the runner reads of the Gemma graph it compiles (the graph passes itself; a structural type keeps the graph file and this one acyclic).

    interface CompiledGemmaTarget {
        config: { text: { enableMoeBlock: boolean; numKvSharedLayers: number } };
        decode: MlxRowTable<MlxCapturingForward>;
        embed: { encode(ids: MlxArray): MlxArray };
        embedScale: number;
        finalNorm: { forward(h: MlxArray): MlxArray };
        layers: readonly {
            forward(
                x: MlxArray,
                cache: Cache,
                phase: "decode",
                donor: null,
                perLayerInput: null,
            ): { donor: { dispose(): void } | null; h: MlxArray };
        }[];
        perLayerWidth: number;
        traceDecode?: MlxCapturingForward;
        traceLayers?: readonly {
            forward(
                x: MlxArray,
                cache: Cache,
                phase: "decode",
                donor: null,
                perLayerInput: null,
            ): { donor: { dispose(): void } | null; h: MlxArray };
        }[];
        traceLogits?: (hidden: MlxArray) => MlxArray;
        logitsFromHidden(hidden: MlxArray): MlxArray;
        materializeGraphConstants(): void;
    }
    Index
    config decode embed embedScale finalNorm layers perLayerWidth traceDecode? traceLayers? traceLogits?
    logitsFromHidden materializeGraphConstants
    config: { text: { enableMoeBlock: boolean; numKvSharedLayers: number } }
    decode: MlxRowTable<MlxCapturingForward>

    The one-row decode operation the trace records.

    embed: { encode(ids: MlxArray): MlxArray }
    embedScale: number
    finalNorm: { forward(h: MlxArray): MlxArray }
    layers: readonly {
        forward(
            x: MlxArray,
            cache: Cache,
            phase: "decode",
            donor: null,
            perLayerInput: null,
        ): { donor: { dispose(): void } | null; h: MlxArray };
    }[]

    A decoder layer reading its cache for one decode position. A donor layer returns its read, which the runner disposes.

    perLayerWidth: number
    traceDecode?: MlxCapturingForward

    Written graphs name their plain operations for the enclosing trace.

    traceLayers?: readonly {
        forward(
            x: MlxArray,
            cache: Cache,
            phase: "decode",
            donor: null,
            perLayerInput: null,
        ): { donor: { dispose(): void } | null; h: MlxArray };
    }[]
    traceLogits?: (hidden: MlxArray) => MlxArray

    Settings
    Member Visibility

    On This Page
    Properties
    Methods