Interface PixelInput

Raw image pixels that a denoising method takes as its own prefill input.

interface PixelInput {
    preprocess(
        bytes: Uint8Array,
    ): Promise<{ pixels: MlxArray; softTokens: number }>;
    spliceTokens(
        rawIds: number[],
        softTokens: number,
        tokenIds: { boi: number; eoi: number; image: number },
    ): number[];
}

Index

Methods

preprocess spliceTokens

Methods

preprocess

preprocess(bytes: Uint8Array): Promise<{ pixels: MlxArray; softTokens: number }>
Pixels are owned by the caller once returned.
Parameters
- bytes: Uint8Array
Returns Promise<{ pixels: MlxArray; softTokens: number }>
- Defined in packages/inference/src/contracts/mlx/media.ts:71

spliceTokens

spliceTokens(
    rawIds: number[],
    softTokens: number,
    tokenIds: { boi: number; eoi: number; image: number },
): number[]
Insert softTokens image placeholders (one image) into rendered prompt ids.
Parameters
- rawIds: number[]
- softTokens: number
- tokenIds: { boi: number; eoi: number; image: number }
Returns number[]
- Defined in packages/inference/src/contracts/mlx/media.ts:73

Interface PixelInput

Index

Methods

Methods

preprocess

Parameters

Returns Promise<{ pixels: MlxArray; softTokens: number }>

spliceTokens

Parameters

Returns number[]

Settings

On This Page