Interface VisionEncoder<P>

Common contract for image encoders: preprocess image bytes into an encoder-specific representation, then turn it into language-space soft tokens [1, softTokens, hidden] (pre-divided by embed_scale). Encoders own their preprocessing.

interface VisionEncoder<
    P extends { softTokens: number } = { softTokens: number },
> {
    cacheIdentity?: string;
    dispose?(): void;
    features(pre: P): MlxArray;
    preprocess(bytes: Uint8Array): Promise<P>;
}

Type Parameters

P extends { softTokens: number } = { softTokens: number }

Index

Properties

Methods

dispose? features preprocess

Properties

`Optional` `Readonly`cacheIdentity

cacheIdentity?: string

Methods

`Optional`dispose

dispose?(): void
Returns void
- Defined in packages/inference/src/contracts/mlx/media.ts:65

features

features(pre: P): MlxArray
Parameters
- pre: P
Returns MlxArray
- Defined in packages/inference/src/contracts/mlx/media.ts:64

preprocess

preprocess(bytes: Uint8Array): Promise<P>
Parameters
- bytes: Uint8Array
Returns Promise<P>
- Defined in packages/inference/src/contracts/mlx/media.ts:63