Class DiffusionVisionTower

constructor

• new DiffusionVisionTower(
      weights: Weights,
      config: ModelConfig,
  ): DiffusionVisionTower

  Parameters

  • weights: Weights
  • config: ModelConfig

  Returns DiffusionVisionTower

Readonlycfg

ReadonlydefaultOutputLength

ReadonlyembedEps

ReadonlyembedProj

ReadonlyinputProj

Readonlylayers

ReadonlymaxPatches

ReadonlypatchSize

ReadonlyposTable

ReadonlyrootHidden

ReadonlystdBias

ReadonlystdScale

getImageFeatures

• getImageFeatures(
      pixels: MlxArray,
      dbg?: (n: string, a: MlxArray) => void,
  ): MlxArray

  get_image_features = embed_vision(vision_tower(pixels)): MultimodalEmbedder (RMSNormNoScale → projection 1152→2816). NO /embed_scale.

  Parameters

  • pixels: MlxArray
  • Optionaldbg: (n: string, a: MlxArray) => void

  Returns MlxArray

preprocess

• preprocess(bytes: Uint8Array): Promise<{ pixels: MlxArray; softTokens: number }>

  Decode + aspect-preserving resize (multiples of pooling·patch=48, within the max_patches budget) + rescale 1/255 + channel-first. Mirrors optiq's gemma4 preprocess_images (BICUBIC, do_rescale, do_normalize=False). Returns pixel_values [1,3,H,W] f32 and the per-image soft-token count.

  Parameters

  • bytes: Uint8Array

  Returns Promise<{ pixels: MlxArray; softTokens: number }>

visionTower

• visionTower(pixels: MlxArray, dbg?: (n: string, a: MlxArray) => void): MlxArray

  Full tower forward over channel-first pixel_values [1,3,H,W] -> standardized pooled hidden [1, numRealSoft, hidden] (BEFORE the MultimodalEmbedder).

  Parameters

  • pixels: MlxArray
  • Optionaldbg: (n: string, a: MlxArray) => void

  Returns MlxArray