mlx-bun library API
    Preparing search index...

    Module @mlx-bun/inference/state

    BatchedKVCache
    BatchedQuantizedKVCache
    BatchedRotatingCache
    BatchedRotatingQuantCache
    BatchedSSMCache
    BatchedTurboQuantKVCache
    Bf16FirstKv4Head256Cache
    Bf16FirstQuantizedKVCache
    Bf16FirstRotatingQuantizedKVCache
    Bf16FirstTurboQuantKVCache
    ContinuationPersistence
    CostSizeRetention
    DelayedQuantizedKVCache
    DelayedRotatingQuantizedKVCache
    DelayedTurboQuantKVCache
    DSAIndexCache
    EncoderCache
    KVCache
    KvCacheService
    LruRetention
    MLACache
    MlxStateRows
    PromptCache
    QuantizedKVCache
    RotatingKVCache
    RotatingQuantizedKVCache
    SpeculativeRotatingKVCache
    SpillQueue
    SsdCacheStore
    SsdDurabilityCoordinator
    SSMCache
    TieredPromptCache
    TurboQuantCodec
    TurboQuantKVCache
    AcquiredState
    AffineKernels
    CacheCodec
    CacheCodecProvider
    CacheHeaderEntry
    CacheServiceGraph
    CloneContext
    ColdTier
    ContinuationAttempt
    ContinuationRequest
    ContinuationState
    DSAIndexGeometry
    DurabilityFlushResult
    DurabilityPromptCache
    DurabilitySnapshotStats
    KvCacheParts
    KvCacheReport
    KvCacheServiceOptions
    KvCacheSettings
    KvFileHeader
    KvLoadExpect
    KvSaveMeta
    LoadContext
    LoadedKvCache
    MLACacheGeometry
    MLACompressedState
    MlxRowLayout
    PromptCacheEntry
    PublishedState
    RequestStateCache
    RetentionCandidate
    RetentionPolicy
    RowSpeculativeState
    SavedStateBudget
    SavedStateOptions
    SnapshotContext
    SpillItem
    SpillSink
    SsdIndexEntry
    SsdStoreOptions
    SsmSpecRound
    StateRequest
    TensorSlot
    TurboQuantKernels
    TurboQuantTensor
    AffineKernelSet
    CacheKind
    ContinuationStore
    MlxPrefixCache
    deferredTurboQuantKernels
    denseTurboQuantKernels
    legacyCacheCodecs
    unchangedKv
    uncomposedTurboQuantKernels
    activeQuantViews
    attachmentBytes
    batchRowKvBytes
    bindCacheRollback
    bindRowCacheRollback
    bindRowStepRollback
    buildBatchedRotatingMask
    cacheBytes
    cacheHeaderMinimumReusableOffset
    cacheHeadersMinimumReusableOffset
    cacheHeadersTrimmable
    captureCacheAttachment
    cloneAttachments
    cloneKvCaches
    cloneSingleRowState
    commonPrefixLength
    completeConversion
    composeAtLayer
    composedAffineKernels
    composedFullAttentionCache
    composedSlidingAttentionCache
    concatDecodeSlot
    convertToAffine
    convertToTurboQuant
    createCacheCodecProvider
    createKvCacheService
    createKvMaintenance
    disposeAttachments
    disposeTurboQuant
    kv4Head256KernelSet
    leaseCacheState
    leaseCacheStates
    loadKvCache
    loadKvCacheAsync
    materializeCheckpoint
    minimumReusableOffset
    quantConcatDecodeSlot
    readKvHeader
    restoreCacheAttachment
    rollbackVerifyBlock
    rollsBackVerifyBlock
    rotatingWindowMask
    saveKvCache
    saveKvCacheAsync
    tiledAffineKernels
    tiledCausalAffineKernels
    transitionDue
    turboQuantDecodeSlot
    turboQuantFusedDecode
    unfusedAffineKernels
    cacheLayerKind → cacheLayerKind
    CacheLayerKind → CacheLayerKind
    CheckpointAttachment → CheckpointAttachment
    FUSED_SDPA_HEAD_DIMS → FUSED_SDPA_HEAD_DIMS
    kvBytesAt → kvBytesAt
    kvGeometry → kvGeometry
    KvGeometry → KvGeometry
    KvMaintenance → KvMaintenance
    kvQuantBytesPerElement → kvQuantBytesPerElement
    KvQuantOverride → KvQuantOverride
    KvScheme → KvScheme
    KvSchemeKind → KvSchemeKind
    kvSchemeOf → kvSchemeOf
    KvSchemeOptions → KvSchemeOptions
    resolveKvScheme → resolveKvScheme
    sdpaFallbackBytes → sdpaFallbackBytes

    Settings
    Member Visibility

    On This Page
    Classes
    Interfaces
    Type Aliases
    Variables
    Functions
    activeQuantViewsattachmentBytesbatchRowKvBytesbindCacheRollbackbindRowCacheRollbackbindRowStepRollbackbuildBatchedRotatingMaskcacheBytescacheHeaderMinimumReusableOffsetcacheHeadersMinimumReusableOffsetcacheHeadersTrimmablecaptureCacheAttachmentcloneAttachmentscloneKvCachescloneSingleRowStatecommonPrefixLengthcompleteConversioncomposeAtLayercomposedAffineKernelscomposedFullAttentionCachecomposedSlidingAttentionCacheconcatDecodeSlotconvertToAffineconvertToTurboQuantcreateCacheCodecProvidercreateKvCacheServicecreateKvMaintenancedisposeAttachmentsdisposeTurboQuantkv4Head256KernelSetleaseCacheStateleaseCacheStatesloadKvCacheloadKvCacheAsyncmaterializeCheckpointminimumReusableOffsetquantConcatDecodeSlotreadKvHeaderrestoreCacheAttachmentrollbackVerifyBlockrollsBackVerifyBlockrotatingWindowMasksaveKvCachesaveKvCacheAsynctiledAffineKernelstiledCausalAffineKernelstransitionDueturboQuantDecodeSlotturboQuantFusedDecodeunfusedAffineKernels
    References