Class CostSizeRetention

GreedyDual size/frequency: saved prefill tokens are the cost proxy. Advancing age on eviction lets old popularity decay without wall timers.

Implements

RetentionPolicy

Index

Constructors

Properties

Methods

accessed evicted victim

Constructors

constructor

new CostSizeRetention(): CostSizeRetention

Returns CostSizeRetention

Properties

`Readonly`name

name: "cost-size"

Methods

accessed

accessed(entry: RetentionCandidate): void
Parameters
- entry: RetentionCandidate
Returns void
Implementation of RetentionPolicy.accessed
- Defined in packages/inference/src/state/retention-policy.ts:29

evicted

evicted(entry: RetentionCandidate): void
Parameters
- entry: RetentionCandidate
Returns void
Implementation of RetentionPolicy.evicted
- Defined in packages/inference/src/state/retention-policy.ts:36

victim

victim<T extends RetentionCandidate>(entries: readonly T[]): T
Type Parameters
- T extends RetentionCandidate
Parameters
- entries: readonly T[]
Returns T
Implementation of RetentionPolicy.victim
- Defined in packages/inference/src/state/retention-policy.ts:32