Incremental sharded writer for model-scale streaming transforms (27B fold/
quantize): tensors are added one at a time; when the current shard reaches
the byte ceiling it is SAVED AND ITS ARRAYS DISPOSED immediately, so peak
memory is one shard plus the tensor in flight — never the whole model
(the whole-list writeShardedSafetensors path holds every array until the
caller disposes them, which OOMs a 51 GB model on a 32 GB box).
Shards are written under temp names (the final -of-NNNNN count is unknown
until finish()) and renamed at finish, which also writes the index.
add() takes ownership of the array (evaluates it now, disposes after save).
Incremental sharded writer for model-scale streaming transforms (27B fold/ quantize): tensors are added one at a time; when the current shard reaches the byte ceiling it is SAVED AND ITS ARRAYS DISPOSED immediately, so peak memory is one shard plus the tensor in flight — never the whole model (the whole-list writeShardedSafetensors path holds every array until the caller disposes them, which OOMs a 51 GB model on a 32 GB box).
Shards are written under temp names (the final
-of-NNNNNcount is unknown until finish()) and renamed at finish, which also writes the index. add() takes ownership of the array (evaluates it now, disposes after save).