sonicboom/trace/tensor
Research prototype. Pure Scheme; no libsonicboom. A concrete tensor (“ctensor”) is a dtype, a static shape and an owned
The bytevector is the storage, not a serialised view: ctensor-ref and ctensor-set! read and write it in place, and bytevector->ctensor shares it (reshape/view aliasing). Materialising a whole tensor into Scheme values is explicit: ctensor->list / ctensor->nested / ctensor-write are the only such interfaces, and printing streams from the storage.
It exists only so that the reference interpreter and eager mode have something to compute on; it is not a SonicBoom runtime tensor.
make-ctensor
procedure
A tensor built from a dtype, a shape and flat data.
DATA is a flat list or vector of Scheme values, or a bytevector of packed dtype values. A bytevector is copied; to share storage, use bytevector->ctensor.
Arguments:
dtype: the element dtype; see valid-dtype?.shape: a list of static non-negative exact integers.data: a flat list or vector with one value per element, or a bytevector of
Returns: a new ctensor.
Example:
(use-modules (sonicboom trace tensor))
(ctensor->list (make-ctensor 'float32 '(2 2) '(1.0 2.0 3.0 4.0)))=> (1.0 2.0 3.0 4.0)
PyTorch: torch.tensor
Note: The shape is given separately from the flat data; PyTorch infers it from the nesting of its input.
ctensor?
record type
A concrete reference tensor: a dtype, a static shape and owned storage.
The bytes are row-major and native-endian in the dtype’s native representation. ctensor? tests the type; ctensor-dtype and ctensor-shape read the dtype and the shape; ctensor-data exposes the backing bytevector.
PyTorch: torch.Tensor
Note: This is the tracing prototype’s reference tensor, written in pure Scheme. It models a tensor for the tracer at compile time; it is not the runtime tensor of (sonicboom tensor).
ctensor-dtype
record type
A concrete reference tensor: a dtype, a static shape and owned storage.
The bytes are row-major and native-endian in the dtype’s native representation. ctensor? tests the type; ctensor-dtype and ctensor-shape read the dtype and the shape; ctensor-data exposes the backing bytevector.
PyTorch: torch.Tensor
Note: This is the tracing prototype’s reference tensor, written in pure Scheme. It models a tensor for the tracer at compile time; it is not the runtime tensor of (sonicboom tensor).
ctensor-shape
record type
A concrete reference tensor: a dtype, a static shape and owned storage.
The bytes are row-major and native-endian in the dtype’s native representation. ctensor? tests the type; ctensor-dtype and ctensor-shape read the dtype and the shape; ctensor-data exposes the backing bytevector.
PyTorch: torch.Tensor
Note: This is the tracing prototype’s reference tensor, written in pure Scheme. It models a tensor for the tracer at compile time; it is not the runtime tensor of (sonicboom tensor).
ctensor-data
procedure
The bytevector backing T.
The bytevector is the tensor’s storage, not a copy: writing to it changes the tensor.
Arguments:
t: a ctensor.
Returns: a bytevector of (ctensor-nbytes T) bytes.
Example:
(use-modules (sonicboom trace tensor))
(bytevector-length (ctensor-data (make-ctensor 'float32 '(2) '(1.0 2.0))))=> 8
PyTorch: none
Note: The result is the raw bytevector, not a NumPy array.
bytevector->ctensor
procedure
A tensor that shares BV’s storage.
BV must hold exactly one packed element per element of SHAPE in DTYPE. No copy is made: writes through ctensor-set! are visible in BV, and writes to BV are visible through the tensor.
Arguments:
dtype: the element dtype.shape: a list of static non-negative exact integers.bv: a bytevector of (shape-numel SHAPE) times (dtype-size DTYPE) bytes.
Returns: a new ctensor sharing BV.
Example:
(use-modules (sonicboom trace tensor) (rnrs bytevectors))
(let* ((bv (make-bytevector 8 0))
(t (bytevector->ctensor 'int32 '(2) bv)))
(ctensor-set! t 0 7)
(bytevector-s32-native-ref bv 0))=> 7
PyTorch: torch.frombuffer
Note: torch.frombuffer also views a buffer without copying.
ctensor-copy
procedure
An independent tensor with the same dtype, shape and bytes.
Arguments:
t: a ctensor.
Returns: a new ctensor; changing one does not affect the other.
Example:
(use-modules (sonicboom trace tensor))
(ctensor->list (ctensor-copy (make-ctensor 'float32 '(2) '(1.0 2.0))))=> (1.0 2.0)
PyTorch: torch.Tensor.clone
ctensor-nbytes
procedure
The size of T’s storage in bytes.
Arguments:
t: a ctensor.
Returns: an exact integer.
Example:
(use-modules (sonicboom trace tensor))
(ctensor-nbytes (make-ctensor 'float32 '(2 3) '(0.0 0.0 0.0 0.0 0.0 0.0)))=> 24
PyTorch: torch.Tensor.nbytes
dtype-size
procedure
The number of bytes per element of DTYPE.
Arguments:
dtype: a valid dtype.
Returns: 1, 2, 4 or 8.
Example:
(use-modules (sonicboom trace tensor))
(dtype-size 'float64)=> 8
PyTorch: none
ctensor-numel
procedure
The number of elements in T.
Arguments:
t: a ctensor.
Returns: an exact integer.
Example:
(use-modules (sonicboom trace tensor))
(ctensor-numel (make-ctensor 'float32 '(2 3) '(0.0 0.0 0.0 0.0 0.0 0.0)))=> 6
PyTorch: torch.Tensor.numel
ctensor-ref
procedure
Element I of T as a Scheme value.
The index is flat and row-major. The value is an inexact real for a float dtype, an exact integer for an integer dtype, and a boolean for bool.
Arguments:
t: a ctensor.i: the flat element index, from 0 to (ctensor-numel T) - 1.
Returns: the element.
Example:
(use-modules (sonicboom trace tensor))
(ctensor-ref (make-ctensor 'float32 '(2) '(1.5 2.5)) 1)=> 2.5
PyTorch: none
Note: PyTorch indexes with one subscript per axis; this index is flat.
ctensor-set!
procedure
Store V as element I of T.
The value is converted to T’s dtype: floats round to the dtype’s precision (nearest, ties to even) and integers must be within the dtype’s range.
Arguments:
t: a ctensor.i: the flat element index.v: the new value.
Returns: an unspecified value.
Example:
(use-modules (sonicboom trace tensor))
(let ((t (make-ctensor 'float32 '(2) '(1.0 2.0))))
(ctensor-set! t 0 5.0)
(ctensor->list t))=> (5.0 2.0)
PyTorch: none
Note: The write is direct; PyTorch item assignment is subscript-based and records autograd history.
ctensor->list
procedure
All elements of T as a flat Scheme list in row-major order.
Arguments:
t: a ctensor.
Returns: a list with (ctensor-numel T) elements.
Example:
(use-modules (sonicboom trace tensor))
(ctensor->list (make-ctensor 'float32 '(2) '(1.0 2.0)))=> (1.0 2.0)
PyTorch: torch.Tensor.tolist
ctensor->nested
procedure
The elements of T regrouped into nested lists following its shape.
A rank-0 tensor gives its single element rather than an empty list.
Arguments:
t: a ctensor.
Returns: a nested list, or the single element of a rank-0 tensor.
Example:
(use-modules (sonicboom trace tensor))
(ctensor->nested (make-ctensor 'float32 '(2 2) '(1.0 2.0 3.0 4.0)))=> ((1.0 2.0) (3.0 4.0))
PyTorch: torch.Tensor.tolist
ctensor-write
procedure
Write T’s values to PORT as nested lists in shape order.
The values are streamed from storage; no whole-tensor list is built.
Arguments:
t: a ctensor.port: an output port.
Returns: an unspecified value.
Example:
(use-modules (sonicboom trace tensor))
(call-with-output-string
(lambda (p)
(ctensor-write (make-ctensor 'float32 '(2 2) '(1.0 2.0 3.0 4.0)) p)))=> "((1.0 2.0) (3.0 4.0))"
PyTorch: none
list->ctensor
procedure
A tensor built from nested lists.
The nesting gives the shape; a bare number or boolean gives a rank-0 tensor.
Arguments:
nested: a rectangular nested list of numbers or booleans.dtype: optional; the default is bool when the elements are booleans and
Returns: a new ctensor.
Example:
(use-modules (sonicboom trace tensor))
(ctensor->nested (list->ctensor '((1 2) (3 4))))=> ((1.0 2.0) (3.0 4.0))
PyTorch: torch.tensor
scalar->ctensor
procedure
A rank-0 tensor holding X.
Arguments:
x: a number or a boolean.dtype: optional; the default is bool for a boolean and float32 otherwise.
Returns: a rank-0 ctensor.
Example:
(use-modules (sonicboom trace tensor))
(ctensor-item (scalar->ctensor 3.0))=> 3.0
PyTorch: torch.tensor
ctensor-item
procedure
The single element of a one-element tensor.
T may have any shape as long as it has exactly one element.
Arguments:
t: a ctensor with one element.
Returns: the element.
Example:
(use-modules (sonicboom trace tensor))
(ctensor-item (make-ctensor 'float32 '(1 1) '(7.0)))=> 7.0
PyTorch: torch.Tensor.item
make-leaf
procedure
A leaf with VALUE and an optional gradient.
The value is a ctensor, or a symbolic tensor while the tracer runs.
Arguments:
value: the tensor value.grad: optional; the gradient, initially absent.
Returns: a new leaf.
Example:
(use-modules (sonicboom trace tensor))
(ctensor->list (leaf-value (make-leaf (make-ctensor 'float32 '(2) '(1.0 2.0)))))=> (1.0 2.0)
PyTorch: none
Note: A PyTorch parameter is a tensor that owns its gradient; a leaf keeps the value and the gradient in separate slots.
leaf?
record type
A tensor value and its gradient slot.
A leaf is what a traced training step differentiates. leaf-value holds the tensor (a ctensor, or the symbolic tensor the tracer substitutes for it) and leaf-grad holds its gradient, if any. leaf? tests the type; leaf-value and leaf-grad read the slots and set-leaf-value! and set-leaf-grad! replace them. The object keeps its identity across calls while the slots change.
PyTorch: torch.nn.Parameter
Note: The nearest PyTorch notion is a leaf tensor with requires_grad; here the value and the gradient are two slots of an explicit object.
leaf-value
record type
A tensor value and its gradient slot.
A leaf is what a traced training step differentiates. leaf-value holds the tensor (a ctensor, or the symbolic tensor the tracer substitutes for it) and leaf-grad holds its gradient, if any. leaf? tests the type; leaf-value and leaf-grad read the slots and set-leaf-value! and set-leaf-grad! replace them. The object keeps its identity across calls while the slots change.
PyTorch: torch.nn.Parameter
Note: The nearest PyTorch notion is a leaf tensor with requires_grad; here the value and the gradient are two slots of an explicit object.
set-leaf-value!
record type
A tensor value and its gradient slot.
A leaf is what a traced training step differentiates. leaf-value holds the tensor (a ctensor, or the symbolic tensor the tracer substitutes for it) and leaf-grad holds its gradient, if any. leaf? tests the type; leaf-value and leaf-grad read the slots and set-leaf-value! and set-leaf-grad! replace them. The object keeps its identity across calls while the slots change.
PyTorch: torch.nn.Parameter
Note: The nearest PyTorch notion is a leaf tensor with requires_grad; here the value and the gradient are two slots of an explicit object.
leaf-grad
record type
A tensor value and its gradient slot.
A leaf is what a traced training step differentiates. leaf-value holds the tensor (a ctensor, or the symbolic tensor the tracer substitutes for it) and leaf-grad holds its gradient, if any. leaf? tests the type; leaf-value and leaf-grad read the slots and set-leaf-value! and set-leaf-grad! replace them. The object keeps its identity across calls while the slots change.
PyTorch: torch.nn.Parameter
Note: The nearest PyTorch notion is a leaf tensor with requires_grad; here the value and the gradient are two slots of an explicit object.
set-leaf-grad!
record type
A tensor value and its gradient slot.
A leaf is what a traced training step differentiates. leaf-value holds the tensor (a ctensor, or the symbolic tensor the tracer substitutes for it) and leaf-grad holds its gradient, if any. leaf? tests the type; leaf-value and leaf-grad read the slots and set-leaf-value! and set-leaf-grad! replace them. The object keeps its identity across calls while the slots change.
PyTorch: torch.nn.Parameter
Note: The nearest PyTorch notion is a leaf tensor with requires_grad; here the value and the gradient are two slots of an explicit object.
make-lr-tensor
procedure
A one-element float32 learning-rate tensor for a traced training step.
Passed to a training step as #:lr, the tensor becomes a graph input rather than an attribute, so a schedule can change the rate without retracing. Read the rate with leaf-value and replace it with set-leaf-value!.
Arguments:
device: optional; ’cpu (the default) or ’cuda, accepted for call-sitevalue: keyword; the initial rate (default 1e-3).
Returns: a leaf holding a rank-0 float32 tensor.
Example:
(use-modules (sonicboom trace tensor))
(ctensor-item (leaf-value (make-lr-tensor 'cpu #:value 0.5)))=> 0.5
PyTorch: none
Note: PyTorch passes the learning rate as a number to the optimizer.
shape-numel
procedure
The number of elements of SHAPE.
Arguments:
shape: a list of non-negative exact integers.
Returns: the product of the dimensions; 1 for the empty shape.
Example:
(use-modules (sonicboom trace tensor))
(shape-numel '(2 3 4))=> 24
PyTorch: none
float-dtype?
procedure
Whether DTYPE is a floating-point dtype.
Arguments:
dtype: a symbol.
Returns: #t for float32, float16, bfloat16 and float64; #f otherwise.
Example:
(use-modules (sonicboom trace tensor))
(float-dtype? 'float16)=> #t
PyTorch: none
int-dtype?
procedure
Whether DTYPE is an integer dtype.
Arguments:
dtype: a symbol.
Returns: #t for int8, uint8, int16, int32 and int64; #f otherwise.
Example:
(use-modules (sonicboom trace tensor))
(int-dtype? 'int64)=> #t
PyTorch: none
valid-dtype?
procedure
Whether DTYPE is one of the dtypes a ctensor accepts.
Arguments:
dtype: a symbol.
Returns: #t for float32, float16, bfloat16, float64, int8, uint8, int16, int32, int64 and bool; #f otherwise.
Example:
(use-modules (sonicboom trace tensor))
(valid-dtype? 'bfloat16)=> #t
PyTorch: none
coerce-value
procedure
Convert X to the Scheme representation used for DTYPE.
A float dtype accepts any real and returns it inexact; an integer dtype accepts an integer value and returns it exact; bool accepts a boolean or a real and returns a boolean.
Arguments:
dtype: a valid dtype.x: the value to convert.
Returns: the converted Scheme value.
Example:
(use-modules (sonicboom trace tensor))
(coerce-value 'int32 2.0)=> 2
PyTorch: none