Class torch

TORCH: Tensor Operations with the Reasoning Capacity of Humans

Torch is a powerful library for tensor computations and deep learning, offering a comprehensive set of tools for creating and manipulating multidimensional arrays. It provides a wide range of mathematical operations, and it includes a neural network module (torch.nn) that facilitates the construction of complex neural architectures through a modular approach, with various layer types and activation functions readily available. Torch also implements automatic differentiation, enabling efficient gradient computation for training neural networks, and offers optimization algorithms like Adam for parameter updates. Additionally, it includes utilities for saving and loading models, making it a versatile and complete framework for developing and deploying machine learning solutions.

  1. Torch is a neural net matrix multiplication library that uses PyTorch API syntax for tensors and neural nets.
  2. Uses GPU.js acceleration to translate matmul into WebGL shader code. GPU.js does matmul faster than PyTorch.
  3. Neural Net API: MultiHeadSelfAttention, FullyConnected, Block, Embedding, PositionalEmbedding, ReLU, Softmax, Dropout, LayerNorm, CrossEntropyLoss.
  4. Other Neural Nets: For LSTMs and CNNs, use Tensorflow.js or Brain.js
1. Tensor Creation:
  - `tensor()`: Creates a new Tensor filled with given data
  - `zeros()`: Creates a new Tensor filled with zeros
  - `ones()`: Creates a new Tensor filled with ones
  - `randn()`: Creates a new Tensor filled with random values from a normal distribution
  - `rand()`: Creates a new Tensor filled with random values from a uniform distribution

2. Tensor Properties and Methods:
  - `backward()`: Performs backpropagation from this tensor backwards
  - `zero_grad()`: Clears the gradients stored in this tensor
  - `tolist()`: Returns the tensor's data as a JavaScript Array
  - Properties: `data`, `length`, `ndims`, `grad`

3. Basic Arithmetic Operations:
  - `add()`, `sub()`, `mul()`, `div()`: Element-wise arithmetic operations
  - `matmul()`: Matrix multiplication between two tensors
  - `pow()`: Element-wise power operation

4. Statistical Operations:
  - `sum()`: Gets the sum of the Tensor over a specified dimension
  - `mean()`: Gets the mean of the Tensor over a specified dimension
  - `variance()`: Gets the variance of the Tensor over a specified dimension

5. Tensor Manipulation:
  - `transpose()`: Transposes the tensor along two consecutive dimensions
  - `at()`: Returns elements from the tensor based on given indices
  - `masked_fill()`: Fills elements in the tensor based on a condition

6. Mathematical Functions:
  - `sqrt()`: Element-wise square root
  - `exp()`: Element-wise exponentiation
  - `log()`: Element-wise natural logarithm

7. Neural Network Layers (torch.nn):
  - `Linear()`: Applies a linear transformation
  - `MultiHeadSelfAttention()`: Applies a self-attention layer
  - `Embedding()`: Creates an embedding table for vocabulary
  - Activation functions: `ReLU()`, `Softmax()`

8. Optimization and Loss:
  - `optim.Adam()`: Adam optimizer for updating model parameters
  - `nn.CrossEntropyLoss()`: Computes Cross Entropy Loss

torch

Function

tensor(data, requires_grad = false, device = 'cpu') Creates a new Tensor filled with the given data

Function

zeros(*shape, requires_grad = false, device = 'cpu') Creates a new Tensor filled with zeros

Function

ones(*shape, requires_grad = false, device = 'cpu') Creates a new Tensor filled with ones

Function

tril(*shape, requires_grad = false, device = 'cpu') Creates a new 2D lower triangular Tensor

Function

randn(*shape, requires_grad = false, device = 'cpu', xavier = false) Creates a new Tensor filled with random values from a normal distribution

Function

rand(*shape, requires_grad = false, device = 'cpu') Creates a new Tensor filled with random values from a uniform distribution

Function

randint(low, high, *shape, requires_grad = false, device = 'cpu') Creates a new Tensor filled with random integers Tensor Methods:

Method

tensor.backward() Performs backpropagation from this tensor backwards

Method

tensor.zero_grad() Clears the gradients stored in this tensor

Method

tensor.zero_grad_graph() Clears the gradients stored in this tensor and all tensors that led to it

Method

tensor.tolist() Returns the tensor's data as a JavaScript Array

Function

add(a, b) Performs element-wise addition of two tensors

Function

sub(a, b) Performs element-wise subtraction of two tensors

Function

neg(a) Returns the element-wise opposite of the given Tensor

Function

mul(a, b) Performs element-wise multiplication of two tensors

Function

div(a, b) Performs element-wise division of two tensors

Function

matmul(a, b) Performs matrix multiplication between two tensors

Function

sum(a, dim, keepdims = false) Gets the sum of the Tensor over a specified dimension

Function

mean(a, dim, keepdims = false) Gets the mean of the Tensor over a specified dimension

Function

variance(a, dim, keepdims = false) Gets the variance of the Tensor over a specified dimension

Function

transpose(a, dim1, dim2) Transposes the tensor along two consecutive dimensions

Function

at(a, index1, index2) Returns elements from the tensor based on given indices

Function

masked_fill(a, condition, value) Fills elements in the tensor based on a condition

Function

pow(a, n) Returns tensor raised to element-wise power

Function

sqrt(a) Returns element-wise square root of the tensor

Function

exp(a) Returns element-wise exponentiation of the tensor

Function

log(a) Returns element-wise natural log of the tensor

torch.nn Neural Network Layers:

Method

nn.Linear(in_size, out_size, device, bias, xavier) Applies a linear transformation to the input tensor

Method

nn.MultiHeadSelfAttention(in_size, out_size, n_heads, n_timesteps, dropout_prob, device) Applies a self-attention layer on the input tensor

Function

nn.FullyConnected(in_size, out_size, dropout_prob, device, bias) Applies a fully-connected layer on the input tensor

Function

nn.Block(in_size, out_size, n_heads, n_timesteps, dropout_prob, device) Applies a transformer Block layer on the input tensor

Function

nn.Embedding(in_size, embed_size) Creates an embedding table for vocabulary

Function

nn.PositionalEmbedding(input_size, embed_size) Creates a positional embedding table

Function

nn.ReLU() Applies Rectified Linear Unit activation function

Function

nn.Softmax() Applies Softmax activation function

Function

nn.Dropout(drop_prob) Applies dropout to input tensor

Function

nn.LayerNorm(n_embed) Applies Layer Normalization to input tensor

Function

nn.CrossEntropyLoss() Computes Cross Entropy Loss between target and input tensor

Optimization: optim.Adam(params, lr, reg, betas, eps) Adam optimizer for updating model parameters

Utility Functions:

Function

save(model, file) Saves the model reruning data blob (for you to save)

Function

load(model, loadedData) Loads the model from saved data

Author

PyTorch Contributors, Leao, E. et al (2022), See also: Brain.js

Properties

Tensor Parameter add neg mul div matmul exp log sqrt pow mean masked_fill variance at reshape _reshape transpose tensor randint randn rand tril ones zeros broadcast save load nn optim getShape

Properties

StaticTensor

Tensor: typeof Tensor

Add methods from tensor.js (these methods are accessed with "torch."):

StaticParameter

Parameter: typeof Parameter

Staticadd

add: ((a: any, b: any) => any)

Staticneg

neg: ((a: any) => any)

Staticmul

mul: ((a: any, b: any) => any)

Staticdiv

div: ((a: any, b: any) => any)

Staticmatmul

matmul: ((a: any, b: any) => any)

Staticexp

exp: ((a: any) => any)

Staticlog

log: ((a: any) => any)

Staticsqrt

sqrt: ((a: any) => any)

Staticpow

pow: ((a: any, n: any) => any)

Staticmean

mean: ((a: any, dim?: number, keepdims?: boolean) => any)

Staticmasked_fill

masked_fill: ((a: any, mask: any, condition: any, value: any) => any)

Staticvariance

variance: ((a: any, dim?: number, keepdims?: boolean) => any)

Staticat

at: ((a: any, idx1: any, idx2: any) => any)

Staticreshape

reshape: ((a: any, shape: any) => any)

Static_reshape

_reshape: ((a: any, shape: any) => any)

Statictranspose

transpose: ((a: any, dim1: any, dim2: any) => any)

Statictensor

tensor: ((data: any, requires_grad?: boolean, device?: string) => Tensor)

Staticrandint

randint: ((low?: number, high?: number, shape?: number[], requires_grad?: boolean) => Tensor)

Staticrandn

randn: ((shape: any, requires_grad?: boolean, device?: string, xavier?: boolean) => Tensor)

Staticrand

rand: ((shape: any, requires_grad?: boolean, device?: string) => Tensor)

Statictril

tril: ((shape: any, requires_grad?: boolean, device?: string) => Tensor)

Staticones

ones: ((shape: any, requires_grad?: boolean, device?: string) => Tensor)

Staticzeros

zeros: ((shape: any, requires_grad?: boolean, device?: string) => Tensor)

Staticbroadcast

broadcast: ((a: any, b: any) => Tensor)

Staticsave

save: ((model: any, file: any) => string)

Staticload

load: ((model: any, loadedData: any) => any)

Staticnn

nn: {
    Module: typeof Module;
    Linear: typeof Linear;
    MultiHeadSelfAttention: typeof MultiHeadSelfAttention;
    FullyConnected: typeof FullyConnected;
    Block: typeof Block;
    Embedding: typeof Embedding;
    PositionalEmbedding: typeof PositionalEmbedding;
    ReLU: typeof ReLU;
    Softmax: typeof Softmax;
    Dropout: typeof Dropout;
    LayerNorm: typeof LayerNorm;
    CrossEntropyLoss: typeof CrossEntropyLoss;
}

Add submodules:

Staticoptim

optim: {
    Adam: typeof Adam;
}

StaticgetShape

getShape: ((data: any, shape?: any[]) => any[])

Settings

Member Visibility

On This Page

TORCH: Tensor Operations with the Reasoning Capacity of Humans
Properties