neural-net

Documentation / train/neural-net

Neural Net

torch

Defined in: train/neural-net.js:2224

TORCH: Tensor Operations with the Reasoning Capacity of Humans

Torch is a powerful library for tensor computations and deep learning, offering a comprehensive set of tools for creating and manipulating multidimensional arrays. It provides a wide range of mathematical operations, and it includes a neural network module (torch.nn) that facilitates the construction of complex neural architectures through a modular approach, with various layer types and activation functions readily available. Torch also implements automatic differentiation, enabling efficient gradient computation for training neural networks, and offers optimization algorithms like Adam for parameter updates. Additionally, it includes utilities for saving and loading models, making it a versatile and complete framework for developing and deploying machine learning solutions.

1. Torch is a neural net matrix multiplication library that uses PyTorch API syntax for tensors and neural nets.
2. Uses GPU.js acceleration to translate matmul into WebGL shader code. GPU.js does matmul faster than PyTorch.
3. Neural Net API: MultiHeadSelfAttention, FullyConnected, Block, Embedding, PositionalEmbedding, ReLU, Softmax, Dropout, LayerNorm, CrossEntropyLoss.
4. Other Neural Nets: For LSTMs and CNNs, use Tensorflow.js or Brain.js

Tensor Creation:

tensor(): Creates a new Tensor filled with given data
zeros(): Creates a new Tensor filled with zeros
ones(): Creates a new Tensor filled with ones
randn(): Creates a new Tensor filled with random values from a normal distribution
rand(): Creates a new Tensor filled with random values from a uniform distribution

Tensor Properties and Methods:

backward(): Performs backpropagation from this tensor backwards
zero_grad(): Clears the gradients stored in this tensor
tolist(): Returns the tensor's data as a JavaScript Array
Properties: data, length, ndims, grad

Basic Arithmetic Operations:

add(), sub(), mul(), div(): Element-wise arithmetic operations
matmul(): Matrix multiplication between two tensors
pow(): Element-wise power operation

Statistical Operations:

sum(): Gets the sum of the Tensor over a specified dimension
mean(): Gets the mean of the Tensor over a specified dimension
variance(): Gets the variance of the Tensor over a specified dimension

Tensor Manipulation:

transpose(): Transposes the tensor along two consecutive dimensions
at(): Returns elements from the tensor based on given indices
masked_fill(): Fills elements in the tensor based on a condition

Mathematical Functions:

sqrt(): Element-wise square root
exp(): Element-wise exponentiation
log(): Element-wise natural logarithm

Neural Network Layers (torch.nn):

Linear(): Applies a linear transformation
MultiHeadSelfAttention(): Applies a self-attention layer
Embedding(): Creates an embedding table for vocabulary
Activation functions: ReLU(), Softmax()

Optimization and Loss:

optim.Adam(): Adam optimizer for updating model parameters
nn.CrossEntropyLoss(): Computes Cross Entropy Loss

torch

Author

PyTorch Contributors, Leao, E. et al (2022), See also: Brain.js

Properties

_reshape()

static _reshape: (a: any, shape: any) => any;

Defined in: train/neural-net.js:2242

Parameters

Parameter	Type
a	any
shape	any

Returns

any

add()

static add: (a: any, b: any) => any;

Defined in: train/neural-net.js:2228

Parameters

Parameter	Type
a	any
b	any

Returns

any

at()

static at: (a: any, idx1: any, idx2: any) => any;

Defined in: train/neural-net.js:2240

Parameters

Parameter	Type
a	any
idx1	any
idx2	any

Returns

any

broadcast()

static broadcast: (a: any, b: any) => Tensor;

Defined in: train/neural-net.js:2251

Parameters

Parameter	Type
a	any
b	any

Returns

Tensor

div()

static div: (a: any, b: any) => any;

Defined in: train/neural-net.js:2231

Parameters

Parameter	Type
a	any
b	any

Returns

any

exp()

static exp: (a: any) => any;

Defined in: train/neural-net.js:2233

Parameters

Parameter	Type
a	any

Returns

any

getShape()

static getShape: (data: any, shape: any[]) => any[];

Defined in: train/neural-net.js:2257

Parameters

Parameter	Type	Default value
data	any	undefined
shape	any[]	[]

Returns

any[]

load()

static load: (model: any, loadedData: any) => any;

Defined in: train/neural-net.js:2253

Parameters

Parameter	Type
model	any
loadedData	any

Returns

any

log()

static log: (a: any) => any;

Defined in: train/neural-net.js:2234

Parameters

Parameter	Type
a	any

Returns

any

masked_fill()

static masked_fill: (a: any, mask: any, condition: any, value: any) => any;

Defined in: train/neural-net.js:2238

Parameters

Parameter	Type
a	any
mask	any
condition	any
value	any

Returns

any

matmul()

static matmul: (a: any, b: any) => any;

Defined in: train/neural-net.js:2232

Parameters

Parameter	Type
a	any
b	any

Returns

any

mean()

static mean: (a: any, dim: number, keepdims: boolean) => any;

Defined in: train/neural-net.js:2237

Parameters

Parameter	Type	Default value
a	any	undefined
dim	number	-1
keepdims	boolean	false

Returns

any

mul()

static mul: (a: any, b: any) => any;

Defined in: train/neural-net.js:2230

Parameters

Parameter	Type
a	any
b	any

Returns

any

neg()

static neg: (a: any) => any;

Defined in: train/neural-net.js:2229

Parameters

Parameter	Type
a	any

Returns

any

nn

static nn: object;

Defined in: train/neural-net.js:2255

Name	Type	Defined in
Block	typeof Block	train/neural-net.js:2075
CrossEntropyLoss	typeof CrossEntropyLoss	train/neural-net.js:2082
Dropout	typeof Dropout	train/neural-net.js:2080
Embedding	typeof Embedding	train/neural-net.js:2076
FullyConnected	typeof FullyConnected	train/neural-net.js:2074
LayerNorm	typeof LayerNorm	train/neural-net.js:2081
Linear	typeof Linear	train/neural-net.js:2072
Module	typeof Module	train/neural-net.js:2071
MultiHeadSelfAttention	typeof MultiHeadSelfAttention	train/neural-net.js:2073
PositionalEmbedding	typeof PositionalEmbedding	train/neural-net.js:2077
ReLU	typeof ReLU	train/neural-net.js:2078
Softmax	typeof Softmax	train/neural-net.js:2079

ones()

static ones: (shape: any, requires_grad: boolean, device: string) => Tensor;

Defined in: train/neural-net.js:2249

Parameters

Parameter	Type	Default value
shape	any	undefined
requires_grad	boolean	false
device	string	"cpu"

Returns

Tensor

optim

static optim: object;

Defined in: train/neural-net.js:2256

Name	Type	Defined in
Adam	typeof Adam	train/neural-net.js:2084

Parameter

static Parameter: typeof Parameter;

Defined in: train/neural-net.js:2227

pow()

static pow: (a: any, n: any) => any;

Defined in: train/neural-net.js:2236

Parameters

Parameter	Type
a	any
n	any

Returns

any

rand()

static rand: (shape: any, requires_grad: boolean, device: string) => Tensor;

Defined in: train/neural-net.js:2247

Parameters

Parameter	Type	Default value
shape	any	undefined
requires_grad	boolean	false
device	string	"cpu"

Returns

Tensor

randint()

static randint: (low: number, high: number, shape: number[], requires_grad: boolean) => Tensor;

Defined in: train/neural-net.js:2245

Parameters

Parameter	Type	Default value
low	number	0
high	number	1
shape	number[]	...
requires_grad	boolean	false

Returns

Tensor

randn()

static randn: (shape: any, requires_grad: boolean, device: string, xavier: boolean) => Tensor;

Defined in: train/neural-net.js:2246

Parameters

Parameter	Type	Default value
shape	any	undefined
requires_grad	boolean	false
device	string	"cpu"
xavier	boolean	false

Returns

Tensor

reshape()

static reshape: (a: any, shape: any) => any;

Defined in: train/neural-net.js:2241

Parameters

Parameter	Type
a	any
shape	any

Returns

any

save()

static save: (model: any, file: any) => string;

Defined in: train/neural-net.js:2252

Parameters

Parameter	Type
model	any
file	any

Returns

string

sqrt()

static sqrt: (a: any) => any;

Defined in: train/neural-net.js:2235

Parameters

Parameter	Type
a	any

Returns

any

tensor()

static tensor: (data: any, requires_grad: boolean, device: string) => Tensor;

Defined in: train/neural-net.js:2244

Parameters

Parameter	Type	Default value
data	any	undefined
requires_grad	boolean	false
device	string	"cpu"

Returns

Tensor

Tensor

static Tensor: typeof Tensor;

Defined in: train/neural-net.js:2226

transpose()

static transpose: (a: any, dim1: any, dim2: any) => any;

Defined in: train/neural-net.js:2243

Parameters

Parameter	Type
a	any
dim1	any
dim2	any

Returns

any

tril()

static tril: (shape: any, requires_grad: boolean, device: string) => Tensor;

Defined in: train/neural-net.js:2248

Parameters

Parameter	Type	Default value
shape	any	undefined
requires_grad	boolean	false
device	string	"cpu"

Returns

Tensor

variance()

static variance: (a: any, dim: number, keepdims: boolean) => any;

Defined in: train/neural-net.js:2239

Parameters

Parameter	Type	Default value
a	any	undefined
dim	number	-1
keepdims	boolean	false

Returns

any

zeros()

static zeros: (shape: any, requires_grad: boolean, device: string) => Tensor;

Defined in: train/neural-net.js:2250

Parameters

Parameter	Type	Default value
shape	any	undefined
requires_grad	boolean	false
device	string	"cpu"

Returns

Tensor