Operator List

name	kind	stage	since	labels	description
_conv_depthwise2d	NeuralNetwork	Beta	2.2	aten, Convolution	A depthwise convolution for the conv2d neural network function.
_functional_sym_constrain_range_for_size	Tensor	Beta	5.0	aten, KernelGen	A low-level function used in symbolic shape analysis to restrict the possible numerical range (min/max) of an unbacked symbolic integer.
_unique2	Tensor	Stable	2.1	aten	Returns the unique elements of the input tensor. This is an internal PyTorch function.
_upsample_bicubic2d_aa	NeuralNetwork	Stable	2.2	aten, Reduction	A variant of upsample() that has mode set to bicubic.
_upsample_nearest_exact1d	NeuralNetwork	Beta	5.0	aten, Reduction	Increases the length of a 1D tensor using nearest-neighbor interpolation, ensuring the output aligns with library-standard algorithms like PIL.
abs	Math	Stable	1.0	aten, pointwise	Computes the absolute value of each element in input. This is a simple wrapper of the existing torch abs operator.
abs_	Math	Stable	2.2	aten, pointwise	The in-place version of abs(), which is a simple wrapper of the Torch abs operator.
absolute	Math	Beta	5.0	aaten, KernelGen	This is an alias for abs() with the low-level operations implemented by invoking low-level Torch operators.
acos	Math	Stable	5.0	aten, pointwise	Returns a new tensor with the arccosine (in radians) of each element in input.
add	Math	Stable	1.0	aten, pointwise	Add a scalar or tensor to self tensor. If both alpha and other are specified, each element of other is scaled by alpha before being used.
add_	Math	Stable	2.2	aten, pointwise	The in-place version of add().
addcdiv	LinearAlg	Stable	4.0	aten, pointwise	Performs the element-wise division of tensor1 by tensor2, multiplies the result by the scalar value and adds it to input.
addcmul	LinearAlg	Stable	4.0	aten, pointwise	Performs the element-wise multiplication of tensor1 by tensor2, multiplies the result by the scalar value and adds it to input.
addmm	BLAS	Stable	1.0	aten	Performs a matrix multiplication of the matrices mat1 and mat2. The matrix input is added to the final result.
addmm_out	BLAS	Stable	4.0	aten	Performs a matrix multiplication of the matrices mat1 and mat2. The matrix input is added to the final result.
addmv	LinearAlg	Stable	4.0	aten	Performs a matrix-vector product of the matrix mat and the vector vec. The vector input is added to the final result.
addmv_out	LinearAlg	Stable	4.0	aten	Performs a matrix-vector product of the matrix mat and the vector vec. The vector input is added to the final result.
addr	LinearAlg	Stable	4.0	aten	Performs the outer-product of vectors vec1 and vec2 and adds it to the matrix input.
alias_copy	Tensor	Beta	5.0	aten, KernelGen	Creates a new tensor that shares the same storage data as the original tensor, but without preserving the original tensor’s metadata (like shape or strides) in a way that links future mutations.
alias_copy_out	Tensor	Beta	5.0	aten, KernelGen	A variant of alias_copy() that assigns the output to the out tensor.
all	Math	Stable	2.0	aten, Reduction	Tests if all elements in input evaluate to True.
all_dim	Math	Stable	2.0	aten, Reduction	For each row of input in the given dimension dim, returns True if all elements in the row evaluate to True and False otherwise.
all_dims	Math	Stable	2.0	aten, Reduction	A variant of all.
allclose	Math	Stable	2.1	aten	This function checks if input and other satisfy a condition specified via atol and rtol elementwise, for all elements of input and other.
amax	LinearAlg	Stable	2.0	aten, Reduction	Returns the maximum value of each slice of the input tensor in the given dimension(s) dim.
angle	Math	Stable	3.0	aten, pointwise	Computes the element-wise angle (in radians) of the given input tensor.
any	Math	Stable	2.0	aten, Reduction	Tests if any element in input evaluates to True.
any_dim	Math	Stable	2.0	aten, Reduction	For each row of input in the given dimension dim, returns True if any element in the row evaluate to True and False otherwise.
any_dims	Math	Stable	2.0	aten, Reduction	For each row of input in the given dimensions in dims, returns True if any element in the row evaluate to True and False otherwise. The dims contains tuple of ints indicating the dimensions to reduce.
apply_repetition_penalties	NeuralNetwork	Stable	5.0	fused, vLLM	Modifies logit tensors in place to penalize tokens that have already appeared in the generated sequence.
apply_rotary_pos_emb	NeuralNetwork	Stable	2.0	fused	A method to incorporate positional information into the Transformer architecture. Rotary Positional Embedding (RoPE) applies position-dependent rotation to the query (Q) and key (K) vectors before computing the attention score.
arange	Tensor	Stable	2.1	aten	Returns a 1-D tensor of size ceiling((end−start)/step) with values from the interval [start, end) taken with common difference step beginning from start.
arange_start	tensor	Stable	2.1	aten	A variant of arange, with start and/or step specified.
arcsinh	Math	Beta	5.0	aten, KernelGen	Performs an element-wise inverse hyperbolic sine computation on the given tensor.
arcsinh_	Math	Beta	5.0	aten, KernelGen	The in-place version of arcsinh().
arcsinh_out	Math	Beta	5.0	aten, KernelGen	A variant of arcsinh that allows the output to be assigned to the out tensor.
arctanh_	Math	Beta	5.0	aten, KernelGen	Computes the element-wise inverse hyperbolic tangent of a given input tensor. This is an in-place version.
argmax	LinearAlg	Stable	2.0	aten, Reduction	Returns the indices of the maximum value of all elements in the input tensor.
argmin	LinearAlg	Stable	2.2	aten, Reduction	Returns the indices of the minimum value(s) of the flattened tensor or along a dimension.
asinh_	Math	Beta	5.0	aten, KernelGen	Computes the inverse hyperbolic sine for each element of a tensor in-place.
atan	Math	Stable	4.0	aten, pointwise	Returns a new tensor with the arctangent of the elements (in radians) in the input tensor.
atan_	Math	Stable	4.0	aten, pointwise	The in-place version of atan().
avg_pool2d	NeuralNetwork	Stable	4.1	nn.functional	Applies 2D average-pooling operation in kH \mul kW regions by step size sH \mul sW steps. The number of output features is equal to the number of input planes.
avg_pool2d_backward	NeuralNetwork	Stable	4.1	aten	The backward version of avg_pool2d().
baddbmm	BLAS	Stable	4.1	aten	Performs a batch matrix-matrix product of matrices in batch1 and batch2. input is added to the final result. batch1 and batch2 must be 3-D tensors each containing the same number of matrices.
batch_norm	NeuralNetwork	Stable	3.0	aten	An internal operator used for implementing the BatchNorm functionality.
batch_norm_backward	NeuralNetwork	Stable	3.0	aten	The backward version of batch_norm().
bincount	Reduction	Stable	5.0	aten, pointwise	Count the frequency of each value in an array of non-negative integers.
bitwise_and_scalar	Math	Stable	2.0	aten, pointwise	Computes the bitwise AND of input and other scalar.
bitwise_and_scalar_	Math	Stable	2.2	aten, pointwise	The in-place, scalar version of bitwise_and().
bitwise_and_scalar_tensor	Math	Stable	2.0	aten, pointwise	A variant of bitwise_and().
bitwise_and_tensor	Math	Stable	2.0	aten, pointwise	The Tensor method version of bitwise_and().
bitwise_and_tensor_	Math	Stable	2.2	aten, pointwise	The in-place, Tensor method version of bitwise_and().
bitwise_left_shift	Math	Stable	4.0	aten, pointwise	Computes the left arithmetic shift of input by other bits.
bitwise_not	Math	Stable	2.0	aten, pointwise	Computes the bitwise NOT of the given input tensor.
bitwise_not_	Math	Stable	2.2	aten, pointwise	The in-place version of bitwise_not().
bitwise_or_scalar	Math	Stable	2.0	aten, pointwise	Computes the bitwise OR of scalars input and other.
bitwise_or_scalar_	Math	Stable	2.2	aten	The in-place version of bitwise_or_scalar.
bitwise_or_scalar_tensor	Math	Stable	2.0	aten, pointwise	Computes the bitwise OR of input and other.
bitwise_or_tensor	Math	Stable	2.0	aten, pointwise	Computes the bitwise OR of input and other, this is the Tensor method variant.
bitwise_or_tensor_	Math	Stable	2.2	aten, pointwise	The in-place version of bitwise_or_tensor().
bitwise_right_shift	Math	Stable	4.0	aten, pointwise	Computes the right arithmetic shift of input by other bits.
bmm	BLAS	Stable	1.0	aten	Performs a batch matrix-matrix product of matrices stored in input and mat2.
bmm_out	BLAS	Stable	5.0	aten	Performs a batch matrix-matrix product of matrices stored in input and mat2. This is a variant of bmm with out specified.
cat	Tensor	Stable	2.2	aten	Concatenates the given sequence of tensors in tensors in the given dimension.
ceil	Math	Stable	5.0	aten, pointwise	Returns a new tensor with the ceil of the elements of input, the smallest integer greater than or equal to each element.
ceil_	Math	Stable	5.0	aten, pointwise	The in-place version of ceil().
ceil_out	Math	Stable	5.0	aten, pointwise	A variant of ceil() with out specified.
celu	NeuralNetwork	Stable	4.0	aten, nn.functional, pointwise	Applies the quantized CELU (Continuously Differentiable Exponential Linear Unit) activation function element-wise.
celu_	NeuralNetwork	Stable	4.0	aten, nn.functional, pointwise	The in-place version of celu().
chunk_gated_delta_rule_fwd	Attention	Alpha	5.0	fused, FLA	The forward case for ChunkGatedDeltaRuleFunction with Flash Linear Attention (FLA).
clamp	Math	Stable	2.0	aten, pointwise	Clamps all elements in input into the range [min, max].
clamp_	Math	Stable	2.2	aten, pointwise	The in-place version of clamp().
clamp_min	Math	Stable	4.0	aten, pointwise	A variant of clamp() with min set to min.
clamp_min_	Math	Stable	4.0	aten, pointwise	The in-place version of clamp_().
clamp_tensor	Math	Stable	2.0	aten, pointwise	The tensor version of clamp().
clamp_tensor_	Math	Stable	2.2	aten, pointwise	The in-place, tensor version of clamp().
concat_and_cache_mla	Attention	Beta	3.0	fused, MLA	Writes the latent and RoPE value into KV cache for Multi-head Latent Attention forward case.
constant_pad_nd	NeuralNetwork	Stable	2.2	aten, IR	Pads the input tensor boundaries with a constant value. This is an IR representation, not a public API.
contiguous	Tensor	Beta	4.1	aten	Returns a contiguous in memory tensor containing the same data as self tensor. Introduced in v2.2 and removed from exposed list in v4.1.
conv1d	Convolution	Stable	4.2	aten	Applies a 1D convolution over a quantized 1D input composed of several input planes.
conv2d	Convolution	Stable	4.2	aten	Applies a 2D convolution over a quantized 2D input composed of several input planes.
conv3d	Convolution	Stable	4.2	aten	Applies a 3D convolution over a quantized 3D input composed of several input planes.
copy	Tensor	Beta	4.2	aten, pointwise	As a wrapper of copy_, this operator copies elements from src to out using given template for shapes.
copy_	Tensor	Stable	4.1	aten, pointwise	Copies the elements from src into self tensor and returns self`.
cos	Math	Stable	2.0	aten, pointwise	Returns a new tensor with the cosine of the elements of input given in radians.
cos_	Math	Stable	2.2	aten, pointwise	The in-place version of cos().
count_nonzero	Data	Stable	2.2	aten, Reduction	Counts the number of non-zero values in the tensor input along the given dim. If no dim is specified then all non-zeros in the tensor are counted.
cross_entropy_loss	NeuralNetwork	Removed	3.0	fused, Reduction	Computes the cross entropy loss between input logits and target.
cummax	Math	Stable	3.0	aten, Reduction	Returns a named tuple (values, indices) where values is the cumulative maximum of elements of input in the dimension dim. And indices is the index location of each maximum value found in the dimension dim.
cummin	Math	Stable	2.2	aten, Reduction	Returns a named tuple (values, indices) where values is the cumulative minimum of elements of input in the dimension dim. And indices is the index location of each minimum value found in the dimension dim.
cumsum	LinearAlg	Stable	1.0	aten
cumsum_out	Reduction	Stable	3.0	aten
cutlass_scaled_mm	LinearAlg	Beta	5.0	fused, vLLM
dgeglu	NeuralNetwork	Alpha	4.2	fused, Transformer	Gaussian Error Gated Linear Unit with GELU activation instead of sigmoid function. This is for the backward case.
diag	Tensor	Stable	2.2	aten	If input is a vector (1-D tensor), then returns a 2-D square tensor with the elements of input as the diagonal. If input is a matrix (2-D tensor), then returns a 1-D tensor with the diagonal elements of input.
diag_embed	Tensor	Stable	2.2	aten, pointwise	Creates a tensor whose diagonals of certain 2D planes (specified by dim1 and dim2) are filled by input. To facilitate creating batched diagonal matrices, the 2D planes formed by the last two dimensions of the returned tensor are chosen by default.
diagonal_backward	LinearAlg	Stable	2.2	aten, pointwise	A diagonal operation returns a partial view of input with the its diagonal elements with respect to dim1 and dim2 appended as a dimension at the end of the shape. This is the backward case for diagonal().
digamma_	Math	Beta	5.0	aten, KernelGen	Computes the in-place digamma function, which is the logarithmic derivative of the Gamma function.
dispatch_fused_moe_kernel	MoE	Beta	5.0	fused, Activation, vLLM	Accelerates neural network training by combining token routing (dispatch/all-to-all communication), expert computation (GEMM), and result aggregation into a single GPU kernel.
div_mode	Math	Stable	1.0	aten, pointwise	Divides each element of the input by the corresponding element of other. An optional rounding_mode can be specified.
div_mode_	Math	Stable	2.2	aten, pointwise	The in-place version of div_mode().
dot	BLAS	Stable	3.0	aten	Computes the dot product of two 1D tensors.
dreglu	NeuralNetwork	Alpha	4.2	fused, Transformer	Rectified Gated Linear Unit is a variant of GLU that uses ReLU instead of the sigmoid function for gating. This is the backward case. This operator was introduced in v4.2 but not exposed.
dropout	NeuralNetwork	Stable	1.0	aten, nn.functional	An internal IR for implementing torch.nn.functional.dropout.
dropout_backward	NeuralNetwork	Stable	3.0	aten, nn.functional	The backward case of dropout().
dswiglu	NeuralNetwork	Alpha	5.0	fused, Transformer	Swish-Gated Linear Unit, a variant of GLU with the Swish activation function. This is for the backward case.
elu	NeuralNetwork	Stable	2.2	aten, nn.functional, pointwise	Apply the Exponential Linear Unit (ELU) function element-wise.
elu_	NeuralNetwork	Stable	4.0	aten, pointwise	The in-place version of elu().
elu_backward	NeuralNetwork	Stable	4.0	aten, pointwise	The backward version of elu().
embedding	NeuralNetwork	Stable	2.1	aten, nn.functional	Generate a simple lookup table that looks up embeddings in a fixed dictionary and size. Note that the parameter sequence differs from torch.nn.functional.embedding.
embedding_backward	NeuralNetwork	Stable	3.0	aten	The backward version of embedding().
embedding_dense_backward	NeuralNetwork	Stable	5.0	aten	Calculates the gradient of the weight matrix for a dense embedding layer during backpropagation.
eq	Math	Stable	2.0	aten, pointwise	Computes element-wise equality.
eq_scalar	Math	Stable	2.0	aten, pointwise	Computes equality between scalars.
equal	Math	Stable	5.0	aten, Reduction	Returns True if two tensors have the same size and elements, False otherwise.
erf	Science	Stable	2.1	aten	Computes the error function of input.
erf_	Science	Stable	2.2	aten, pointwise	The in-place version of erf().
exp	Math	Stable	1.0	aten, pointwise	Returns a new tensor with the exponential of the elements of the input tensor input.
exp_	Math	Stable	2.2	aten, pointwise	The in-place version of exp().
exp_out	Math	Stable	4.1	aten, pointwise	A variant of exp2(), with out specified.
exp2	Math	Stable	4.0	aten, pointwise	Computes the base two exponential function of input.
exp2_	Math	Stable	4.0	aten, pointwise	The in-place version of exp2().
exponential_	Distribution	Stable	2.1	aten	Fills self tensor with elements drawn from a PDF (probability density function).
eye	LinearAlg	Stable	3.0	aten, Reduction	Returns a 2-D tensor with ones on the diagonal and zeros elsewhere.
eye_m	LinearAlg	Stable	3.0	aten, Reduction	Triton-based implementation of torch.eye_m(n, m), using 2D tiles to split the matrix into blocks.
fill_scalar	Tensor	Stable	2.2	aten, pointwise	Fills a scalar with the specified value.
fill_scalar_	Tensor	Stable	2.2	aten, pointwise	The in-place version of fill_scalar().
fill_scalar_out	Tensor	Stable	5.0	aten, pointwise	A variant of fill_scalar() that assigns the output to an out tensor.
fill_tensor	Tensor	Stable	2.2	aten, pointwise	Fills a tensor with the specified value.
fill_tensor_	Tensor	Stable	2.2	aten, pointwise	The in-place version of fill_tensor().
fill_tensor_out	Tensor	Stable	5.0	aten, pointwise	A variant of fill_tensor() that assigns the output to an out tensor.
flash_attention_forward	NeuralNetwork	Stable	3.0	aten
flash_attn_varlen_func	NeuralNetwork	Stable	3.1	aten, Attention, FlashAttention	Compute attention for sequences of variable lengths within a single batch. Eliminating the need for padding.
flash_mla	NeuralNetwork	Stable	3.0	fused, Attention, vLLM	A variant of Multi-head Latent Attention (MLA).
flip	Tensor	Stable	2.1	aten, pointwise	Reverse the order of an n-D tensor along given axis in dims.
floor_	Math	Beta	5.0	aten, KernelGen	Performs an in-place element-wise floor operation, rounding each element of a tensor down to the nearest integer less than or equal to itself.
floor_divide	Math	Stable	2.1	aten	Computes input divided by other, elementwise, and floors the result.
floor_divide_	Math	Stable	2.2	aten	Computes input divided by other, elementwise, and floors the result.
fmin	Math	Beta	5.0	aten, KernelGen	Computes the element-wise minimum of two tensors, specially handling NaN values by prioritizing the numerical value. Unlike minimum(), if one input is NaN and the other is a number, fmin() returns the number. It supports broadcasting, type promotion, and operates on both CPU and GPU.
fmin_out	Math	Beta	5.0	aten, KernelGen	A variant of fmin() that assigns the output to the out tensor.
full	Tensor	Stable	2.1	aten, pointwise	Creates a tensor of size size filled with fill_value. The tensor’s dtype is inferred from fill_value.
full_like	Tensor	Stable	2.1	aten, pointwise	Returns a tensor with the same size as input filled with fill_value.
fused_add_rms_norm	NeuralNetwork	Stable	2.0	fused, Normalization
fused_recurrent_gated_delta_rule_fwd	Attention	Alpha	5.0	fused, FLA	The forward case for fused_recurrent_gated_delta_rule used in Flash Linear Attention (FLA).
gather	Tensor	Stable	2.2	aten, Reduction	Gathers values along an axis specified by dim.
gather_backward	Tensor	Stable	2.2	aten, Reduction	The backward version of gather().
ge	Math	Stable	2.0	aten, pointwise	Computes input is greater or equal to other element-wise.
ge_scalar	Math	Stable	2.0	aten, pointwise	The scalar version of ge().
geglu	NeuralNetwork	Alpha	4.2	fused, Activation, Transformer	Gaussian Error Gated Linear Unit with GELU activation instead of sigmoid function.
gelu	NeuralNetwork	Stable	1.0	aten, pointwise, Activation, nn.functional	Apply Cumulative Distribution Function for Gaussian Distribution function element-wise.
gelu_	NeuralNetwork	Stable	2.2	aten, Activation, pointwise	The in-place version of gelu().
gelu_and_mul	NeuralNetwork	Stable	2.0	fused, pointwise, Activation	An activation function for GeGLU.
gelu_backward	NeuralNetwork	Stable	3.0	aten, Activation, pointwise	The backward version of gelu().
get_scheduler_metadata	Attention	Stable	4.0	vLLM	Computes scheduling metadata for attention work partitioning so that CPU computations can be routed to ISA-specific kernel implmentations. The metadata is stored in a tensor.
glu	NeuralNetwork	Stable	3.0	aten, Activation, pointwise	Gated Linear Unit activation for modulating the output of a linear transformation with a gate.
glu_backward	NeuralNetwork	Stable	4.0	aten, Activation, pointwise	The backward version of glu().
group_norm	NeuralNetwork	Stable	2.0	aten, Reduction	An internal IR for applying Group Normalization for last certain number of dimensions.
group_norm_backward	NeuralNetwork	Stable	3.0	aten, Reduction	The backward case for group_norm().
grouped_topk	MoE	Stable	5.0	fused, vLLM	A specialized routing mechanism used in Mixture-of-Experts (MoE) models (like DeepSeek-V3/R1) to select top-k experts by first grouping them, rather than selecting globally.
gt	Math	Stable	2.0	aten, pointwise	Computes that input is greater than other element-wise.
gt_scalar	Math	Stable	2.0	aten, pointwise	The scalar version of gt().
hardsigmoid	NeuralNetwork	Beta	5.0	aten, pointwise, nn.functional, Activation, KernelGen	An activation function that provides a piecewise linear approximation of the standard sigmoid function, mapping inputs to a range between 0 and 1.
hardsigmoid_out	NeuralNetwork	Beta	5.0	aten, pointwise, nn.functional, Activation, KernelGen	A variant of hardsigmoid that supports an output tensor to receive the result.
hardswish_	NeuralNetwork	Beta	5.0	aten, pointwise, KernelGen, Activation	Applies the Hard Swish activation function, commonly used in models like MobileNetV3 to improve accuracy while reducing computational cost compared to traditional Swish. This is an in-place version.
hstack	Tensor	Stable	2.2	aten	Stack tensors in sequence horizontally (column wise). This is equivalent to concatenation along the first axis for 1-D tensors, and along the second axis for all other tensors.
hypot	Math	Beta	5.0	aten, KernelGen	Given the legs of a right triangle, return its hypotenuse. The shapes of both input tensors must be broadcastable.
hypot_out	Math	Beta	5.0	aten, KernelGen	Given the legs of a right triangle, return its hypotenuse. The shapes of both input tensors must be broadcastable. This is a variant of hypot that allows the output to be a different tensor.
i0	Math	Beta	5.0	aten, KernelGen	Computes the modified Bessel function of the first kind of order zero element-wise for a given input tensor.
i0_	Math	Beta	5.0	aten, KernelGen	The inplace version of i0.
i0_out	Math	Beta	5.0	aten, KernelGen	A variant of i0 that assigns the output to the out tensor.
index	Reduction	Stable	4.2	aten	Extract, access or modify specific elements, slices, or subsets of data within a tensor. The location of data is specified for each dimension, starting from index 0.
index_add	Tensor	Stable	2.2	aten	Accumulate the elements of alpha times source into the input tensor by adding to the indices in the order given in index.
index_add_	Tensor	Stable	4.0	aten	The in-place version of index_add().
index_put	Tensor	Stable	2.2	aten	Puts values from the tensor values into the tensor input using the indices specified in indices (which is a tuple of Tensors).
index_put_	Tensor	Stable	3.0	aten	The in-place version of index_put().
index_select	Tensor	Stable	2.1	aten	Returns a new tensor which indexes the input tensor along dimension dim using the entries in index.
inplace_fused_experts	MoE	Beta	5.0	fused, Activation, vLLM	This operator writes output directly to hidden_states.
instance_norm	NeuralNetwork	Removed	3.0	fused	Apply Instance Normalization independently for each channel in every data sample within a batch.
isclose	Math	Stable	2.1	aten, pointwise	Returns a new tensor with boolean elements representing if each element of input is “close” to the corresponding element of other. The closeness is defined with rtol and atol.
isfinite	Math	Stable	2.1	aten, pointwise	Returns a new tensor with boolean elements representing if each element is finite or not.
isin	Data	Stable	2.2	aten	Tests if each element of elements is in test_elements. Returns a boolean tensor of the same shape as elements that is True for elements in test_elements and False otherwise.
isinf	Math	Stable	2.0	aten, pointwise	Tests if each element of input is infinite (positive or negative infinity) or not.
isnan	Math	Stable	2.0	aten, pointwise	Returns a new tensor with boolean elements representing if each element of input is NaN or not.
kron	LinearAlg	Stable	2.2	aten	Computes the Kronecker product of input and other.
layer_norm	NeuralNetwork	Stable	1.0	aten	An internal IR for applying Layer Normalization for last certain number of dimensions.
layer_norm_backward	Reduction	Stable	3.0	aten	The backward case for layer_norm().
le	Math	Stable	2.0	aten, pointwise	Computes that input is less than or equal to other element-wise.
le_scalar	Math	Stable	2.0	aten	The scalar version of le().
lerp_scalar	LinearAlg	Stable	3.0	aten, pointwise	The scalar version of lerp().
lerp_scalar_	LinearAlg	Stable	3.0	aten, pointwise	The in-place, scalar version of lerp().
lerp_tensor	LinearAlg	Stable	3.0	aten, pointwise	Performs a linear interpolation of two tensors start (given by input) and end based on a scalar or tensor weight and returns the resulting out tensor.
lerp_tensor_	LinearAlg	Stable	3.0	aten, pointwise	The in-place version of lerp().
lift_fresh_copy	Tensor	Beta	5.0	aten, KernelGen	Creates a new, independent copy of a tensor within a compiled graph.
linspace	Data	Stable	2.2	aten	Creates a one-dimensional tensor of size steps whose values are evenly spaced from start to end, inclusive.
log	Math	Stable	2.2	aten, pointwise	Returns a new tensor with the natural logarithm of the elements of input.
log_sigmoid	NeuralNetwork	Stable	2.2	aten, pointwise, nn.functional	Applies the Logsigmoid function element-wise.
log_softmax	NeuralNetwork	Stable	3.0	aten, Reduction	An internal IR for applying a softmax followed by a logarithm.
log_softmax_backward	NeuralNetwork	Stable	3.0	aten, Reduction	The backward case for log_softmax().
log1p_	Math	Beta	5.0	aten, KernelGen	Computes the natural logarithm of 1+x(y_i=log_e(x_i+1)) for each element in the input tensor in-place.
logaddexp	Math	Beta	5.0	aten, pointwise, KernelGen	Computes the element-wise logarithm of the sum of the exponentials of two input tensors.
logaddexp_out	Math	Beta	5.0	aten, pointwise, KernelGen	A variant of logaddexp that allows the output to be assigned to an out tensor.
logical_and	Math	Stable	2.2	aten, pointwise	Computes the element-wise logical AND of the given input tensors. Zeros are treated as False and nonzeros are treated as True.
logical_and_	Math	Stable	5.0	aten, pointwise	The in-place version of logical_and().
logical_not	Math	Stable	2.2	aten, pointwise	Computes the element-wise logical NOT of the given input tensor.
logical_or	Math	Stable	2.2	aten, pointwise	Computes the element-wise logical OR of the given input tensors.
logical_or_	Math	Stable	5.0	aten, pointwise	The in-place version of logical_or().
logical_xor	Math	Stable	2.2	aten, pointwise	Computes the element-wise logical XOR of the given input tensors.
logit	LinearAlg	Beta	5.0	aten, pointwise, KernelGen	Returns a new tensor with the logit of the elements of input. input is clamped to [eps, 1-eps] when eps is not None. When eps is None and input<0 or input>1, the function will yield NaN.
logit_	LinearAlg	Beta	5.0	aten, pointwise, KernelGen	The in-place version of logit().
logit_out	LinearAlg	Beta	5.0	aten, pointwise, KernelGen	A variant of logit that allows the output to be assigned to another tensor.
logspace	tensor	Stable	4.0	aten	Creates a one-dimensional tensor of size steps whose values are evenly spaced from base^start to base^end, inclusive, on a logarithmic scale with base base.
lt	Math	Stable	2.0	aten, pointwise	Computes that input is less than other element-wise.
lt_scalar	Math	Stable	2.0	aten, pointwise	The scalar version of lt.
masked_fill	Tensor	Stable	2.2	aten, pointwise	Fills elements of given tensor with value where mask is True.
masked_fill_	tensor	Stable	2.2	aten, pointwise	The in-place version of masked_fill().
masked_scatter	tensor	Stable	4.2	aten	Copies elements from source into the given tensor at positions where the mask is True.
masked_scatter_	tensor	Stable	4.2	aten	The in-place version of masked_scatter().
masked_select	Tensor	Stable	2.1	aten	Returns a new 1-D tensor which indexes the input tensor according to the boolean mask mask which is a BoolTensor.
max	LinearAlg	Stable	2.0	aten, Reduction	Returns the maximum value of all elements in the input tensor.
max_dim	LinearAlg	Stable	2.0	aten, Reduction	Returns a namedtuple (values, indices) where values is the maximum value of each row of the input tensor in the given dimension dim. And indices is the index location of each maximum value found (argmax).
max_pool2d_backward	IR	Stable	4.0	aten	Applies a 2D max pooling over an input signal composed of several input planes. This is an IR representation rather than a public API and it is for the backward step.
max_pool2d_with_indices	IR	Stable	4.0	aten	Applies a 2D max pooling over an input signal composed of several input planes. This is an IR representation rather than a public API.
maximum	Math	Stable	2.1	aten, pointwise	Computes the element-wise maximum of input and other.
mean	LinearAlg	Stable	1.0	aten, Reduction	Returns the mean value of all elements in the input tensor. Input must be floating point or complex.
mean_dim	Reduction	Stable	2.0	aten	Returns the mean value of each row of the input tensor in the given dimension dim. If dim is a list of dimensions, reduce over all of them.
min	LinearAlg	Stable	2.0	aten, Reduction	Returns the minimum value of all elements in the input tensor.
min_dim	LinearAlg	Stable	2.0	aten, Reduction	Returns a namedtuple (values, indices) where values is the minimum value of each row of the input tensor in the given dimension dim. And indices is the index location of each minimum value found (argmin).
minimum	Math	Stable	2.1	aten, pointwise	Computes the element-wise minimum of input and other.
mm	BLAS	Stable	1.0	aten	Performs a matrix multiplication of the two input matrices.
mm_out	BLAS	Stable	3.0	aten	A variant of mm() with out specified.
moe_align_block_size	MoE	Stable	4.0	fused, Reduction, vLLM	Aligns the token distribution across experts to be compatible with block size for matrix multiplication.
moe_align_block_size_triton	MoE	Stable	4.2	fused, Reduction, vLLM	Aligns the token distribution across experts to be compatible with block size for matrix multiplication. This is the Triton version.
moe_sum	MoE	Removed	5.0	fused, Reduction, vLLM	An implementation of Mixture of Experts (MoE) with sum-based aggregation instead of the more common weighted average.
mse_loss	NeuralNetwork	Stable	2.2	aten, pointwise, nn.functional	Compute the element-wise mean squared error, with optional weighting.
mul	Math	Stable	1.0	aten, pointwise	Multiplies input by other.
mul_	Math	Stable	2.2	aten, pointwise	The in-place version of mul().
multinomial	Distribution	Stable	2.1	aten	Returns a tensor where each row contains num_samples indices sampled from the multinomial probability distribution located in the corresponding row of tensor input.
mv	BLAS	Stable	2.0	aten	Performs a matrix-vector product of the matrix input and the vector vec.
nan_to_num	Math	Stable	3.0	aten, pointwise	Replaces NaN, positive infinity, and negative infinity values in input with the values specified by nan, posinf, and neginf, respectively.
ne	Math	Stable	2.0	aten, pointwise	Computes that input is not equal to other element-wise.
ne_scalar	Math	Stable	2.0	aten, pointwise	The scalar version of ne().
neg	Math	Stable	2.0	aten, pointwise	Returns a new tensor with the negative of the elements of input.
neg_	Math	Stable	2.2	aten, pointwise	The in-place version of neg().
nll_loss2d_backward	NeuralNetwork	Stable	2.2	aten, IR	An internal IR for supporting torch.nn.NLLLoss2d, which has been deprecated and is now integrated into the standard torch.nn.NLLLoss. This is the backward case.
nll_loss2d_forward	NeuralNetwork	Stable	2.2	aten, IR	An internal IR for supporting torch.nn.NLLLoss2d, which has been deprecated and is now integrated into the standard torch.nn.NLLLoss. This is the forward case.
nll_loss_backward	NeuralNetwork	Stable	2.2	aten, IR	Compute the negative log likelihood loss. This is the backward case.
nll_loss_forward	NeuralNetwork	Stable	2.2	aten, IR	Compute the negative log likelihood loss. This is the forward case.
nll_loss_nd_backward	NeuralNetwork	Stable	5.0	aten	Measures the performance of a classification model by penalizing low probabilities for correct classe.s This computes the gradients of this loss with respect to model parameters using automatic differentiation.
nll_loss_nd_forward	NeuralNetwork	Stable	5.0	aten	Measures the performance of a classification model by calculating the negative log probability of the true class. This defines the computation flow, transforming input data through layers to produce output predictions.
nonzero	Data	Stable	2.1	aten	Returns a 2-D tensor where each row is the index for a nonzero value. When as_tuple is explicitly set to True, this returns a tuple of 1-D index tensors, allowing for advanced indexing of all nonzero values.
normal_	Distribution	Stable	5.0	aten, pointwise	Returns a tensor of random numbers drawn from separate normal distributions whose mean and standard deviation are given. This is one of the variants that takes a float mean and a float std.
normal_float_tensor	Distribution	Stable	2.1	aten, pointwise	Returns a tensor of random numbers drawn from separate normal distributions whose mean and standard deviation are given. This is one of the variants that takes a float mean and a tensor std.
normal_tensor_float	Distribution	Stable	2.1	aten, pointwise	Returns a tensor of random numbers drawn from separate normal distributions whose mean and standard deviation are given. This is one of the variants that takes a tensor mean and a float std.
normal_tensor_tensor	Distribution	Stable	2.1	aten, pointwise	Returns a tensor of random numbers drawn from separate normal distributions whose mean and standard deviation are given. This is one of the variants that takes a tensor mean and a tensor std.
normed_cumsum	Reduction	Stable	2.1	aten	Get the normalized cumulative sum where each step is divided by the total sum of the dataset, resulting in values ranging from 0 to 1. Internally used by the multinomial operator.
one_hot	NeuralNetwork	Stable	5.0	aten, nn.functional	Takes LongTensor with index values of shape () and returns a tensor of shape (, num_classes) that have zeros everywhere except where the index of last dimension matches the corresponding value of the input tensor, in which case it will be 1.
ones	Tensor	Stable	2.1	aten	Returns a tensor filled with the scalar value 1, with the shape defined by the variable argument size.
ones_like	Tensor	Stable	2.1	aten	Returns a tensor filled with the scalar value 1, with the same size as input.
outer	BLAS	Removed	3.0	fused	Computes outer product of self and the input vector. If the self tensor is a vector of size n and the input tensor is a vector of size m, the out tensor (if specified) must be a matrix of size n * m.
outplace_fused_experts	MoE	Beta	5.0	fused, Activation, vLLM	This operator allocates and returns a new output tensor.
pad	NeuralNetwork	Stable	2.1	aten, pointwise, nn.functional	This pads a tenor using the specified mode.
per_token_group_quant_fp8	Quantization	Alpha	4.2	vLLM	Function to perform per-token-group quantization on an input tensor x. It converts the tensor values into signed float8 values and returns the quantized tensor along with the scaling factor used for quantization.
pixel_unshuffle	NeuralNetwork	Beta	5.0	aten, KernelGen	Rearranges elements from a low-resolution feature map with many channels into a higher-resolution feature map with fewer channels.
pixel_unshuffle_out	NeuralNetwork	Beta	5.0	aten, KernelGen	A variant of pixel_unshuffle that assigns the output to the out tensor.
polar	Math	Stable	3.0	aten, pointwise	Constructs a complex tensor whose elements are Cartesian coordinates corresponding to the polar coordinates with absolute value abs and angle angle.
pow_scalar	Math	Stable	1.0	aten	Takes the power of each element in input with exponent and returns a tensor with the result. The input is a single float, while the exponent is a tensor.
pow_tensor_scalar	Math	Stable	1.0	aten, pointwise	Takes the power of each element in input with exponent and returns a tensor with the result. The input is a tensor, while the exponent is a float.
pow_tensor_scalar_	Math	Stable	2.2	aten, pointwise	This is the in-place version of pow_tensor_scalar().
pow_tensor_tensor	Math	Stable	1.0	aten, pointwise	Takes the power of each element in input with exponent and returns a tensor with the result. The input is a tensor, while the exponent is also a tensor.
pow_tensor_tensor_	Math	Stable	2.2	aten, pointwise	This is the in-place version of pow_tensor_tensor().
prelu	NeuralNetwork	Beta	5.0	aten, Activation, pointwise, nn.functional, KernelGen	An activation function used in neural networks that improves upon ReLU (Rectified Linear Unit) by allowing the network to learn the slope of negative inputs. It performs an element-wise operation that keeps positive values and scales negative values by a learnable parameter.
prod	LinearAlg	Stable	2.0	aten, Reduction	Returns the product of all elements in the input tensor.
prod_dim	Reduction	Stable	2.0	aten	Returns the product of each row of the input tensor in the given dimension dim.
quantile	Data	Stable	2.2	aten	Computes the q-th quantiles of each row of the input tensor along the dimension dim.
rand	Distribution	Stable	2.1	aten	Returns a tensor filled with random numbers from a uniform distribution on the interval [0,1).
rand_like	Distribution	Stable	2.1	aten	Returns a tensor with the same size as input that is filled with random numbers from a uniform distribution on the interval [0,1).
randn	Distribution	Stable	2.1	aten	Returns a tensor filled with random numbers from a normal distribution with mean 0 and variance 1 (also called the standard normal distribution).
randn_like	Distribution	Stable	2.1	aten	Returns a tensor with the same size as input that is filled with random numbers from a normal distribution with mean 0 and variance 1.
randperm	Distribution	Stable	2.2	aten	Returns a random permutation of integers from 0 to n - 1.
reciprocal	Math	Stable	1.0	aten, pointwise	Returns a new tensor with the reciprocal of the elements of input.
reciprocal_	Math	Stable	2.2	aten, pointwise	This is the in-place version of reciprocal().
reflection_pad1d	NeuralNetwork	Beta	5.0	aten, pointwise, KernelGen	Pads the input 3D or 2D tensor (typically representing signals or sequences) by reflecting the boundary values at the edges.
reflection_pad1d_out	NeuralNetwork	Beta	5.0	aten, pointwise, KernelGen	A variant of reflection_pad1d that assigns the output to out tensor.
reflection_pad2d	NeuralNetwork	Beta	5.0	aten, pointwise, KernelGen	Pads the input 3D or 2D tensor (typically representing signals or sequences) by reflecting the boundary values at the both edges.
reflection_pad2d_out	NeuralNetwork	Beta	5.0	aten, pointwise, KernelGen	A variant of reflection_pad2d that assigns the output to out tensor.
reglu	NeuralNetwork	Alpha	4.2	fused, Transformer	Rectified Gated Linear Unit is a variant of GLU that uses ReLU instead of the sigmoid function for gating. This operator was introduced in v4.2 release but not exposed.
relu	NeuralNetwork	Stable	1.0	aten, Activation, pointwise, nn.functional	Apply the RELU (Rectified Linear Unit) activation function element-wise.
relu_	NeuralNetwork	Stable	2.2	aten, pointwise, Activation	This is the in-place version of relu().
relu6	NeuralNetwork	Beta	5.0	aten, pointwise, Activation, KernelGen	Applies the element-wise function f(x)=min(max(0,x),6). This is a variation of the standard ReLU activation function that “caps” its output at a maximum value of 6.
remainder	Math	Stable	2.2	aten	Computes Python’s modulus operation entrywise. The result has the same sign as the divisor other and its absolute value is less than that of other.
remainder_	Math	Stable	2.2	aten	This is the in-place version of remainder().
repeat	Tensor	Stable	2.1	aten	Repeats this tensor along the specified dimensions.
repeat_interleave_self_int	Tensor	Stable	2.2	aten, pointwise	Repeats elements of a tensor. The number of repetitions is specified as an integer repeats.
repeat_interleave_self_tensor	Tensor	Stable	2.2	aten, pointwise	Repeats elements of a tensor. The number of repetitions is specified as a tensor repeats. repeats is broadcasted to fit the shape of the given axis.
repeat_interleave_tensor	Tensor	Stable	2.2	aten, pointwise	Repeats 0 repeats[0] times, 1 repeats[1] times, 2 repeats[2] times, etc.
replication_pad1d	Tensor	Beta	5.0	aten, KernelGen	Pads the edge of a 1D input tensor by repeating the boundary values.
replication_pad1d_out	Tensor	Beta	5.0	aten, KernelGen	A variant of replication_pad1d that assigns the output to the out tensor.
replication_pad3d	NeuralNetwork	Alpha	5.0	aten	Pads the edge of a 3D input tensor by repeating the boundary values.
reshape_and_cache	Attention	Stable	3.0	fused, vLLM	Store the key/value token states into the pre-allcated kv_cache buffers of paged attention.
reshape_and_cache_flash	Attention	Stable	3.0	fused	Store the key/value token states into the pre-allcated kv_cache buffers of paged attention.
resolve_conj	Science	Stable	2.1	aten	Returns a new tensor with materialized conjugation if input’s conjugate bit is set to True, else returns input. The output tensor will always have its conjugate bit set to False.
resolve_neg	Science	Stable	2.1	aten	Returns a new tensor with materialized negation if input’s negative bit is set to True, else returns input. The output tensor will always have its negative bit set to False.
rms_norm	NeuralNetwork	Stable	2.0	aten, nn.functional, Reduction	Apply Root Mean Square Layer Normalization over a mini-batch of inputs.
rms_norm_backward	NeuralNetwork	Stable	2.0	aten, nn.functional, Reduction	This is the backward case for rms_norm.
rms_norm_forward	NeuralNetwork	Stable	2.0	aten, nn.functional, Reducito0n	This is the forward case for rms_norm.
rotary_embedding	Attention	Stable	3.0	vLLM	Apply rotary positional embeddings.
rrelu_with_noise_backward	NeuralNetwork	Beta	5.0	aten, KernelGen	Computes the gradient of the Randomized Leaky ReLU (RReLU) activation function with respect to its input during backpropagation. It uses the noise tensor generated in the forward pass to correctly apply the slope to negative input values.
rsqrt	Math	Stable	1.0	aten, pointwise	Returns a new tensor with the reciprocal of the square-root of each of the elements of input.
rsqrt_	Math	Stable	2.2	aten, pointwise	The in-place version of rsqrt().
rwkv_ka_fusion	RWKV	Stable	4.1	fused	Merges, aligns, and enhances features from different data sources or spatial directions using the efficient, linear-time RWKV framework.
rwkv_mm_sparsity	RWKV	Stable	4.1	fused	Optimized, lossless sparse matrix multiplication in RWKV-7 models.
scaled_dot_product_attention	NeuralNetwork	Stable	2.2	nn.functional, Attention	Computes scaled dot product attention on query, key and value tensors, using an optional attention mask if passed and applying dropout if a probability greater than 0.0 is specified. The optional scale argument can only be specified as a keyword argument.
scaled_dot_product_attention_backward	NeuralNetwork	Stable	2.2	nn.functional, Attention	The backward case for scaled_dot_product_attention.
scaled_dot_product_attention_forward	NeuralNetwork	Stable	2.2	nn.functional, Attention	The forward case for scaled_dot_product_attention.
scaled_softmax_backward	Reduction	Stable	4.2	aten	The backward pass for a scaled softmax function, commonly used in Scaled Dot-Product Attention (SDPA) within Transformer models, computes the gradient of the loss with respect to the input logits, incorporating a scaling factor to stabilize training.
scaled_softmax_forward	Reduction	Stable	4.2	aten	The backward pass for a scaled softmax function, commonly used in Scaled Dot-Product Attention (SDPA) within Transformer models, computes the gradient of the loss with respect to the input logits, incorporating a scaling factor to stabilize training.
scatter	Tensor	Stable	2.2	aten	Writes all values from the tensor src into provided tensor at the indices specified in the index tensor. For each value in src, its output index is specified by its index in src for dimension != dim and by the corresponding value in index for dimension = dim. The optional reduce argument allows specification of an optional reduction operation, which is applied to all values in the tensor src into the tensor at the indices specified in the index.
scatter_	Tensor	Stable	3.0	aten	This is the in-place version of scatter().
scatter_add_	Tensor	Stable	4.2	aten	Adds all values from the tensor src into self at the indices specified in the index tensor in a similar fashion as scatter_(). For each value in src, it is added to an index in self which is specified by its index in src for dimension != dim and by the corresponding value in index for dimension = dim.
select_scatter	Tensor	Stable	2.2	aten	Embeds the values of the src tensor into input at the given index. This function returns a tensor with fresh storage; it does not create a view.
selu	NeuralNetwork	Beta	5.0	aten, pointwise, nn.functional, Activation, KernelGen	Applies an element-wise activation function that induces self-normalizing properties in neural networks. It scales the Exponential Linear Unit (ELU) to ensure activations remain close to zero mean and unit variance.
selu_	NeuralNetwork	Beta	5.0	aten, pointwise, Activation, KernelGen	This is the in-place version of selu.
sgn_	Math	Beta	5.0	aten, KernelGen	Computes the sign of each element in the self tensor, element-wise. This function is an extension of sign() designed to handle complex tensors in addition to real-valued ones.
sigmoid	NeuralNetwork	Stable	2.0	aten, pointwise	Computes the expit (also known as the logistic sigmoid function) of the elements of input.
sigmoid_	NeuralNetwork	Stable	2.2	aten, pointwise	The in-place version of sigmoid().
sigmoid_backward	NeuralNetwork	Stable	3.0	aten, pointwise	The backward version of sigmoid().
silu	NeuralNetwork	Stable	1.0	aten, pointwise, nn.functional	SiLU (Sigmoid Linear Unit), a simple approximation of ReLU but without any discontinuity of the first derivative.
silu_	NeuralNetwork	Stable	2.2	aten, nn.functional, pointwise	The in-place version of silu().
silu_and_mul	Activation	Stable	2.0	fused, pointwise, vLLM	A custom operator in vLLM as activation function for SwiGLU.
silu_and_mul_out	Activation	Stable	2.0	fused, pointwise, vLLM	A variant of silu_and_mul with an extra out argument.
silu_backward	NeuralNetwork	Stable	3.0	aten, pointwise	A variant of silu() for backward case.
sin	Math	Stable	2.0	aten, pointwise	Returns a new tensor with the sine of the elements in the input tensor, where each value in this input tensor is in radians.
sin_	Math	Stable	2.2	aten, pointwise	The in-place version of sin().
sinh_	Math	Beta	5.0	aten, KernelGen	Computes the hyperbolic sine (e^x-e^{-x})/2 of each element in a tensor. This is an in-place version.
skip_layer_norm	NeuralNetwork	Stable	2.0	fused, Transformer	An optimized operation used in Transformer models to improve performance by combining residual connection (skip connection) addition and Layer Normalization (LayerNorm) into a single kernel.
slice_backward	NeuralNetwork	Stable	5.0	aten	An automatic differentiation (autograd) function that computes the gradient of a tensor slicing operation (tensor[start:end]) during backpropagation.
slice_scatter	Tensor	Stable	2.2	aten	Embeds the values of the src tensor into input at the given dimension. This function returns a tensor with fresh storage; it does not create a view.
softmax	NeuralNetwork	Stable	1.0	aten, nn.functional	Apply a softmax function.
softmax_backward	Reduction	Stable	3.0	aten, nn.functional	The in-place version of softmax().
softplus	NeuralNetwork	Stable	4.0	aten, nn.functional, pointwise	Applies element-wise, the function Softplus.
softshrink	NeuralNetwork	Beta	5.0	aten, nn.functional, Activation, KernelGen	Applies the soft shrinkage function element-wise to an input tensor. It is an activation function often used in signal processing and sparse representation, such as image denoising.
softshrink_out	NeuralNetwork	Beta	5.0	aten, nn.functional, Activation, KernelGen	This is a variant of softshrink that supports an output tensor.
sort	Data	Stable	2.2	aten	Sorts the elements of the input tensor along a given dimension in ascending order by value.
sort_stable	Data	Stable	3.0	aten	Sorts the elements of the input tensor along a given dimension in ascending order by value. This is a variant of sort() where stable is set to True to preserve the order of equivalent elements.
sparse_mla_fwd_interface	DSA	Stable	5.0	fused
special_i1	Math	Beta	5.0	aten, pointwise, KernelGen	Computes the modified Bessel function of the first kind of order 1 (I_1(x)) for each element in the input tensor, designed for special mathematical functions.
special_i1_out	Math	Beta	5.0	aten, pointwise, KernelGen	A variant of special_i1 that allows the output to be assigned to another tensor.
sqrt	Math	Stable	4.0	aten, pointwise	Returns a new tensor with the square-root of the elements of input.
sqrt_	Math	Stable	4.0	aten, pointwise	This is the in-place version of sqrt().
stack	Tensor	Stable	2.2	aten	Concatenates a sequence of tensors along a new dimension.
std	Reduction	Stable	4.0	aten	Calculates the standard deviation over the dimensions specified by dim. dim can be a single dimension, list of dimensions, or None to reduce over all dimensions.
sub	Math	Stable	1.0	aten, pointwise	Subtracts other, scaled by alpha, from the input tensor.
sub_	Math	Stable	2.2	aten, pointwise	Subtracts other, scaled by alpha, from the input tensor. This is the in-place version.
sum	LinearAlg	Stable	2.0	aten, Reduction	Returns the sum of all elements in the input tensor.
sum_dim	LinearAlg	Stable	2.0	aten, Reduction	Returns the sum of each row of the input tensor in the given dimension dim. dim is a list of dimensions, reduce over all of them.
sum_dim_out	LinearAlg	Stable	3.0	aten, Reduction	A variant of sum_dim() with the out argument.
sum_out	LinearAlg	Stable	3.0	aten, Reduction	A variant of sum() with the out argument.
swiglu	NeuralNetwork	Stable	5.0	fused, Transformer	Swish-Gated Linear Unit, a variant of GLU with the Swish activation function.
t_copy	Tensor	Beta	5.0	aten, KernelGen	Transpose a 2D tensor into a new tensor with contiguous memory layout.
t_copy_out	Tensor	Beta	5.0	aten, KernelGen	A variant of t_copy() that allows the output to be assigned to the out tensor.
tan	NeuralNetwork	Stable	4.1	aten, pointwise	Returns a new tensor with the tangent of the elements in the input tensor, where each value in this input tensor is in radians.
tan_		Stable	4.1	aten, pointwise	This is the in-place version of tan().
tanh	Math	Stable	2.0	aten, pointwise	Returns a new tensor with the hyperbolic tangent of the elements of input.
tanh_	Math	Stable	2.2	aten, pointwise	This is the in-place version of tanh().
tanh_backward	Math	Stable	3.0	aten, pointwise	This is the backward case for tanh().
threshold	NeuralNetwork	Stable	3.0	aten, nn.functional, pointwise	Apply a threshold to each element of the input Tensor.
threshold_backward	NeuralNetwork	Stable	3.0	aten, nn.functional, pointwise	This is the backward version for threshold.
tile	Tensor	Stable	2.1	aten	Constructs a tensor by repeating the elements of input. The dims argument specifies the number of repetitions in each dimension.
to_copy	Tensor	Beta	4.1	aten, pointwise
topk	Tensor	Stable	2.1	aten	Returns the k largest elements of the given input tensor along a given dimension. If dim is not given, the last dimension of the input is chosen. If largest is False then the k smallest elements are returned.
topk_softmax	MoE	Stable	4.0	fused, vLLM	Selects the k most likely next-token candicates, sets all others to zero, and renormalize the prbabilities of these top candidates.
trace	Reduction	Stable	4.0	aten	Returns the sum of the elements of the diagonal of the input 2-D matrix.
tril	BLAS	Beta	5.0	aten, KernelGen	Returns the lower triangular part of an input matrix (or a batch of matrices) and sets all other elements to zero.
triu	BLAS	Stable	1.0	aten	Returns the upper triangular part of a matrix (2-D tensor) or batch of matrices input, the other elements of the result tensor out are set to 0.
triu_	NeuralNetwork	Stable	5.0	aten	The in-place version of triu().
true_divide	Math	Stable	2.1	aten, pointwise	Divides each element of the input input by the corresponding element of other. Note that torch.divide() is an alias of torch.div() and torch.true_divide() is an alias of torch.div() with rounding_mode=None.
true_divide_	Math	Stable	2.1	aten	This is the in-place version of true_divide().
true_divide_out	Math	Stable	4.2	aten	This is an variant of true_divide() with an out argument.
trunc_divide	Math	Stable	2.1	aten	The div function with rounding_mode set to trunc.
unfold_backward	NeuralNetwork	Stable	5.0	aten, nn.functional	An operator for calculating the gradient of the unfold operation during backpropagation. It takes the gradient of the unfolded output and accumulates it back into the original input shape, reversing sliding local block extraction and resolving overlaps.
uniform_	Distribution	Stable	2.1	aten	Fills self tensor with numbers sampled from the continuous uniform distribution.
upsample_bicubic2d	NeuralNetwork	Stable	5.0	aten, Reduction	A variant of upsample() that has mode set to bicubic.
upsample_linear1d	NeuralNetwork	Stable	5.0	aten	Upsamples the input, using linear mode. The input has to be 3 dimensional, and the output_size is an optional tuple of ints.
upsample_nearest1d	NeuralNetwork	Stable	5.0	aten	Upsamples the input, using nearest neighbours’ pixel values. The input has to be 3 dimensional, and the output_size is an optional tuple of ints.
upsample_nearest2d	NeuralNetwork	Stable	2.2	aten	Upsamples the input, using nearest neighbours’ pixel values. The input has to be 4 dimensional. The scales can be provided with scales_h and scales_w.
upsample_nearest3d	NeuralNetwork	Stable	5.0	aten	Performs 3D nearest-neighbor interpolation to increase the spatial size of volumetric data, such as 5D tensors. It scales up inputs by copying values from the nearest pixel/voxel, without calculating new values through linear interpolation.
var_mean	LinearAlg	Stable	2.0	aten, Reduction	Calculates the variance and mean over the dimensions specified by dim. dim can be a single dimension, list of dimensions, or None to reduce over all dimensions.
vdot	BLAS	Stable	2.2	aten	Computes the dot product of two 1D vectors along a dimension.
vector_norm	LinearAlg NeuralNetwork	Stable	2.0	aten, Reduction	Computes a vector norm.
vstack	Tensor	Stable	2.2	aten	Stack tensors in sequence vertically (row wise).
weight_norm	NeuralNetwork	Stable	3.0	fused	Reparameterizes a module’s weight tensor by decoupling its magnitude (g) from its direction (v). It is a hook that compute the actual weight before each forward pass.
weight_norm_interface	NeuralNetwork	Stable	2.2	aten, fused	Apply weight normalization to neural network layers, decoupling the magnitued of a weight tensor from its direction. It is used to stabilize training, particularly for models with small batch sizes.
weight_norm_interface_backward	NeuralNetwork	Stable	3.0	aten, fused	Computes the gradients for weight normalization during the backward pass. It calculates the necessary derivatives for updating both the magnitude (g) and direction (v) parameters of a weight-normalized layer, based on gradients received from the previous operation.
where_scalar_other	Tensor	Stable	2.1	aten, pointwise	Return a tensor of elements selected from either input or other, depending on condition.
where_scalar_self	Tensor	Stable	2.1	aten, pointwise	Returns a tensor of elements selected from either input or other, where input is a tensor while other is a scalar.
where_self	Tensor	Stable	2.1	aten, pointwise	Returns a LongTensor. This operation is identical to torch.nonzero(condition, as_tuple=True).
where_self_out	Tensor	Stable	2.2	aten, pointwise	This is a variant of where_self() with an argument out.
zero	Tensor	Beta	5.0	aten, KernelGen	Fills tensor with zeros.
zero_	Tensor	Stable	5.0	aten	Fills self tensor with zeros.
zero_out	Tensor	Beta	5.0	aten, KernelGen	Fills tensor with zeros but assign the output to the out tensor.
zeros	Tensor	Stable	2.1	aten	Returns a tensor filled with the scalar value 0, with the shape defined by the variable argument size.
zeros_like	Tensor	Stable	2.1	aten	Returns a tensor filled with the scalar value 0, with the same size as input.