Prepared by Sanasam Ranbir Singh
Arithmetic Operation on Tensors
---
Tensor can store data with wide ranges of dimensions, starting from scalar (zero dimensional data) to any dimension. Arithmetic operations like plus, minus, multiplication, division can be applied between two tensors. Element-wise operations between tensors will be performed.
For example
Given two tensors $a$ and $b$, for performing an arithmetic operation, the following compatibilities are checked.
> - If the shape of the respective component dimensions of the tensors are same. Then, they are compatible.
> - For the mismatched component dimensions, one of them has shape 1, Then, they are compatible.
> - Otherwise, they are not compatible.
If the dimension and/or shapes of $a$ and $b$ are not same, but compatible. Then apply Broadcast over the lower dimensional tensor.
Broadcast (Stretch): Replicate the values
The following example shows multiplication of a vector $a = [1,2,3]$ and scalar $b=2$.
The shape of the tensor $a$ = (3), i.e., dimension=1
The shape of the tensor $b$ = (1), i.e., dimension=1. A scalar can also be consider as a vector of single element.
The Shapes are different i.e., $(3)!= (1)$, but one of the component dimension is 1. So, apply stretching the tensor with dimension 1, i.e., replicate the values of the tensor till the matching shape of the particular component dimension.
The value of the scalar is replicated till the matching shape. A scalar can always perform arthmetic operations (+,-,*,/) with a vector.
The following example shows the case of higher dimension.
Another Example:
The following example are not compatitble.
---
import tensorflow as tfBetween Two Scalar Tensors
The scalar tensors are always compatible.
x = tf.constant(5)
y = tf.constant(7)
z = x + y
print(z)tf.Tensor(12, shape=(), dtype=int32)
Between Two Vector Tensors
Given two tensors $x = [1,2,3]$ and $y = [4,5,6]$, $Shape(x)=Shape(y)=> (3)=(3)$. They are compatible.
x = tf.constant([1,2,3])
y = tf.constant([4,5,6])
z = x + y
print(z)tf.Tensor([5 7 9], shape=(3,), dtype=int32)
Between Two Matrix Tensors
Given the following two tensors $x$ and $y$,
\begin{equation}
x=
\begin{pmatrix}
1 & 2 & 3\\
4 & 5 & 6\\
\end{pmatrix}
\end{equation}
and
\begin{equation}
y=
\begin{pmatrix}
7 & 8 & 9\\
10 & 11 & 12\\
\end{pmatrix}
\end{equation}
$Shape(x)=Shape(y)=> (2,3)=(2,3)$. They are compatible.
x = tf.constant([[1,2,3],[4,5,6]])
print("x: ")
tf.print(x)
y = tf.constant([[4,5,6],[7,8,9]])
print("y: ")
tf.print(y)
y.numpy()
z = x + y
print("z: ")
tf.print(z)x: [[1 2 3] [4 5 6]] y: [[4 5 6] [7 8 9]] z: [[5 7 9] [11 13 15]]
Between Sclar and Vector Tensors
Given a scalar tensor $x = 2 $ and vector tensor $y = [4,5,6]$, $Shape(x)\neq Shape(y)=> ( )\neq(3)$.
Since scalar can also be defined as vector of one element, broadcast can be appled. Hence, they are compatible.
y = tf.constant([1,2,3])
x = tf.constant(2)
z = y * x
tf.print(z)[2 4 6]
Between Vector and Matrix Tensors
Given a tensor $x = [1,2,3]$ and a matrix tensor
\begin{equation}
y=
\begin{pmatrix}
4 \\
5 \\
6\\
\end{pmatrix}
\end{equation}
$Shape(x)\neq Shape(y)=> (3)\neq(3\times 1)$.
Since one of all the mismatched component dimensions is 1, broadcast can be appled. Hence, they are compatible.
x = tf.constant([1,2,3])
y = tf.constant([[4],[5],[6]])
z = x + y
tf.print(z)[[5 6 7] [6 7 8] [7 8 9]]
Another Example
Given a tensor $x = [1,2,3,4]$ and a matrix tensor
\begin{equation}
y=
\begin{pmatrix}
4 &6&7&8 \\
9 &10&11&12\\
\end{pmatrix}
\end{equation}
$Shape(x)\neq Shape(y)=> (3)\neq(2\times 3)$.
That means,
x = tf.constant([1,2,3,4])
y = tf.constant([[4,6,7,8],[9,10,11,12]])
z = y + x
tf.print(z)[[5 8 10 12] [10 12 14 16]]
Incompatible Example
Given $x = [1,2,3]$ and $y = [1,2,3,4]$,
$Shape(x)\neq Shape(y)=> (3)\neq(4)$.Not Compatible
x = tf.constant([1,2,3])
y = tf.constant([4,5,6,8])
z = x + y
print(z)---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
Input In [32], in <cell line: 3>()
1 x = tf.constant([1,2,3])
2 y = tf.constant([4,5,6,8])
----> 3 z = x + y
4 print(z)
File F:\Anaconda\lib\site-packages\tensorflow\python\util\traceback_utils.py:153, in filter_traceback.<locals>.error_handler(*args, **kwargs)
151 except Exception as e:
152 filtered_tb = _process_traceback_frames(e.__traceback__)
--> 153 raise e.with_traceback(filtered_tb) from None
154 finally:
155 del filtered_tb
File F:\Anaconda\lib\site-packages\tensorflow\python\framework\ops.py:7164, in raise_from_not_ok_status(e, name)
7162 def raise_from_not_ok_status(e, name):
7163 e.message += (" name: " + name if name is not None else "")
-> 7164 raise core._status_to_exception(e) from None
InvalidArgumentError: Incompatible shapes: [3] vs. [4] [Op:AddV2] Another example
Given a tensor $x = [1,2,3]$ and a matrix tensor
\begin{equation}
y=
\begin{pmatrix}
4 &6&7&8 \\
9 &10&11&12\\
\end{pmatrix}
\end{equation}
$Shape(x)\neq Shape(y)=> (3)\neq(2\times 4)$. That means,
x = tf.constant([1,2,3])
y = tf.constant([[4,5,6,8],[9,10,11,12]])
z = x + y
print(z)---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
Input In [33], in <cell line: 3>()
1 x = tf.constant([1,2,3])
2 y = tf.constant([[4,5,6,8],[9,10,11,12]])
----> 3 z = x + y
4 print(z)
File F:\Anaconda\lib\site-packages\tensorflow\python\util\traceback_utils.py:153, in filter_traceback.<locals>.error_handler(*args, **kwargs)
151 except Exception as e:
152 filtered_tb = _process_traceback_frames(e.__traceback__)
--> 153 raise e.with_traceback(filtered_tb) from None
154 finally:
155 del filtered_tb
File F:\Anaconda\lib\site-packages\tensorflow\python\framework\ops.py:7164, in raise_from_not_ok_status(e, name)
7162 def raise_from_not_ok_status(e, name):
7163 e.message += (" name: " + name if name is not None else "")
-> 7164 raise core._status_to_exception(e) from None
InvalidArgumentError: Incompatible shapes: [3] vs. [2,4] [Op:AddV2] Summary
In this lesson, we have learnt
> - Different arthmetic operations on tensors.
> - Compatibility of two tensors for performing arithmetic operations.
> - Broadcasting values of a tensor.