# Getting a Handle on Vector for Machine Learning

The word **vector** comes from the Latin for “carrier”, reason might be the nature of vector to move from one location to another. In physics, a Vector is an object that moves around a physical space. In Data Science and Machine Learning we generalize this idea to think vector as a list of attributes of an object. So Vectors are used to represent objects, observations and predictions. Brownlee (2018)

Say we want to represent a house having 2 bedrooms, 1 bathroom, 150-meter square and cost around $ 20,000. Based on the information we know about this house, we can represent the house as a vector:

$$house = \begin{pmatrix} 2 \\ 1 \\ 150 \\ 20000 \end{pmatrix}$$

Based on this vector, our machine learning predicts that there is 45% chance that this house will give me profit if I buy, 5% chance that it will be a bad deal(loss) and 50% chance that the price of house will not change at all. We can represent this prediction in a vector as:

$$class\_probabilities = \begin{pmatrix} 0.45 \\ 0.05 \\ 0.5 \end{pmatrix}$$

## Vectors in Python

There are multiple ways we can define a vector in Python, the simplest one being a list:

`1` | ```
house = [2, 1, 150, 20000]
``` |

Python Lists are very good, but in Machine Learning we have deal with lot of scientific calculations,so it is much convenient to represent a vector in Python as a NumPy array as Numpy array does the thing what we expect when we add, multiply vectors.

1 2 | ```
import numpy as np
house = np.array([2, 1, 150, 20000])
``` |

## Vector Arithmetic

In this section, we will illustrate simple vector-vector arithmetic, where all operations are performed element-wise.

### Vector Addition

Two vector of equal length when added produce a new vector of the same length.
Each element of the new vector at index **i** is calculated as the sum of the other element at that particular index **i**.

1 2 3 4 5 6 7 8 9 | ```
import numpy as np
house1 = np.array([2,1,150, 20000])
house2 = np.array([2, 1, 200, 29000])
house3 = house1 + house2
print(house3)
[ 4, 2, 350, 49000]
``` |

### Vector Subtraction

Two vectors of equal length can be subtracted to produce a new vector of the same length. Each element of the new vector at index **i** is calculated as the difference of the other elements at that particular index **i**.

### Vector Multiplication

Two vector of equal length can be multiplied together to create a new vector of same length. Like with addition and subtraction, vector multiplication is done element wise.

$$a=b * c$$1 2 3 4 5 6 7 8 9 10 11 | ```
import numpy as np
a = np.array([1, 2, 3])
b = np.array([2, 3, 4])
c = a * b
print(c)
[ 2 6 12]
``` |

### Vector Division

Two vector of equal length can be divided element wise to calculate a new vector of same length.

## Vector Norms

The term **Norm** is typically used to represent the length or magnitude of a vector. The norm of vector **v** is denoted by **||v||**.

Norm Properties

1. For any vector \(\mathbf{v}\), \(\mathbf{||v||}\) is a non-negative real number.$$\\$$ 2. For any vector \(\mathbf{v}\), \(\mathbf{||v||}\) is zero if and only if \(\mathbf{v}\) is a zero vector.$$\\$$ 3. Triangle inequality, For any vectors \(\mathbf{x}\) and \(\mathbf{y}\), \[||x + y|| \le ||x|| + ||y||\]$$\\$$ 4. For any vector \(\mathbf{v}\) and any scalar \(\alpha \), \[||\alpha v|| = |\alpha |||v||\]$$\\$$### Different Vector Norms

#### 1. Manhattan Norm or Taxicab Norm

The \(L^1\) Norm or Manhattan Norm or Taxicab Norm of vector \(\mathbf{v}\) \( \in \) \(R^n\) is defined as \[||v||_1 = \sum\limits_{i = 1}^n {{|v_i|}} \]. $$\\$$ The \(L^1\) norm is calculated as the sum of the absolute vector values, where the absolute value of a scalar uses the notation \(\mathbf{|v|}\). \[||v||_1 = |v_1| + |v_2| + |v_3| + .. + |v_n|\]#### 2. Euclidean Norm

The \(L^2\) Norm or Euclidean of vector \(\mathbf{v}\) \( \in \) \(R^n\) is defined as \[||v||_2 = \sqrt{\sum\limits_{i = 1}^n {v_i^2}} \]. $$\\$$ The \(L^2\) norm simply calculates the Euclidean distance from the origin to the point identified by \(\mathbf{v}\). $$\\$$ The \(L^2\) norm is calculated as the square root of the sum of the squared vector values \[||v||_2 = \sqrt {v_1^2 + v_2^2 + v_3^2 + .. + v_n^2} \]#### 3. Vector Max Norm

The \(L^\infty\) norm or vector max norm of vector \(\mathbf{v}\) \( \in \) \(R^n\) is defined as \[||v||_\infty = \mathop {\max |v_i|}\limits_i \] The \(L^\infty\) norm is calculated by returning the the maximum value of vector as \[||v||_\infty = \max (v_1,v_2,..,v_n)\]1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | ```
# Import numpy
import numpy as np
# Import Linear Algebra package from NumPy
import numpy.linalg as LA
# Define a simple vector
v = np.array([1,2,3])
# Calculate L1 norm
L1_norm = LA.norm(v,1)
print(L1_norm) # 6.0
# Calculate L2 norm
L2_norm = LA.norm(v, 2)
print(L2_norm) # 3.7416573867739413
# Calculate max norm
# For max norm import infinity
from math import inf
max_norm = LA.norm(v, inf)
print(max_norm) # 3.0
``` |

MarcPeterDeisenroth (2019) IanGoodfellow (2019) Ageron (2019)

## Bibliography

Ageron. Math - Linear Algebra. url: https://github.com/ageron/handson-ml/blob/master/math_linear_algebra.ipynb, 2019. [Online; accessed 19-April-2019]. ↩

Jason Brownlee.
*Basics of Linear Algebra for Machine Learning*.
Machine Learning Mastery, 2018. ↩

Aaron Courville Ian Goodfellow, Yoshua Bengio.
*Deep Learning*.
Www.deeplearningbook.org.
www.deeplearningbook.org, 2019. ↩

Cheng Soon Ong Marc Peter Deisenroth, A. Aldo Faisal.
*Mathematics for Machine Learning*.
Cambridge University Press.
Cambridge University Press, 2019. ↩