Linear algebra for machine learning


  1. Distance between two points
  2. Dot product and angle between 2 vectors
  3. Equation of a line, plane
  4. Projection of a vector
  5. Distance of a point from a plane
  6. Equation of a circle

1. Distance between two points

  • Let A & B be two points in 2 dimensional geometry.

    • Euclidean distance

      Example:

    • Manhatten distance

      In case the movement on diagonal side is allowed,

    • Minkowski distance (Generalisation)

      where p = 1, 2, 3 ...
    
      if p == 1, minkowski distance will converge into Manhatten distance
    
      if p == 2, minkowski distance will converge into Euclidean distance
    
  • If A & B points are in n dimensional geometry, then

    Then,

    1. Euclidean distance(A,B) =

    2. Manhatten distance(A,B) =

    3. Minkowski distance(A,B) =

2. Dot product and angle between 2 vectors

a & b points are in n dimensional vectors, i.e.,

Dot product of a & b = a.b

It is same as matrix multiplication of a & b vectors, i.e.,

Note: By default, a vector is column vector if not mentioned explicitly. So here a & b are column vectors

Also we know that : (proof)

where a = distance of a from origin & b = distance of b from origin

3. Equation of a line, plane

Line in 2D = Plane in 3D = hyper plane in nD

  • 2 dimensional geometry

    The equation of a line in 2D is

    which can also be written as

    If line passes through origin (0, 0) then y-intercept becomes 0. So equation becomes

  • 3 dimensional geometry

    Equation of a plane in 3D passing through the origin (0,0)

  • n dimensional geometry

    Equation of a plane in nD passing through the origin (0,0)

    i.e.,

    We know that if a.b = 0, then a is perpendicular to b ( if = 0, then )

    is perpendicular to any point on plane , provided plane passes through origin

4. Projection of a vector

Let & be two vectors and be the angle between them.

is the projection of on

Then

Multiply both sides by

We know that

If is a unit vector, then

5. Distance of a point from a plane

Let be any point in dimensional geometry, which is at a distance from the hyper plane .

Let be a vector passing through origin (0,0)

Distance of point from the plane is the projection of on

projection of on =

If is a unit vector, then

is positive since is less than 90

Similary we can calculate the distance of the point from the plane . Since is greater than , the distance will be negative.

In this way, we can decide in which side of the plane a point lies.

6. Equation of a circle

Let be the center of the circle & be the locus of the center of the circle.

Let be the radius of the circle

If the center of the cirlce is origin i.e., , then the equation of the circle is given as


I will update this post as & when I get time. Please contact me regarding any queries or suggestions.