Matrix Multiplication

Matrix multiplication is interesting. The way we calculate it's different. Its use cases also differ from regular multiplication.

Matrix multiplication and Projection

Matrix multiplication is nothing but projection. When a matrix is multiplied by an input of different dimension, then the result is nothing but the input being projected into a different space dimension.

This is because matrix multiplication changes the output dimension. The output always gets the column size of the matrix.

Important use cases

It mostly transforms data from one form to another. For example, graphics use it to rotate and scale,

Rules of matrix multiplication

1. The first matrix's columns must equal the second matrix's rows.
2. The result has the first matrix's rows and the second matrix's columns.

Example - matrix A is m x n. Matrix B is n x p. The result C is m x p.

Multiply Accumulate

Multiply Accumulate (MAC)

Matrix multiplication is a series of multiply accumulate steps. We multiply a set of values. Then we add the products. This gives the final value for each element in the result.

To find one element of the result, take a row from the first matrix. Take the matching column from the second matrix. Multiply their elements pair by pair. Then add up the products.

Code Implementation

// 1. Loop through rows of A
for (int i = 0; i < rowsA; i++) {
    // 2. Loop through columns of B
    for (int j = 0; j < colsB; j++) {
    // 3. Loop to calculate the dot product
        for (int k = 0; k < colsA; k++) {
            result[i][j] += matrixA[i][k] * matrixB[k][j];
        }
    }
}

Rank Concept

Rank defines how the matrix's multiplication's output direction changes for all its inputs.

Maximum possible value of a matrix's rank

Maximum rank is always the minimum of the row and column length.

$\text{Rank} \le \text{minimum of (Rows, Columns)}$