(directly go to documentation on : Dot, ., InProduct, CrossProduct, Outer, o, ZeroVector, BaseVector, Identity, ZeroMatrix, Diagonal, DiagonalMatrix, OrthogonalBasis, OrthonormalBasis, Normalize, Transpose, Determinant, Trace, Inverse, Minor, CoFactor, MatrixPower, SolveMatrix, CharacteristicEquation, EigenValues, EigenVectors, Sparsity, Cholesky. )

16. Linear Algebra

This chapter describes the commands for doing linear algebra. They can be used to manipulate vectors, represented as lists, and matrices, represented as lists of lists.

Dot, . get dot product of tensors
InProduct inner product of vectors (deprecated)
CrossProduct outer product of vectors
Outer, o get outer tensor product
ZeroVector create a vector with all zeroes
BaseVector base vector
Identity make identity matrix
ZeroMatrix make a zero matrix
Diagonal extract the diagonal from a matrix
DiagonalMatrix construct a diagonal matrix
OrthogonalBasis create an orthogonal basis
OrthonormalBasis create an orthonormal basis
Normalize normalize a vector
Transpose get transpose of a matrix
Determinant determinant of a matrix
Trace trace of a matrix
Inverse get inverse of a matrix
Minor get principal minor of a matrix
CoFactor cofactor of a matrix
MatrixPower get nth power of a square matrix
SolveMatrix solve a linear system
CharacteristicEquation get characteristic polynomial of a matrix
EigenValues get eigenvalues of a matrix
EigenVectors get eigenvectors of a matrix
Sparsity get the sparsity of a matrix
Cholesky find the Cholesky Decomposition


Dot, . -- get dot product of tensors

Standard library
Calling format:
Dot(t1,t2)
t1 . t2
Precedence: 30

Parameters:
t1,t2 -- tensor lists (currently only vectors and matrices are supported)

Description:
Dot returns the dot (aka inner) product of two tensors t1 and t2. The last index of t1 and the first index of t2 are contracted. Currently Dot works only for vectors and matrices. Dot-multiplication of two vectors, a matrix with a vector (and vice versa) or two matrices yields either a scalar, a vector or a matrix.

Examples:
In> Dot({1,2},{3,4})
Out> 11;
In> Dot({{1,2},{3,4}},{5,6})
Out> {17,39};
In> Dot({5,6},{{1,2},{3,4}})
Out> {23,34};
In> Dot({{1,2},{3,4}},{{5,6},{7,8}})
Out> {{19,22},{43,50}};

Or, using the "."-Operator:

In> {1,2} . {3,4}
Out> 11;
In> {{1,2},{3,4}} . {5,6}
Out> {17,39};
In> {5,6} . {{1,2},{3,4}}
Out> {23,34};
In> {{1,2},{3,4}} . {{5,6},{7,8}}
Out> {{19,22},{43,50}};

See also:
Outer , Cross , IsScalar , IsVector , IsMatrix .


InProduct -- inner product of vectors (deprecated)

Standard library
Calling format:
InProduct(a,b)

Parameters:
a, b -- vectors of equal length

Description:
The inner product of the two vectors "a" and "b" is returned. The vectors need to have the same size.

This function is superceded by the . operator.

Examples:
In> {a,b,c} . {d,e,f};
Out> a*d+b*e+c*f;

See also:
Dot , CrossProduct .


CrossProduct -- outer product of vectors

Standard library
Calling format:
CrossProduct(a,b)
a X b
Precedence: 30

Parameters:
a, b -- three-dimensional vectors

Description:
The cross product of the vectors "a" and "b" is returned. The result is perpendicular to both "a" and "b" and its length is the product of the lengths of the vectors. Both "a" and "b" have to be three-dimensional.

Examples:
In> {a,b,c} X {d,e,f};
Out> {b*f-c*e,c*d-a*f,a*e-b*d};

See also:
InProduct .


Outer, o -- get outer tensor product

Standard library
Calling format:
Outer(t1,t2)
t1 o t2
Precedence: 30

Parameters:
t1,t2 -- tensor lists (currently only vectors are supported)

Description:
Outer returns the outer product of two tensors t1 and t2. Currently Outer work works only for vectors, i.e. tensors of rank 1. The outer product of two vectors yields a matrix.

Examples:
In> Outer({1,2},{3,4,5})
Out> {{3,4,5},{6,8,10}};
In> Outer({a,b},{c,d})
Out> {{a*c,a*d},{b*c,b*d}};

Or, using the "o"-Operator:

In> {1,2} o {3,4,5}
Out> {{3,4,5},{6,8,10}};
In> {a,b} o {c,d}
Out> {{a*c,a*d},{b*c,b*d}};

See also:
Dot , Cross .


ZeroVector -- create a vector with all zeroes

Standard library
Calling format:
ZeroVector(n)

Parameters:
n -- length of the vector to return

Description:
This command returns a vector of length "n", filled with zeroes.

Examples:
In> ZeroVector(4)
Out> {0,0,0,0};

See also:
BaseVector , ZeroMatrix , IsZeroVector .


BaseVector -- base vector

Standard library
Calling format:
BaseVector(k, n)

Parameters:
k -- index of the base vector to construct

n -- dimension of the vector

Description:
This command returns the "k"-th base vector of dimension "n". This is a vector of length "n" with all zeroes except for the "k"-th entry, which contains a 1.

Examples:
In> BaseVector(2,4)
Out> {0,1,0,0};

See also:
ZeroVector , Identity .


Identity -- make identity matrix

Standard library
Calling format:
Identity(n)

Parameters:
n -- size of the matrix

Description:
This commands returns the identity matrix of size "n" by "n". This matrix has ones on the diagonal while the other entries are zero.

Examples:
In> Identity(3)
Out> {{1,0,0},{0,1,0},{0,0,1}};

See also:
BaseVector , ZeroMatrix , DiagonalMatrix .


ZeroMatrix -- make a zero matrix

Standard library
Calling format:
ZeroMatrix(n)
ZeroMatrix(n, m)

Parameters:
n -- number of rows

m -- number of columns

Description:
This command returns a matrix with n rows and m columns, completely filled with zeroes. If only given one parameter, it returns the square n by n zero matrix.

Examples:
In> ZeroMatrix(3,4)
Out> {{0,0,0,0},{0,0,0,0},{0,0,0,0}};
In> ZeroMatrix(3)
Out> {{0,0,0},{0,0,0},{0,0,0}};

See also:
ZeroVector , Identity .


Diagonal -- extract the diagonal from a matrix

Standard library
Calling format:
Diagonal(A)

Parameters:
A -- matrix

Description:
This command returns a vector of the diagonal components of the matrix A.

Examples:
In> Diagonal(5*Identity(4))
Out> {5,5,5,5};
In> Diagonal(HilbertMatrix(3))
Out> {1,1/3,1/5};

See also:
DiagonalMatrix , IsDiagonal .


DiagonalMatrix -- construct a diagonal matrix

Standard library
Calling format:
DiagonalMatrix(d)

Parameters:
d -- list of values to put on the diagonal

Description:
This command constructs a diagonal matrix, that is a square matrix whose off-diagonal entries are all zero. The elements of the vector "d" are put on the diagonal.

Examples:
In> DiagonalMatrix(1 .. 4)
Out> {{1,0,0,0},{0,2,0,0},{0,0,3,0},{0,0,0,4}};

See also:
Identity , ZeroMatrix .


OrthogonalBasis -- create an orthogonal basis

Standard library
Calling format:
OrthogonalBasis(W)

Parameters:
W - A linearly independent set of row vectors (aka a matrix)

Description:
Given a linearly independent set W (constructed of rows vectors), this command returns an orthogonal basis V for W, which means that span(V) = span(W) and InProduct(V[i],V[j]) = 0 when i != j. This function uses the Gram-Schmidt orthogonalization process.

Examples:
In> OrthogonalBasis({{1,1,0},{2,0,1},{2,2,1}}) 
Out> {{1,1,0},{1,-1,1},{-1/3,1/3,2/3}};

See also:
OrthonormalBasis , InProduct .


OrthonormalBasis -- create an orthonormal basis

Standard library
Calling format:
OrthonormalBasis(W)

Parameters:
W - A linearly independent set of row vectors (aka a matrix)

Description:
Given a linearly independent set W (constructed of rows vectors), this command returns an orthonormal basis V for W. This is done by first using OrthogonalBasis(W), then dividing each vector by its magnitude, so as the give them unit length.

Examples:
In> OrthonormalBasis({{1,1,0},{2,0,1},{2,2,1}})
Out> {{Sqrt(1/2),Sqrt(1/2),0},{Sqrt(1/3),-Sqrt(1/3),Sqrt(1/3)},
	{-Sqrt(1/6),Sqrt(1/6),Sqrt(2/3)}};

See also:
OrthogonalBasis , InProduct , Normalize .


Normalize -- normalize a vector

Standard library
Calling format:
Normalize(v)

Parameters:
v -- a vector

Description:
Return the normalized (unit) vector parallel to v: a vector having the same direction but with length 1.

Examples:
In> v:=Normalize({3,4})
Out> {3/5,4/5};
In> v . v
Out> 1;

See also:
InProduct , CrossProduct .


Transpose -- get transpose of a matrix

Standard library
Calling format:
Transpose(M)

Parameters:
M -- a matrix

Description:
Transpose returns the transpose of a matrix M. Because matrices are just lists of lists, this is a useful operation too for lists.

Examples:
In> Transpose({{a,b}})
Out> {{a},{b}};


Determinant -- determinant of a matrix

Standard library
Calling format:
Determinant(M)

Parameters:
M -- a matrix

Description:
Returns the determinant of a matrix M.

Examples:
In> A:=DiagonalMatrix(1 .. 4)
Out> {{1,0,0,0},{0,2,0,0},{0,0,3,0},{0,0,0,4}};
In> Determinant(A)
Out> 24;


Trace -- trace of a matrix

Standard library
Calling format:
Trace(M)

Parameters:
M -- a matrix

Description:
Trace returns the trace of a matrix M (defined as the sum of the elements on the diagonal of the matrix).

Examples:
In> A:=DiagonalMatrix(1 .. 4)
Out> {{1,0,0,0},{0,2,0,0},{0,0,3,0},{0,0,0,4}};
In> Trace(A)
Out> 10;


Inverse -- get inverse of a matrix

Standard library
Calling format:
Inverse(M)

Parameters:
M -- a matrix

Description:
Inverse returns the inverse of matrix M. The determinant of M should be non-zero. Because this function uses Determinant for calculating the inverse of a matrix, you can supply matrices with non-numeric (symbolic) matrix elements.

Examples:
In> A:=DiagonalMatrix({a,b,c})
Out> {{a,0,0},{0,b,0},{0,0,c}};
In> B:=Inverse(A)
Out> {{(b*c)/(a*b*c),0,0},{0,(a*c)/(a*b*c),0},
{0,0,(a*b)/(a*b*c)}};
In> Simplify(B)
Out> {{1/a,0,0},{0,1/b,0},{0,0,1/c}};

See also:
Determinant .


Minor -- get principal minor of a matrix

Standard library
Calling format:
Minor(M,i,j)

Parameters:
M -- a matrix

i, j - positive integers

Description:
Minor returns the minor of a matrix around the element ( i, j). The minor is the determinant of the matrix obtained from M by deleting the i-th row and the j-th column.

Examples:
In> A := {{1,2,3}, {4,5,6}, {7,8,9}};
Out> {{1,2,3},{4,5,6},{7,8,9}};
In> PrettyForm(A);

/                    \
| ( 1 ) ( 2 ) ( 3 )  |
|                    |
| ( 4 ) ( 5 ) ( 6 )  |
|                    |
| ( 7 ) ( 8 ) ( 9 )  |
\                    /
Out> True;
In> Minor(A,1,2);
Out> -6;
In> Determinant({{2,3}, {8,9}});
Out> -6;

See also:
CoFactor , Determinant , Inverse .


CoFactor -- cofactor of a matrix

Standard library
Calling format:
CoFactor(M,i,j)

Parameters:
M -- a matrix

i, j - positive integers

Description:
CoFactor returns the cofactor of a matrix around the element ( i, j). The cofactor is the minor times (-1)^(i+j).

Examples:
In> A := {{1,2,3}, {4,5,6}, {7,8,9}};
Out> {{1,2,3},{4,5,6},{7,8,9}};
In> PrettyForm(A);

/                    \
| ( 1 ) ( 2 ) ( 3 )  |
|                    |
| ( 4 ) ( 5 ) ( 6 )  |
|                    |
| ( 7 ) ( 8 ) ( 9 )  |
\                    /
Out> True;
In> CoFactor(A,1,2);
Out> 6;
In> Minor(A,1,2);
Out> -6;
In> Minor(A,1,2) * (-1)^(1+2);
Out> 6;

See also:
Minor , Determinant , Inverse .


MatrixPower -- get nth power of a square matrix

Standard library
Calling format:
MatrixPower(mat,n)

Parameters:
mat -- a square matrix

n -- an integer

Description:
MatrixPower(mat,n) returns the nth power of a square matrix mat. For positive n it evaluates dot products of mat with itself. For negative n the nth power of the inverse of mat is returned. For n=0 the identity matrix is returned.

Example:
In> A:={{1,2},{3,4}}
Out> {{1,2},{3,4}};
In> MatrixPower(A,0)
Out> {{1,0},{0,1}};
In> MatrixPower(A,1)
Out> {{1,2},{3,4}};
In> MatrixPower(A,3)
Out> {{37,54},{81,118}};
In> MatrixPower(A,-3)
Out> {{-59/4,27/4},{81/8,-37/8}};

See also:
IsSquareMatrix , Inverse , Dot .


SolveMatrix -- solve a linear system

Standard library
Calling format:
SolveMatrix(M,v)

Parameters:
M -- a matrix

v -- a vector

Description:
SolveMatrix returns the vector x that satisfies the equation M*x=v. The determinant of M should be non-zero.

Examples:
In> A := {{1,2}, {3,4}};
Out> {{1,2},{3,4}};
In> v := {5,6};
Out> {5,6};
In> x := SolveMatrix(A, v);
Out> {-4,9/2};
In> A * x;
Out> {5,6};

See also:
Inverse , Solve , PSolve , Determinant .


CharacteristicEquation -- get characteristic polynomial of a matrix

Standard library
Calling format:
CharacteristicEquation(matrix,var)

Parameters:
matrix -- a matrix

var -- a free variable

Description:
CharacteristicEquation returns the characteristic equation of "matrix", using "var". The zeros of this equation are the eigenvalues of the matrix, Det(matrix-I*var);

Examples:
In> A:=DiagonalMatrix({a,b,c})
Out> {{a,0,0},{0,b,0},{0,0,c}};
In> B:=CharacteristicEquation(A,x)
Out> (a-x)*(b-x)*(c-x);
In> Expand(B,x)
Out> (b+a+c)*x^2-x^3-((b+a)*c+a*b)*x+a*b*c;

See also:
EigenValues , EigenVectors .


EigenValues -- get eigenvalues of a matrix

Standard library
Calling format:
EigenValues(matrix)

Parameters:
matrix -- a square matrix

Description:
EigenValues returns the eigenvalues of a matrix. The eigenvalues x of a matrix M are the numbers such that M*v=x*v for some vector.

It first determines the characteristic equation, and then factorizes this equation, returning the roots of the characteristic equation Det(matrix-x*identity).

Examples:
In> M:={{1,2},{2,1}}
Out> {{1,2},{2,1}};
In> EigenValues(M)
Out> {3,-1};

See also:
EigenVectors , CharacteristicEquation .


EigenVectors -- get eigenvectors of a matrix

Standard library
Calling format:
EigenVectors(A,eigenvalues)

Parameters:
matrix -- a square matrix

eigenvalues -- list of eigenvalues as returned by EigenValues

Description:
EigenVectors returns a list of the eigenvectors of a matrix. It uses the eigenvalues and the matrix to set up n equations with n unknowns for each eigenvalue, and then calls Solve to determine the values of each vector.

Examples:
In> M:={{1,2},{2,1}}
Out> {{1,2},{2,1}};
In> e:=EigenValues(M)
Out> {3,-1};
In> EigenVectors(M,e)
Out> {{-ki2/ -1,ki2},{-ki2,ki2}};

See also:
EigenValues , CharacteristicEquation .


Sparsity -- get the sparsity of a matrix

Standard library
Calling format:
Sparsity(matrix)
Parameters:
matrix -- a matrix

Description:
The function Sparsity returns a number between 0 and 1 which represents the percentage of zero entries in the matrix. Although there is no definite critical value, a sparsity of 0.75 or more is almost universally considered a "sparse" matrix. These type of matrices can be handled in a different manner than "full" matrices which speedup many calculations by orders of magnitude.

Examples:
In> Sparsity(Identity(2))
Out> 0.5;
In> Sparsity(Identity(10))
Out> 0.9;
In> Sparsity(HankelMatrix(10))
Out> 0.45;
In> Sparsity(HankelMatrix(100))
Out> 0.495;
In> Sparsity(HilbertMatrix(10))
Out> 0;
In> Sparsity(ZeroMatrix(10,10))
Out> 1;


Cholesky -- find the Cholesky Decomposition

Standard library
Calling format:
Cholesky(A)

Parameters:
A -- a square positive definite matrix

Description:
Cholesky returns a upper triangular matrix R such that Transpose(R)*R = A. The matrix A must be positive definite, Cholesky will notify the user if the matrix is not. Some families of positive definite matrices are all symmetric matrices, diagonal matrices with positive elements and Hilbert matrices.

Examples:
In> A:={{4,-2,4,2},{-2,10,-2,-7},{4,-2,8,4},{2,-7,4,7}}
Out> {{4,-2,4,2},{-2,10,-2,-7},{4,-2,8,4},{2,-7,4,7}};
In> R:=Cholesky(A);
Out> {{2,-1,2,1},{0,3,0,-2},{0,0,2,1},{0,0,0,1}};
In> Transpose(R)*R = A
Out> True;
In> Cholesky(4*Identity(5))
Out> {{2,0,0,0,0},{0,2,0,0,0},{0,0,2,0,0},{0,0,0,2,0},{0,0,0,0,2}};
In> Cholesky(HilbertMatrix(3))
Out> {{1,1/2,1/3},{0,Sqrt(1/12),Sqrt(1/12)},{0,0,Sqrt(1/180)}};
In> Cholesky(ToeplitzMatrix({1,2,3}))
In function "Check" :
CommandLine(1) : "Cholesky: Matrix is not positive definite"

See also:
IsSymmetric , IsDiagonal , Diagonal .