1. Symbolic algebra algorithms


1.1 Sparse representations


The sparse tree data structure

Yacas has a sparse tree object for use as a storage for storing (key,value) pairs for which the following properties hold:

In addition, for multiplication the following rule is obeyed:

The last is optional. For multivariate polynomials (described elsewhere) both hold, but for matrices, only the addition property holds. The function MultiplyAddSparseTrees (described below) should not be used in these cases.


Internal structure

A key is defined to be a list of integer numbers (n[1], ..., n[m]). Thus for a two-dimensional key, one item in the sparse tree database could be reflected as the (key,value) pair {{1,2},3} , which states that element (1,2) has value 3. (Note: this is not the way it is stored in the database!).

The storage is recursive. The sparse tree begins with a list of objects {n1,tree1} for values of n1 for the first item in the key. The tree1 part then contains a sub-tree for all the items in the database for which the value of the first item in the key is n1.

The above single element could be created with

In> r:=CreateSparseTree({1,2},3)
Out> {{1,{{2,3}}}};
CreateSparseTree makes a database with exactly one item. Items can now be obtained from the sparse tree with SparseTreeGet.

In> SparseTreeGet({1,2},r)
Out> 3;
In> SparseTreeGet({1,3},r)
Out> 0;
And values can also be set or changed:

In> SparseTreeSet({1,2},r,Current+5)
Out> 8;
In> r
Out> {{1,{{2,8}}}};
In> SparseTreeSet({1,3},r,Current+5)
Out> 5;
In> r
Out> {{1,{{3,5},{2,8}}}};
The variable Current represents the current value, and can be used to determine the new value. SparseTreeSet destructively modifies the original, and returns the new value. If the key pair was not found, it is added to the tree.

The sparse tree can be traversed, one element at a time, with SparseTreeScan:

In> SparseTreeScan(Hold({{k,v},Echo({k,v})}),2,r)
{1,3} 5 
{1,2} 8 

An example of the use of this function could be multiplying a sparse matrix with a sparse vector, where the entire matrix can be scanned with SparseTreeScan, and each non-zero matrix element A[i][j] can then be multiplied with a vector element v[j], and the result added to a sparse vector w[i], using the SparseTreeGet and SparseTreeSet functions. Multiplying two sparse matrices would require two nested calls to SparseTreeScan to multiply every item from one matrix with an element from the other, and add it to the appropriate element in the resulting sparse matrix.

When the matrix elements A[i][j] are defined by a function f(i,j) (which can be considered a dense representation), and it needs to be multiplied with a sparse vector v[j], it is better to iterate over the sparse vector v[j]. Representation defines the most efficient algorithm to use in this case.

The API to sparse trees is:


1.2 Implementation of multivariate polynomials

This section describes the implementation of multivariate polynomials in Yacas.

Concepts and ideas are taken from the books [Davenport et al. 1989] and [von zur Gathen et al. 1999].


Definitions

The following definitions define multivariate polynomials, and the functions defined on them that are of interest for using such multivariates.

A term is an object which can be written as

c*x[1]^n[1]*x[2]^n[2]*...*x[m]^n[m]

for m variables ( x[1], ..., x[m]). The numbers n[m] are integers. c is called a coefficient, and x[1]^n[1]*x[2]^n[2]*...*x[m]^n[m] a monomial.

A multivariate polynomial is taken to be a sum over terms.

We write c[a]*x^a for a term, where a is a list of powers for the monomial, and c[a] the coefficient of the term.

It is useful to define an ordering of monomials, to be able to determine a canonical form of a multivariate.

For the currently implemented code the lexicographic order has been chosen:

This method is called lexicographic because it is similar to the way words are ordered in a usual dictionary.

For all algorithms (including division) there is some freedom in the ordering of monomials. One interesting advantage of the lexicographic order is that it can be implemented with a recursive data structure, where the first variable, x[1] can be treated as the main variable, thus presenting it as a univariate polynomial in x[1] with all its terms grouped together.

Other orderings can be used, by re-implementing a part of the code dealing with multivariate polynomials, and then selecting the new code to be used as a driver, as will be described later on.

Given the above ordering, the following definitions can be stated:

For a non-zero multivariate polynomial

f=Sum(a,a[max],a[min],c[a]*x^a)

with a monomial order:

The above define access to the leading monomial, which is used for divisions, gcd calculations and the like. Thus an implementation needs be able to determine {mdeg(f),lc(f)} . Note the similarity with the (key,value) pairs described in the sparse tree section. mdeg(f) can be thought of as a 'key', and lc(f) as a 'value'.

The multicontent, multicont(f), is defined to be a term that divides all the terms in f, and is the term described by ( Min(a), Gcd(c)), with Gcd(c) the GCD of all the coefficients, and Min(a) the lowest exponents for each variable, occurring in f for which c is non-zero.

The multiprimitive part is then defined as pp(f):=f/ multicont(f).

For a multivariate polynomial, the obvious addition and (distributive) multiplication rules hold:

(a+b) + (c+d) := a+b+c+d 

a*(b+c) := (a*b)+(a*c)

These are supported in the Yacas system through a multiply-add operation:

muadd(f,t,g):=f+t*g.

This allows for both adding two polynomials ( t:=1), or multiplication of two polynomials by scanning one polynomial, and multiplying each term of the scanned polynomial with the other polynomial, and adding the result to the polynomial that will be returned. Thus there should be an efficient muadd operation in the system.


Representation

For the representation of polynomials, on computers it is natural to do this in an array: ( a[1], a[2], ..., a[n]) for a univariate polynomial, and the equivalent for multivariates. This is called a dense representation, because all the coefficients are stored, even if they are zero. Computers are efficient at dealing with arrays. However, in the case of multivariate polynomials, arrays can become rather large, requiring a lot of storage and processing power even to add two such polynomials. For instance, x^200*y^100*z^300+1 could take 6000000 places in an array for the coefficients. Of course variables could be substituted for the single factors, p:=x^200 etc., but it requires an additional ad hoc step.

An alternative is to store only the terms for which the coefficients are non-zero. This adds a little overhead to polynomials that could efficiently be stored in a dense representation, but it is still little memory, whereas large sparse polynomials are stored in acceptable memory too. It is of importance to still be able to add, multiply divide and get the leading term of a multivariate polynomial, when the polynomial is stored in a sparse representation.

For the representation, the data structure containing the (exponents,coefficient) pair can be viewed as a database holding (key,value) pairs, where the list of exponents is the key, and the coefficient of the term is the value stored for that key. Thus, for a variable set {x,y} the list {{1,2},3} represents 3*x*y^2.

Yacas stores multivariates internally as MultiNomial (vars, terms), where vars is the ordered list of variables, and terms some object storing all the (key, value) pairs representing the terms. Note we keep the storage vague: the terms placeholder is implemented by other code, as a database of terms. The specific representation can be configured at startup (this is described in more detail below).

For the current version, Yacas uses the 'sparse tree' representation, which is a recursive sparse representation. For example, for a variable set {x,y,z}, the 'terms' object contains a list of objects of form {deg,terms}, one for each degree deg for the variable 'x' occurring in the polynomial. The 'terms' part of this object is then a sub-sparse tree for the variables {y,z}.

An explicit example:

In> MM(3*x^2+y)
Out> MultiNomial({x,y},{{2,{{0,3}}},{0,{{1,1},
  {0,0}}}});
The first item in the main list is {2,{{0,3}}}, which states that there is a term of the form x^2*y^0*3. The second item states that there are two terms, x^0*y^1*1 and x^0*y^0*0=0.

This representation is sparse:

In> r:=MM(x^1000+x)
Out> MultiNomial({x},{{1000,1},{1,1}});
and allows for easy multiplication:

In> r*r
Out> MultiNomial({x},{{2000,1},{1001,2},
  {2,1},{0,0}});
In> NormalForm(%)
Out> x^2000+2*x^1001+x^2;


Internal code organization

The implementation of multivariates can be divided in three levels.

At the top level are the routines callable by the user or the rest of the system: MultiDegree, MultiDivide, MultiGcd, Groebner, etc. In general, this is the level implementing the operations actually desired.

The middle level does the book-keeping of the MultiNomial(vars,terms) expressions, using the functionality offered by the lowest level.

For the current system, the middle level is in multivar.rep/ sparsenomial.ys, and it uses the sparse tree representation implemented in sparsetree.ys.

The middle level is called the 'driver', and can be changed, or re-implemented if necessary. For instance, in case calculations need to be done for which dense representations are actually acceptable, one could write C++ implementing above-mentioned database structure, and then write a middle-level driver using the code. The driver can then be selected at startup. In the file 'yacasinit.ys' the default driver is chosen, but this can be overridden in the .yacasrc file or some file that is loaded, or at the command line, as long as it is done before the multivariates module is loaded (which loads the selected driver). Driver selection is as simple as setting a global variable to contain a file name of the file implementing the driver:

Set(MultiNomialDriver,
  "multivar.rep/sparsenomial.ys");
where "multivar.rep/sparsenomial.ys" is the file implementing the driver (this is also the default driver, so the above command would not change any thing).

The choice was made for static configuration of the driver before the system starts up because it is expected that there will in general be one best way of doing it, given a certain system with a certain set of libraries installed on the operating system, and for a specific version of Yacas. The best version can then be selected at start up, as a configuration step. The advantage of static selection is that no overhead is imposed: there is no performance penalty for the abstraction layers between the three levels.


Driver interface

The driver should implement the following interface:


1.3 Integration

Integration can be performed by the function Integrate, which has two calling conventions:

Integrate can have its own set of rules for specific integrals, which might return a correct answer immediately. Alternatively, it calls the function AntiDeriv, to see if the anti-derivative can be determined for the integral requested. If this is the case, the anti-derivative is used to compose the output.

If the integration algorithm cannot perform the integral, the expression is returned unsimplified.


The integration algorithm

This section describes the steps taken in doing integration.


General structure

The integration starts at the function Integrate, but the task is delegated to other functions, one after the other. Each function can deem the integral unsolvable, and thus return the integral unevaluated. These different functions offer hooks for adding new types of integrals to be handled.


Expression clean-up

Integration starts by first cleaning up the expression, by calling TrigSimpCombine to simplify expressions containing multiplications of trigonometric functions into additions of trigonometric functions (for which the integration rules are trivial), and then passing the result to Simplify.


Generalized integration rules

For the function AntiDeriv, which is responsible for finding the anti-derivative of a function, the code splits up expressions according to the additive properties of integration, eg. integration of a+b is the same as integrating a + integrating b.


Integration tables

For elementary functions, Yacas uses integration tables. For instance, the fact that the anti-derivative of Cos(x) is Sin(x) is declared in an integration table.

For the purpose of setting up the integration table, a few declaration functions have been defined, which use some generalized pattern matchers to be more flexible in recognizing expressions that are integrable.


Integrating simple functions of a variable

For functions like Sin(x) the anti-derivative can be declared with the function IntFunc.

The calling sequence for IntFunc is

IntFunc(variable,pattern,antiderivative)

For instance, for the function Cos(x) there is a declaration:

IntFunc(x,Cos(_x),Sin(x));

The fact that the second argument is a pattern means that each occurrence of the variable to be matched should be referred to as _x, as in the example above.

IntFunc generalizes the integration implicitly, in that it will set up the system to actually recognize expressions of the form Cos(a*x+b), and return Sin(a*x+b)/a automatically. This means that the variables a and b are reserved, and can not be used in the pattern. Also, the variable used (in this case, _x is actually matched to the expression passed in to the function, and the variable var is the real variable being integrated over. To clarify: suppose the user wants to integrate Cos(c*y+d) over y, then the following variables are set:

When functions are multiplied by constants, that situation is handled by the integration rule that can deal with univariate polynomials multiplied by functions, as a constant is a polynomial of degree zero.


Integrating functions containing expressions of the form a*x^2+b

There are numerous expressions containing sub-expressions of the form a*x^2+b which can easily be integrated.

The general form for declaring anti-derivatives for such expressions is:

IntPureSquare(variable, pattern, sign2, sign0,
  antiderivative)
Here IntPureSquare uses MatchPureSquared to match the expression.

The expression is searched for the pattern, where the variable can match to a sub-expression of the form a*x^2+b, and for which both a and b are numbers and a*sign2>0 and b*sign0>0.

As an example:

IntPureSquare(x,num_IsFreeOf(var)/(_x),
  1,1,(num/(a*Sqrt(b/a)))*
  ArcTan(var/Sqrt(b/a)));
declares that the anti-derivative of c/(a*x^2+b) is

c/(a*Sqrt(b/a))*ArcTan(x/Sqrt(b/a)),

if both a and b are positive numbers.


1.4 Transforms

Currently the only tranform defined is LaplaceTransform, which has the calling convention:

It has been setup much like the integration algorithm. If the transformation algorithm cannot perform the transform, the expression (in theory) is returned unsimplified. Some cases may still erroneously return Undefined or Infinity.


The LaplaceTransform algorithm

This section describes the steps taken in doing a Laplace transform.


General structure

LaplaceTransform is immediately handed off to LapTran. This is done because if the last LapTran rule is met, the Laplace transform couldn't be found and it can then return LaplaceTransform unevaluated.


Operational Properties

The first rules that are matched against utilize the various operational properties of LaplaceTransform, such as:

The last operational property dealing with integration is not yet fully bug-tested, it sometimes returns Undefined or Infinity if the integral returns such.


Transform tables

For elementary functions, Yacas uses transform tables. For instance, the fact that the Laplace transform of Cos(t) is s/(s^2+1) is declared in a transform table.

For the purpose of setting up the transform table, a few declaration functions have been defined, which use some generalized pattern matchers to be more flexible in recognizing expressions that are transformable.


Transforming simple functions

For functions like Sin(t) the transform can be declared with the function LapTranDef.

The calling sequence for LapTranDef is

LapTranDef( in, out )

Currently in must be a variable of _t and out must be a function of s. For instance, for the function Cos(t) there is a declaration:

LapTranDef( Cos(_t),                    s/(s^2+1) );

The fact that the first argument is a pattern means that each occurrence of the variable to be matched should be referred to as _t, as in the example above.

LapTranDef generalizes the transform implicitly, in that it will set up the system to actually recognize expressions of the form Cos(a*t) and Cos(t/a) , and return the appropriate answer. The way this is done is by three separate rules for case of t itself, a*t and t/a. This is similar to the MatchLinear function that Integrate uses, except LaplaceTransforms must have b=0.


Further Directions

Currenlty Sin(t)*Cos(t) cannot be transformed, because it requires a convolution integral. This will be implemented soon. The inverse laplace transform will be implement soon also.


1.5 Finding real roots of polynomials

This section deals with finding roots of polynomials in the field of real numbers.

Without loss of generality, the coefficients a[i] of a polynomial

p=a[n]*x^n+...+a[0]

can be considered to be rational numbers, as real-valued numbers are truncated in practice, when doing calculations on a computer.

Assuming that the leading coefficient a[n]=1, the polynomial p can also be written as

p=p[1]^n[1]*...*p[m]^n[m],

where p[i] are the m distinct irreducible monic factors of the form p[i]=x-x[i], and n[i] are multiplicities of the factors. Here the roots are x[i] and some of them may be complex. However, complex roots of a polynomial with real coefficients always come in conjugate pairs, so the corresponding irreducible factors should be taken as p[i]=x^2+c[i]*x+d[i]. In this case, there will be less than m irreducible factors, and all coefficients will be real.

To find roots, it is useful to first remove the multiplicities, i.e. to convert the polynomial to one with multiplicity 1 for all irreducible factors, i.e. find the polynomial p[1]*...*p[m]. This is called the "square-free part" of the original polynomial p.

The square-free part of the polynomial p can be easily found using the polynomial GCD algorithm. The derivative of a polynomial p can be written as:

p'=Sum(i,1,m,p[1]^n[1]*...*n[i]*p[i]^(n[i]-1)*(D(x)p[i])*...*p[m]^n[m]).

The g.c.d. of p and p' equals

Gcd(p,p')=Factorize(i,1,m,p[i]^(n[i]-1)).

So if we divide p by Gcd(p,p'), we get the square-free part of the polynomial:

SquareFree(p):=Div(p,Gcd(p,p'))=p[1]*...*p[m].

In what follows we shall assume that all polynomials are square-free with rational coefficients. Given any polynomial, we can apply the functions SquareFree and Rationalize and reduce it to this form. The function Rationalize converts all numbers in an expression to rational numbers. The function SquareFree returns the square-free part of a polynomial. For example:

In> Expand((x+1.5)^5)
Out> x^5+7.5*x^4+22.5*x^3+33.75*x^2+25.3125*x
+7.59375;
In> SquareFree(Rationalize(%))
Out> x/5+3/10;
In> Simplify(%*5)
Out> (2*x+3)/2;
In> Expand(%)
Out> x+3/2;


Sturm sequences

For a polynomial p(x) of degree n, the Sturm sequence p[0], p[1], ... p[n] is defined by the following equations (following the book [Davenport et al. 1989]):

p[0]=p(x),

p[1]=p'(x),

p[i]= -remainder(p[i-2],p[i-1]),

where remainder(p,q) is the remainder of division of polynomials p, q.

The polynomial p can be assumed to have no multiple factors, and thus p and p' are relatively prime. The sequence of polynomials in the Sturm sequence are (up to a minus sign) the consecutive polynomials generated by Euclid's algorithm for the calculation of a greatest common divisor for p and p', so the last polynomial p[n] will be a constant.

In Yacas, the function SturmSequence(p) returns the Sturm sequence of p, assuming p is a univariate polynomial in x, p=p(x).

Given a Sturm sequence S=SturmSequence(p) of a polynomial p, the variation in the Sturm sequence V(S,y) is the number of sign changes in the sequence p[0], p[1] , ... , p[n], evaluated at point y, and disregarding zeroes in the sequence.

Sturm's theorem states that if a and b are two real numbers which are not roots of p, and a<b, then the number of roots between a and b is V(S,a)-V(S,b). A proof can be found in Knuth, The Art of Computer Programming, Volume 2, Seminumerical Algorithms.

For a and b, the values -Infinity and Infinity can also be used. In these cases, V(S,Infinity) is the number of sign changes between the leading coefficients of the elements of the Sturm sequence, and V(S,-Infinity) the same, but with a minus sign for the leading coefficients for which the degree is odd.

Thus, the number of real roots of a polynomial is V(S,-Infinity)-V(S,Infinity). The function NumRealRoots(p) returns the number of real roots of p.

The function SturmVariations(S,y) returns the number of sign changes between the elements in the Sturm sequence S, at point x=y:

In> p:=x^2-1
Out> x^2-1;
In> S:=SturmSequence(p)
Out> {x^2-1,2*x,1};
In> SturmVariations(S,-Infinity)- \
SturmVariations(S,Infinity)
Out> 2;
In> NumRealRoots(p)
Out> 2;
In> p:=x^2+1
Out> x^2+1;
In> S:=SturmSequence(p)
Out> {x^2+1,2*x,-1};
In> SturmVariations(S,-Infinity)- \
SturmVariations(S,Infinity)
Out> 0;
In> NumRealRoots(p)
Out> 0;


Finding bounds on real roots

Armed with the variations in the Sturm sequence given in the previous section, we can now find the number of real roots in a range ( a, b), for a<b. We can thus bound all the roots by subdividing ranges until there is only one root in each range. To be able to start this process, we first need some upper bounds of the roots, or an interval that contains all roots. Davenport gives limits on the roots of a polynomial given the coefficients of the polynomial, as

Abs(a)<=Max(Abs(a[n-1]/a[n]),Abs(a[n-2]/a[n])^(1/2),...,Abs(a[0]/a[n])^(1/n)),

for all real roots a of p. This gives the upper bound on the absolute value of the roots of the polynomial in question. if p(0)!=0, the minimum bound can be obtained also by considering the upper bound of p(1/x)*x^n, and taking 1/bound.

We thus know that given

a[max]=MaximumBound(p)

and

a[min]=MinimumBound(p)

for all roots a of polynomial, either

-a[max]<=a<= -a[min]

or

a[min]<=a<=a[max].

Now we can start the search for the bounds on all roots. The search starts with initial upper and lower bounds on ranges, subdividing ranges until a range contains only one root, and adding that range to the resulting list of bounds. If, when dividing a range, the middle of the range lands on a root, care must be taken, because the bounds should not be on a root themselves. This can be solved by observing that if c is a root, p contains a factor x-c, and thus taking p(x+c) results in a polynomial with all the roots shifted by a constant -c, and the root c moved to zero, e.g. p(x+c) contains a factor x. Thus a new ranges to the left and right of c can be determined by first calculating the minimum bound M of p(x+c)/x. When the original range was ( a, b), and c=(a+b)/2 is a root, the new ranges should become ( a, c-M) and ( c+M, b).

In Yacas, MimimumBound(p) returns the lower bound described above, and MaximumBound(p) returns the upper bound on the roots in p. These bounds are returned as rational numbers. BoundRealRoots(p) returns a list with sublists with the bounds on the roots of a polynomial:

In> p:=(x+20)*(x+10)
Out> (x+20)*(x+10);
In> MinimumBound(p)
Out> 10/3;
In> MaximumBound(p)
Out> 60;
In> BoundRealRoots(p)
Out> {{-95/3,-35/2},{-35/2,-10/3}};
In> N(%)
Out> {{-31.6666666666,-17.5},
  {-17.5,-3.3333333333}};

It should be noted that since all calculations are done with rational numbers, the algorithm for bounding the roots is very robust. This is important, as the roots can be very unstable for small variations in the coefficients of the polynomial in question (see Davenport).


Finding real roots given the bounds on the roots

Given the bounds on the real roots as determined in the previous section, two methods for finding roots are available: the secant method or the Newton method, where the function is locally approximated by a line, and extrapolated to find a new estimate for a root. This method converges quickly when "sufficiently" near a root, but can easily fail otherwise. The secant method can easily send the search to infinity.

The bisection method is more robust, but slower. It works by taking the middle of the range, and checking signs of the polynomial to select the half-range where the root is. As there is only one root in the range ( a, b), in general it will be true that p(a)*p(b)<0, which is assumed by this method.

Yacas finds the roots by first trying the secant method, starting in the middle of the range, c=(a+b)/2. If this fails the bisection method is tried.

The function call to find the real roots of a polynomial p in variable x is FindRealRoots(p), for example:

In> p:=Expand((x+3.1)*(x-6.23))
Out> x^2-3.13*x-19.313;
In> FindRealRoots(p)
Out> {-3.1,6.23};
In> p:=Expand((x+3.1)^3*(x-6.23))
Out> x^4+3.07*x^3-29.109*x^2-149.8199\ 
In> *x-185.59793;
In> p:=SquareFree(Rationalize( \ 
In> Expand((x+3.1)^3*(x-6.23))))
Out> (-160000*x^2+500800*x+3090080)/2611467;
In> FindRealRoots(p)
Out> {-3.1,6.23};