Problem:Let A be the 3 by 4 matrix given by, 
> A := matrix(3,4,[a,0,b,2, a,a,4,4, 0,a,2,b]);
[a 0 b 2] [ ] A := [a a 4 4] [ ] [0 a 2 b]
suppose A is the augmented matrix for a linear system.
Find the values of a and b for which the system have:


Let us perform elementary row operations to get the rref of A. To save typing we alias the maple commands that we need. 
> alias(P=swaprow,Q=mulrow,R=addrow):
First permute rows 1 and 2 and then (assuming a is not zero) divide the new first row by a. 
> A2 := Q( P(A,1,2), 1,1/a );
[1 1 4/a 4/a] [ ] A2 := [a 0 b 2 ] [ ] [0 a 2 b ]
to get zeroes below the leading 1 we replace the second row with the result of adding (a) times the first row to the second row. 
> A3 := R(A2,1,2,a);
[1 1 4/a 4/a] [ ] A3 := [0 a 4 + b 2 ] [ ] [0 a 2 b ]
now replace the third row by the result of adding the second to the third row. 
> A4 := R(A3,2,3,1);
[1 1 4/a 4/a ] [ ] A4 := [0 a 4 + b 2 ] [ ] [0 0 2 + b 2 + b]
To get rid of the condition that a must be different than zero we can multiply the first row by a. 
> A5 := Q(A4,1,a);
[a a 4 4 ] [ ] A5 := [0 a 4 + b 2 ] [ ] [0 0 2 + b 2 + b]> sol := backsub(A5);
[ 2 + b 2 + b ] sol := [ , , 1] [ a a ]
Therefore there is a unique solution when a is not zero AND b is not 2. Notice that when b =2 the last row of A5 is a row of zeroes. When b=2 have, 
> A6 := subs({b=2},evalm(A5));
[a a 4 4] [ ] A6 := [0 a 2 2] [ ] [0 0 0 0]> sol1 := backsub(A6);
[ 1 + _t[1] 1 + _t[1] ] sol1 := [2 , 2 , _t[1]] [ a a ]
Thus, when b=2 but a is not 0 we have a oneparameter solution. Maple uses _t[1] as the parameter. Now when both a=0 and b=2 the system becomes, 
> A7 := subs({a=0,b=2},evalm(A5));
[0 0 4 4] [ ] A7 := [0 0 2 2] [ ] [0 0 0 0]
By adding (1/2)row1 to row 2 we get 
> A8 := R(A7,1,2,1/2);
[0 0 4 4] [ ] A8 := [0 0 0 0] [ ] [0 0 0 0]> sol2 := backsub(A8);
sol2 := [_t[2], _t[1], 1]
so when b=2 and a=0 the system has a twoparameter set of solutions. In this case all we know is that z=1 but x and y are free to take any values. The only other possibility to consider is when a=0 but b is not equal to 2, 
> A9 := subs({a=0},evalm(A5));
[0 0 4 4 ] [ ] A9 := [0 0 4 + b 2 ] [ ] [0 0 2 + b 2 + b]
since b is not 2, we can divide the third row by b2 to get, 
> A10 := Q(A9,3,1/(b2));
[0 0 4 4] [ ] A10 := [0 0 4 + b 2] [ ] [0 0 1 1]
and performing 2 more R operations we get A10 in rref. 
> A11 := R( R(A10,3,1,4), 3,2,4b );
[0 0 0 0 ] [ ] A11 := [0 0 0 2  b] [ ] [0 0 1 1 ]
well actually A11 is not in rref... we need to reorder the rows and divide row2 by (2b). 
> A12 := P(Q(A11,2,1/(2b)),1,3);
[0 0 1 1] [ ] A12 := [0 0 0 1] [ ] [0 0 0 0]
Now it is clear that the system is inconsistent. Look at the second row it says that 0=1 ! 