Problem4:Find the solution to the system of equations with augmented matrix given by, 
> ;
[1 0 0 7 8] [ ] [0 1 0 3 2] [ ] [0 0 1 1 5]

Assume the unknowns are (w,x,y,z). Start from the last row working upwards. The last equation is, 
> Eq3 := y+z = 5;
Eq3 := y + z = 5
from where we get that 
> y := 5 z:
The second equation is, 
> Eq2 := x + 3* z = 2;
Eq2 := x + 3 z = 2
from where we obtain that, 
> x := 2  3*z:
finally the first equation is, 
> Eq1 := w  7*z = 8;
Eq1 := w  7 z = 8
from where we obtain that, 
> w := 8 + 7*z:
so there are infinitely many solutions to the system. All the points on the straight line 
> {[w,x,y,z]};
{[8 + 7 z, 2  3 z, 5  z, z]}
where z is an arbitrary scalar.
Ofcourse there are faster ways to get this answer with maple... here is one line solution, 
> backsub(matrix(3,5,[1,0,0,7,8, 0,1,0,3,2, 0,0,1,1,5]));
[8 + 7 _t[1], 2  3 _t[1], 5  _t[1], _t[1]]
_t[1] is maple funny way (to make sure the name is not being used by you) of writing the free scalar parameter z 