% ----------------------------------------------------------------------

\chapter
[Rank and the Gauss-Jordan]
{Matrix rank and the Gauss-Jordan decomposition}
\label{gjrank}

\index{matrix rudiments}
\index{rudiments}
Chapter~\ref{matrix} has brought the matrix and its rudiments, the
latter including
\bi
  \item lone-element matrix~$E$ (\S~\ref{matrix:180.30}),
  \item the null matrix~$0$ (\S~\ref{matrix:180.25}),
  \item the rank-$r$ identity matrix~$I_r$ (\S~\ref{matrix:180.22}),
  \item the general identity matrix~$I$ and the scalar matrix~$\lambda I$ (\S~\ref{matrix:180.35}),
  \item the elementary operator~$T$ (\S~\ref{matrix:320}),
  \item the quasielementary operator~$P$, $D$, $L_{[k]}$ or $U_{[k]}$ (\S~\ref{matrix:325}), and
  \item the unit triangular matrix~$L$ or~$U$ (\S~\ref{matrix:330}).
\ei
Such rudimentary forms have useful properties, as we have seen.  The
general matrix~$A$ does not necessarily have any of these properties,
but it turns out that one can factor any matrix whatsoever into a
product of rudiments which do have the properties, and that several
orderly procedures are known to do so.  The simplest of these, and
indeed one of the more useful, is the Gauss-Jordan decomposition.  This
chapter introduces it.

Section~\ref{matrix:180} has de\"emphasized the concept of matrix
dimensionality $m \times n$, supplying in its place the new concept of
matrix rank.  However, that section has actually defined rank only for
the rank-$r$ identity matrix~$I_r$.  In fact all matrices have rank.
This chapter explains.

Before treating the Gauss-Jordan decomposition and the matter of matrix
rank as such, however, we shall find it helpful to prepare two
preliminaries thereto: (i)~the matter of the linear independence of
vectors; and (ii)~the elementary similarity transformation.  The chapter
begins with these.

Except in \S~\ref{gjrank:337}, the chapter demands more rigor than one
likes in such a book as this.  However, it is hard to see how to avoid
the rigor here, and logically the chapter cannot be omitted.  We will
drive through the chapter in as few pages as can be managed, and then
onward to the more interesting matrix topics of chapters~\ref{mtxinv}
and~\ref{eigen}.

% ----------------------------------------------------------------------

\section{Linear independence}
\label{gjrank:335}
\index{linear independence}
\index{linear dependence}
\index{independence}

\index{linear combination}
\index{weighted sum}
\index{sum!weighted}
\index{triviality}
\index{coefficient!nontrivial}
Linear independence is a significant possible property of a set of
vectors---whether the set be the several columns of a matrix, the
several rows, or some other vectors---the property being
defined as follows.  A vector is \emph{linearly independent} if its role
cannot be served by the other vectors in the set.  More formally,
the~$n$ vectors of the set $\{\ve a_1,\ve a_2,\ve a_3,\ve a_4,\ve
a_5,\ldots,\ve a_n\}$ are linearly independent if and only if none of
them can be expressed as a linear combination---a weighted sum---of the
others.  That is, the several~$\ve a_k$ are linearly independent iff
\bq{gjrank:335:10}
    \alpha_1\ve a_1
  + \alpha_2\ve a_2
  + \alpha_3\ve a_3
  + \cdots
  + \alpha_n\ve a_n
  \neq 0
\eq
for all nontrivial~$\alpha_k$, where ``nontrivial~$\alpha_k$'' means the
several~$\alpha_k$, at least one of which is nonzero
(\emph{trivial}~$\alpha_k$, by contrast, would be
$\alpha_1=\alpha_2=\alpha_3=\cdots=\alpha_n=0$).  Vectors which can
combine nontrivially to reach the null vector are by definition
\emph{linearly dependent.}

\index{zero vector}
\index{null vector}
\index{$0$ (zero)!vector}
\index{vector!zero or null}
Linear independence is a property of vectors.  Technically the property
applies to scalars, too, inasmuch as a scalar resembles a one-element
vector---so, any nonzero scalar alone is linearly independent---but there
is no such thing as a linearly independent pair of scalars, because one
of the pair can always be expressed as a complex multiple of the other.
Significantly but less obviously, there is also no such thing as a
linearly independent set which includes the null
vector;~(\ref{gjrank:335:10}) forbids it.  Paradoxically, even the
single-member, $n=1$ set consisting only of $\ve a_1=0$ is, strictly
speaking, not linearly independent.

\index{empty set}
\index{edge case}
For consistency of definition, we regard the empty, $n=0$ set as linearly
independent, on the technical ground that the only possible linear
combination of the empty set is trivial.%
\footnote{
  This is the kind of thinking which typically governs
  mathematical edge cases.  One could define the empty set to be
  linearly dependent if one really wanted to, but what then of the
  observation that adding a vector to a linearly dependent set never
  renders the set independent?  Surely in this light it is preferable
  just to define the empty set as independent in the first place.
  Similar thinking makes $0!=1$, $\sum_{k=0}^{-1} a_kz^k = 0$, and~2
  not~1 the least prime, among other examples.
}

\index{vector!arbitrary}
If a linear combination of several independent vectors~$\ve a_k$ forms a
vector~$\ve b$, then one might ask: can there exist a different linear
combination of the same vectors~$\ve a_k$ which also forms~$\ve b$?
That is, if
\[
    \beta_1\ve a_1
  + \beta_2\ve a_2
  + \beta_3\ve a_3
  + \cdots
  + \beta_n\ve a_n
  = \ve b,
\]
where the several~$\ve a_k$ satisfy~(\ref{gjrank:335:10}), then is
\[
    \beta'_1\ve a_1
  + \beta'_2\ve a_2
  + \beta'_3\ve a_3
  + \cdots
  + \beta'_n\ve a_n
  = \ve b
\]
possible?  To answer the question, suppose that it were possible.  The
difference of the two equations then would be
\[
    (\beta_1'-\beta_1)\ve a_1
  + (\beta_2'-\beta_2)\ve a_2
  + (\beta_3'-\beta_3)\ve a_3
  + \cdots
  + (\beta_n'-\beta_n)\ve a_n
  = 0.
\]
According to~(\ref{gjrank:335:10}), this could only be so if the
coefficients in the last equation where trivial---that is, only if
  $\beta_1'-\beta_1=0$,
  $\beta_2'-\beta_2=0$,
  $\beta_3'-\beta_3=0$,
  \ldots,
  $\beta_n'-\beta_n=0$.
But this says no less than that the two linear combinations, which we
had supposed to differ, were in fact one and the same.  One concludes
therefore that, \emph{if a vector~$\ve b$ can be expressed as a linear
combination of several linearly independent vectors~$\ve a_k$, then it
cannot be expressed as any other combination of the same vectors.}  The
combination is unique.

\index{visualization, geometrical}
\index{geometrical visualization}
\index{abstraction}
Linear independence can apply in any dimensionality, but it helps to
visualize the concept geometrically in three dimensions, using the
three-dimensional geometrical vectors of \S~\ref{trig:230}.  Two such
vectors are independent so long as they do not lie along the same line.
A third such vector is independent of the first two so long as it does
not lie in their common plane.  A fourth such vector (unless it points
off into some unvisualizable fourth dimension) cannot possibly then be
independent of the three.

We discuss the linear independence of vectors in this, a chapter on
matrices, because (\S~\ref{matrix:120}) a matrix is essentially a
sequence of vectors---either of column vectors or of row vectors,
depending on one's point of view.  As we shall see in
\S~\ref{gjrank:340}, the important property of matrix \emph{rank}
depends on the number of linearly independent columns or rows a matrix
has.

% ----------------------------------------------------------------------

\section{The elementary similarity transformation}
\label{gjrank:337}
\index{elementary similarity transformation}
\index{similarity transformation}

Section~\ref{matrix:321} and its~(\ref{matrix:321:60}) have
introduced the \emph{similarity transformation}
% bad break
$CAC^{-1}$
or~$C^{-1}AC$, which arises when an operator~$C$ commutes respectively
rightward or leftward past a matrix~$A$.  The similarity transformation
has several interesting properties, some of which we are now
prepared to discuss, particularly in the case in which the operator
happens to be an elementary, $C=T$.  In this case, the several rules of
Table~\ref{gjrank:337:table} obtain.
\begin{table}
  \caption{Some elementary similarity transformations.}
  \label{gjrank:337:table}
  \settowidth\tla{${U^{\{k\}}}'$}
  \nc\xx{$L,U,L_{[k]},U_{[k]},L^{[k]},U^{[k]},L^{\{k\}},U^{\{k\}},L_\|^{\{k\}},U_\|^{\{k\}}$}
  \settowidth\tlb{\xx}
  \bqb
    T_{[i \lra j]} I T_{[i \lra j]} &=& I \\
    T_{[i \lra j]} P T_{[i \lra j]} &=& P' \\
    T_{[i \lra j]} D T_{[i \lra j]} &=& D' = D
      + \left([D]_{jj}-[D]_{ii}\right)E_{ii}
      + \left([D]_{ii}-[D]_{jj}\right)E_{jj} \\
    T_{[i \lra j]} D T_{[i \lra j]} &=& \makebox[\tla][l]{$D$}
      \ \ \mbox{if $[D]_{ii} = [D]_{jj}$} \\
    T_{[i \lra j]} L^{[k]}T_{[i \lra j]} &=& \makebox[\tla][l]{$L^{[k]}$}
      \ \ \mbox{if $i<k$ and $j<k$} \\
    T_{[i \lra j]} U^{[k]}T_{[i \lra j]} &=& \makebox[\tla][l]{$U^{[k]}$}
      \ \ \mbox{if $i>k$ and $j>k$} \\
    T_{[i \lra j]} L^{\{k\}}T_{[i \lra j]} &=& \makebox[\tla][l]{${L^{\{k\}}}'$}
      \ \ \mbox{if $i>k$ and $j>k$} \\
    T_{[i \lra j]} U^{\{k\}}T_{[i \lra j]} &=& \makebox[\tla][l]{${U^{\{k\}}}'$}
      \ \ \mbox{if $i<k$ and $j<k$} \\
    T_{[i \lra j]} L_\|^{\{k\}}T_{[i \lra j]} &=& \makebox[\tla][l]{${L_\|^{\{k\}}}'$}
      \ \ \mbox{if $i>k$ and $j>k$} \\
    T_{[i \lra j]} U_\|^{\{k\}}T_{[i \lra j]} &=& \makebox[\tla][l]{${U_\|^{\{k\}}}'$}
      \ \ \mbox{if $i<k$ and $j<k$} \\
    T_{\alpha[i]} I T_{(1/\alpha)[i]} &=& I \\
    T_{\alpha[i]} D T_{(1/\alpha)[i]} &=& D \\
    T_{\alpha[i]} A T_{(1/\alpha)[i]} &=& \makebox[\tla][l]{$A'$} \ \ %
      \parbox[t]{\tlb}{
        where~$A$ is any of \\ \xx
      }
    \\
    T_{\alpha[ij]} I T_{-\alpha[ij]} &=& I \\
    T_{\alpha[ij]} D T_{-\alpha[ij]} &=& D + \left([D]_{jj}-[D]_{ii}\right)\alpha E_{ij} \neq D' \\
    T_{\alpha[ij]} D T_{-\alpha[ij]} &=& \makebox[\tla][l]{$D$}
      \ \ \mbox{if $[D]_{ii} = [D]_{jj}$} \\
    T_{\alpha[ij]} L T_{-\alpha[ij]} &=& \makebox[\tla][l]{$L'$} \ \ \mbox{if $i > j$} \\
    T_{\alpha[ij]} U T_{-\alpha[ij]} &=& \makebox[\tla][l]{$U'$} \ \ \mbox{if $i < j$}
  \eqb
\end{table}

Most of the table's rules are fairly obvious if the meaning of the symbols is
understood, though to grasp some of the rules it helps to sketch the relevant
matrices on a sheet of paper.  Of course rigorous symbolic proofs can be constructed
after the pattern of \S~\ref{matrix:330.30}, but they reveal little or nothing
sketching the matrices does not.  In Table~\ref{gjrank:337:table} as
elsewhere, the symbols~$P$, $D$, $L$ and~$U$
represent the quasielementaries and unit triangular matrices of
\S\S~\ref{matrix:325} and~\ref{matrix:330}.  The symbols~$P'$, $D'$, $L'$
and~$U'$ also represent quasielementaries and unit triangular matrices, only not
necessarily the same ones~$P$, $D$, $L$ and~$U$ do.

The rules of Table~\ref{gjrank:337:table}
permit one to commute some but not all elementaries past a
quasielementary operator or unit triangular matrix
without fundamentally altering the character of the quasielementary
operator or unit triangular matrix, and sometimes
without changing it at all.  The rules find use among other
places in the Gauss-Jordan decomposition of \S~\ref{gjrank:341}.

% ----------------------------------------------------------------------

\section{The Gauss-Jordan decomposition}
\label{gjrank:341}
\index{Gauss, Carl Friedrich (1777--1855)}
\index{Jordan, Wilhelm (1842--1899)}
\index{Gauss-Jordan decomposition}
\index{decomposition!Gauss-Jordan}
\index{Gauss-Jordan factorization}
\index{factorization!Gauss-Jordan}

\index{reversibility}
\index{$LU$ decomposition}
\index{decomposition!$LU$}
The \emph{Gauss-Jordan decomposition} of an arbitrary, dimension-limited, $m
\times n$ matrix~$A$ is%
\footnote{
  Most introductory linear algebra texts this writer has met call
  the Gauss-Jordan decomposition instead the ``$LU$ decomposition'' and
  include fewer factors in it, typically merging~$D$ into~$L$ and
  omitting~$K$ and~$S$.  They also omit~$I_r$, since their matrices have
  pre-defined dimensionality.  Perhaps the reader will agree that the
  decomposition is cleaner as presented here.
}
\bq{gjrank:341:GJ}
  \begin{split}
    A = G_> I_r G_<
      & = P D L U I_r K S, \\
    G_< &\equiv K S, \\
    G_> &\equiv P D L U,
  \end{split}
\eq
where
\bi
  \item
    $P$ and~$S$ are general interchange operators (\S~\ref{matrix:325.10});
  \item
    $D$ is a general scaling operator (\S~\ref{matrix:325.20});
  \item
    $L$ and~$U$ are respectively unit lower and unit upper triangular matrices (\S~\ref{matrix:330});
  \item
    $K=L_\|^{\{r\}T}$ is the transpose of a parallel unit lower triangular
    matrix, being thus a parallel unit upper triangular matrix
    (\S~\ref{matrix:330.50});
  \item
    $G_>$ and~$G_<$ are composites%
    \footnote{
      One can pronounce~$G_>$ and~$G_<$ respectively as~``$G$ acting
      rightward'' and~``$G$ acting leftward.''  The letter~$G$ itself
      can be regarded as standing for ``Gauss-Jordan,'' but admittedly
      it is chosen as much because otherwise we were running out of
      available Roman capitals!
    }
    as defined by~(\ref{gjrank:341:GJ}); and
  \item
    $r$ is an unspecified rank.
\ei
The Gauss-Jordan decomposition is also called the \emph{Gauss-Jordan
factorization.}

Whether all possible dimension-limited, $m\times n$ matrices~$A$
have a Gauss-Jordan decomposition
(they do, in fact) is a matter this section addresses.  However---at
least for matrices which do have one---because~$G_>$ and~$G_<$ are
composed of invertible factors, one can left-multiply the equation $A =
G_> I_r G_<$ by~$G_>^{-1}$ and right-multiply it by~$G_<^{-1}$ to obtain
\bq{gjrank:341:GJinv}
  \begin{split}
    U^{-1} L^{-1} D^{-1} P^{-1} A S^{-1} K^{-1} & =
      G_>^{-1} A G_<^{-1} = I_r, \\
    S^{-1} K^{-1} &= G_<^{-1}, \\
    U^{-1} L^{-1} D^{-1} P^{-1} &= G_>^{-1},
  \end{split}
\eq
the Gauss-Jordan's complementary form.

\subsection{Motive}
\label{gjrank:341.01}

Equation~(\ref{gjrank:341:GJ}) seems inscrutable.  The equation itself is
easy enough to read, but just as there are many ways to factor a scalar
($\mbox{0xC} = [4][3] = [2]^2[3] = [2][6]$, for example), there are
likewise many ways to factor a matrix.  Why choose this particular way?

There are indeed many ways.  We shall meet some of the others in
\S\S~\ref{mtxinv:460}, \ref{eigen:423}, \ref{eigen:520}
and~\ref{eigen:600}.
The Gauss-Jordan decomposition we meet here however
has both significant theoretical properties and useful practical
applications, and in any case needs less advanced preparation to
appreciate than the others, and (at least as developed in this book)
precedes the others logically.  It emerges naturally when one posits a
pair of dimension-limited, square,
$n \times n$ matrices,~$A$ and~$A^{-1}$, for which
$A^{-1} A = I_n$, where~$A$ is known and~$A^{-1}$ is to be determined.
[The~$A^{-1}$ here is the $A^{-1(n)}$ of eqn.~\ref{matrix:321:20}.
However, it is only supposed here that $A^{-1} A = I_n$;
it is not \emph{yet} claimed that $A A^{-1} = I_n$.  ``Square'' means
that the matrix has an $n\times n$ active region rather than an
$m\times n$, $m\neq n$, where ``active region'' is defined as in
\S~\ref{matrix:180.25}.]

To determine~$A^{-1}$ is not an entirely trivial problem.  The
matrix~$A^{-1}$ such that $A^{-1} A = I_n$ may or may not exist (usually
it does exist if~$A$ is square, but even then it may not, as
we shall soon see), and even if it does exist,
how to determine it is not immediately obvious.  And still, if one can
determine~$A^{-1}$, that is only for square~$A$; what if~$A$, having an
$m\times n$, $m\neq n$, active region, were not
square?  In the present subsection however we are not trying to prove
anything, only to motivate, so for the moment let us suppose an~$A$ for
which~$A^{-1}$ does exist, let us confine our attention to square~$A$,
and let us seek~$A^{-1}$ by
left-multiplying~$A$ by a sequence $\prod T$ of elementary row
operators, each of which makes the matrix more nearly resemble~$I_n$.
When~$I_n$ is finally achieved, then we shall have that
\[
  \left(\prod T\right) (A) = I_n,
\]
or, left-multiplying by~$I_n$ and observing that $I_n^2=I_n$,
\[
  (I_n) \left(\prod T\right) (A) = I_n,
\]
which implies that
\[
  A^{-1} = (I_n) \left(\prod T\right).
\]
The product of elementaries which transforms~$A$ to~$I_n$, truncated
(\S~\ref{matrix:180.23}) to $n \times n$ dimensionality, itself
constitutes~$A^{-1}$.  This observation is what motivates the
Gauss-Jordan decomposition.

{
  \nc\fz[1]{\mbox{\footnotesize$\ds{#1}$}}
  \settowidth\tla{\footnotesize $-0$}
  By successive steps,%
  \footnote{
    Theoretically, all elementary operators including the ones here have
    extended-operational form~(\S~\ref{matrix:180.35}), but all
    those~$\cdots$ ellipses clutter the page too much.  Only the $2
    \times 2$ active regions are shown here.
  }
  a concrete example:
  \bqb
    A &=& \fz{\left[ \br{rr} 2 & -4 \\ 3 & -1 \er \right]}, \\
      \fz{
        \left[ \br{rr} \frac 1 2 & 0 \\ 0 & 1 \er \right]
      }
    A &=& \fz{\left[ \br{rr} 1 & -2 \\ 3 & -1 \er \right]}, \\
      \fz{
        \left[ \br{rr} 1 & 0 \\ -3 & 1 \er \right]
        \left[ \br{rr} \frac 1 2 & 0 \\ 0 & 1 \er \right]
      }
    A &=& \fz{\left[ \br{rr} 1 & -2 \\ 0 & 5 \er \right]}, \\
      \fz{
        \left[ \br{rr} 1 & 0 \\ 0 & \frac 1 5 \er \right]
        \left[ \br{rr} 1 & 0 \\ -3 & 1 \er \right]
        \left[ \br{rr} \frac 1 2 & 0 \\ 0 & 1 \er \right]
      }
    A &=& \fz{
        \left[ \br{rr} 1 & -2 \\ 0 & 1 \er \right]
      }, \\
      \fz{
        \left[ \br{rr} 1 & 2 \\ 0 & 1 \er \right]
        \left[ \br{rr} 1 & 0 \\ 0 & \frac 1 5 \er \right]
        \left[ \br{rr} 1 & 0 \\ -3 & 1 \er \right]
        \left[ \br{rr} \frac 1 2 & 0 \\ 0 & 1 \er \right]
      }
    A &=& \fz{\left[ \br{rr} 1 & \makebox[\tla][r]{$0$} \\ 0 & 1 \er \right]}, \\
      \fz{
        \left[ \br{rr} 1 & 0 \\ 0 & 1 \er \right]
        \left[ \br{rr} 1 & 2 \\ 0 & 1 \er \right]
        \left[ \br{rr} 1 & 0 \\ 0 & \frac 1 5 \er \right]
        \left[ \br{rr} 1 & 0 \\ -3 & 1 \er \right]
        \left[ \br{rr} \frac 1 2 & 0 \\ 0 & 1 \er \right]
      }
    A &=& \fz{\left[ \br{rr} 1 & \makebox[\tla][r]{$0$} \\ 0 & 1 \er \right]}.
  \eqb
  Hence,
  \[
    \renewcommand\arraystretch{1.3}
    A^{-1} = \fz{
      \left[ \br{rr} 1 & 0 \\ 0 & 1 \er \right]
      \left[ \br{rr} 1 & 2 \\ 0 & 1 \er \right]
      \left[ \br{rr} 1 & 0 \\ 0 & \frac 1 5 \er \right]
      \left[ \br{rr} 1 & 0 \\ -3 & 1 \er \right]
      \left[ \br{rr} \frac 1 2 & 0 \\ 0 & 1 \er \right]
    }
    = \fz{ \left[ \br{rr} -\frac 1 {\mathrm A} & \frac 2 5 \\ -\frac 3 {\mathrm A} & \frac 1 5 \er \right] }.
  \]
  Using the elementary commutation identity
  that $T_{\beta[m]}T_{\alpha[mj]} = T_{\alpha\beta[mj]}T_{\beta[m]}$,
  from Table~\ref{matrix:Txchg3}, to
  group like operators, we have that
  \[
    \renewcommand\arraystretch{1.3}
    A^{-1} = \fz{
      \left[ \br{rr} 1 & 0 \\ 0 & 1 \er \right]
      \left[ \br{rr} 1 & 2 \\ 0 & 1 \er \right]
      \left[ \br{rr} 1 & 0 \\ -\frac 3 5 & 1 \er \right]
      \left[ \br{rr} 1 & 0 \\ 0 & \frac 1 5 \er \right]
      \left[ \br{rr} \frac 1 2 & 0 \\ 0 & 1 \er \right]
    }
    = \fz{ \left[ \br{rr} -\frac 1 {\mathrm A} & \frac 2 5 \\ -\frac 3 {\mathrm A} & \frac 1 5 \er \right] };
  \]
  or, multiplying the two scaling elementaries to merge them into a
  single general scaling operator (\S~\ref{matrix:325.20}),
  \[
    \renewcommand\arraystretch{1.3}
    A^{-1} = \fz{
      \left[ \br{rr} 1 & 0 \\ 0 & 1 \er \right]
      \left[ \br{rr} 1 & 2 \\ 0 & 1 \er \right]
      \left[ \br{rr} 1 & 0 \\ -\frac 3 5 & 1 \er \right]
      \left[ \br{rr} \frac 1 2 & 0 \\ 0 & \frac 1 5 \er \right]
    }
    = \fz{ \left[ \br{rr} -\frac 1 {\mathrm A} & \frac 2 5 \\ -\frac 3 {\mathrm A} & \frac 1 5 \er \right] }.
  \]
  The last equation is written symbolically as
  \[
    A^{-1} = I_2 U^{-1} L^{-1} D^{-1},
  \]
  from which
  \[
    A = D L U I_2 = \fz{
      \left[ \br{rr} 2 & 0 \\ 0 & 5 \er \right]
      \left[ \br{rr} 1 & 0 \\ \frac 3 5 & 1 \er \right]
      \left[ \br{rr} 1 & -2 \\ 0 & 1 \er \right]
      \left[ \br{rr} 1 & 0 \\ 0 & 1 \er \right]
    } = \fz{\left[ \br{rr} 2 & -4 \\ 3 & -1 \er \right]}.
  \]
}

Now, admittedly, the equation $A = D L U I_2$ is
not~(\ref{gjrank:341:GJ})---or rather, it is~(\ref{gjrank:341:GJ}), but
only in the special case that $r = 2$ and $P = S = K = I$---which begs
the question: why do we need the factors~$P$, $S$
and~$K$ in the first place?  The answer regarding~$P$ and~$S$ is
that these factors respectively gather row and column interchange elementaries,
of which the example given has used none but which other examples sometimes
need or want, particularly to avoid dividing by zero when they encounter
a zero in an inconvenient cell of the matrix (the reader might try
reducing $A=[0\;1; 1\;0]$ to~$I_2$, for instance; a row or column
interchange is needed here).  Regarding~$K$, this factor comes into
play when~$A$ has broad rectangular ($m<n$) rather than square ($m=n$)
shape, and also
sometimes when one of the rows of~$A$ happens to be a linear combination
of the others.  The last point, we are not quite ready to detail yet,
but at present we are only motivating not proving,
so if the reader will accept the other factors and suspend
judgment on~$K$ until the actual need for it emerges in
\S~\ref{gjrank:341.10}, step~\ref{gjrank:341:s75}, then we will proceed
on this basis.

\subsection{Method}
\label{gjrank:341.05}

The Gauss-Jordan decomposition of a matrix~$A$ is not discovered at one
stroke but rather is gradually built up, elementary by elementary.   It
begins with the equation
\[
  A = I I I I A I I,
\]
where the six~$I$ hold the places of the six Gauss-Jordan
factors~$P$, $D$, $L$, $U$, $K$ and~$S$ of~(\ref{gjrank:341:GJ}).
By successive elementary operations, the~$A$ on the right is gradually
transformed into~$I_r$, while the six~$I$ are gradually transformed into
the six Gauss-Jordan factors.  The decomposition thus ends with the
equation
\[
  A = P D L U I_r K S,
\]
which is~(\ref{gjrank:341:GJ}).  In between, while the several matrices
are gradually being transformed, the equation is represented as
\bq{gjrank:341:05}
  A = \tilde P \tilde D \tilde L \tilde U \tilde I \tilde K \tilde S,
\eq
where the initial value of~$\tilde I$ is~$A$ and the initial values
of~$\tilde P$, $\tilde D$, etc., are all~$I$.

Each step of the transformation goes as follows.  The matrix~$\tilde I$
is left- or right-multiplied by an elementary operator~$T$.  To
compensate, one of the six factors is right- or left-multiplied
by~$T^{-1}$.  Intervening factors are multiplied by both~$T$
and~$T^{-1}$, which multiplication constitutes an elementary similarity
transformation as described in \S~\ref{gjrank:337}.  For example,
\[
  A = \tilde P
    \left(\tilde DT_{(1/\alpha)[i]}\right)
    \left(T_{\alpha[i]}\tilde LT_{(1/\alpha)[i]}\right)
    \left(T_{\alpha[i]}\tilde UT_{(1/\alpha)[i]}\right)
    \left(T_{\alpha[i]}\tilde I\right)
    \tilde K \tilde S,
\]
which is just~(\ref{gjrank:341:05}), inasmuch as the adjacent
elementaries cancel one another; then,
\[
  \begin{split}
    \tilde I &\la T_{\alpha[i]}\tilde I, \\
    \tilde U &\la T_{\alpha[i]}\tilde UT_{(1/\alpha)[i]}, \\
    \tilde L &\la T_{\alpha[i]}\tilde LT_{(1/\alpha)[i]}, \\
    \tilde D &\la \tilde DT_{(1/\alpha)[i]},
  \end{split}
\]
thus associating the operation with the appropriate factor---in this
case,~$\tilde D$.  Such elementary row and column operations are repeated
until $\tilde I=I_r$, at which point~(\ref{gjrank:341:05}) has become the
Gauss-Jordan decomposition~(\ref{gjrank:341:GJ}).

\subsection{The algorithm}
\label{gjrank:341.10}
\index{Gauss-Jordan algorithm}
\index{algorithm!Gauss-Jordan}

% I have checked the algorithm against one 4x5 case using GNU Octave.
% It seems to work, including row and column swaps (one of each).

Having motivated the Gauss-Jordan decomposition in
\S~\ref{gjrank:341.01} and having proposed a basic method to pursue it
in \S~\ref{gjrank:341.05}, we shall now establish a definite, orderly,
failproof algorithm to achieve it.  Broadly, the algorithm
\bi
  \item
    copies~$A$, a dimension-limited, $m\times n$ matrix (not necessarily
    square), into the variable working matrix~$\tilde I$
    (step~\ref{gjrank:341:s10} below),
  \item
    reduces~$\tilde I$ by suitable row (and maybe column) operations to unit upper
    triangular form (steps~\ref{gjrank:341:s18}
    through~\ref{gjrank:341:s50}),
  \item
    establishes a rank~$r$ (step~\ref{gjrank:341:s55}), and
  \item
    reduces the now unit triangular~$\tilde I$ further to the rank-$r$ identity
    matrix~$I_r$  (steps~\ref{gjrank:341:s60}
    through~\ref{gjrank:341:s85}).
\ei
Specifically, the algorithm decrees the following steps.  (The steps as
written include many parenthetical remarks---so many that some steps
seem to consist more of parenthetical remarks than of actual algorithm.
The remarks are unnecessary to execute the algorithm's steps as such.
They are however necessary to explain and to justify the algorithm's
steps to the reader.)
\begin{enumerate}
  \item \label{gjrank:341:s10}
    Begin by initializing
    \[
      \br{c}
        \tilde P \la I, \
        \tilde D \la I, \
        \tilde L \la I, \
        \tilde U \la I, \
        \tilde K \la I, \
        \tilde S \la I, \\
        \setlength\arraycolsep{0.30\arraycolsep}
        \br{rcl}
          \tilde I &\la& A, \\
          i &\la& 1,
        \er
      \er
    \]
    where~$\tilde I$ holds the part of~$A$ remaining to be decomposed,
    where~$i$ is a row index, and where the others are the variable
    working matrices of~(\ref{gjrank:341:05}).  (The eventual goal will
    be to factor all of~$\tilde I$ away, leaving $\tilde I=I_r$, though
    the precise value of~$r$ will not be known until
    step~\ref{gjrank:341:s55}.  Since~$A$ is by definition a
    dimension-limited $m \times n$ matrix, one naturally need not
    store~$A$ beyond the $m \times n$ active region.  What is less clear
    until one has read the whole algorithm, but nevertheless true, is
    that one also need not store the dimension-limited~$\tilde I$ beyond
    the $m \times n$ active region.  The other six variable working
    matrices each have extended-operational form, but they also confine
    their activity to well-defined regions: $m \times m$
    for~$\tilde P$, $\tilde D$, $\tilde L$ and~$\tilde U$; $n \times n$
    for~$\tilde K$ and~$\tilde S$.  One need store none of the matrices
    beyond these bounds.)
  \item \label{gjrank:341:s18} \index{pivot}
    (Besides arriving at this point from step~\ref{gjrank:341:s10} above, the
    algorithm also re\"enters here from step~\ref{gjrank:341:s50} below.
    From step~\ref{gjrank:341:s10}, $\tilde I = A$ and $\tilde L =
    I$, so this step~\ref{gjrank:341:s18} though logical seems unneeded.  The need
    grows clear once one has read through step~\ref{gjrank:341:s50}.)
    Observe that neither the $i$th row of~$\tilde I$ nor any row below it has
    an entry left of the $i$th column, that~$\tilde I$ is all-zero
    below-leftward of and directly leftward of (though not directly
    below) the \emph{pivot} element~$\tdi_{ii}$.%
    \footnote{
      The notation~$\tdi_{ii}$ looks interesting, but this is accidental.
      The~$\tdi$ relates not to the doubled, subscribed index~$ii$ but
      to~$\tilde I$.  The notation~$\tdi_{ii}$ thus means $[\tilde
      I]_{ii}$---in other words, it means the current $ii$th element of
      the variable working matrix~$\tilde I$.
    }
    Observe also that above the $i$th row, the matrix has proper unit upper
    triangular form (\S~\ref{matrix:330}).  Regarding the other factors,
    notice that~$\tilde L$ enjoys the major partial unit triangular
    form $L^{\{i-1\}}$ (\S~\ref{matrix:330.05}) and that $\tilde d_{kk} = 1$
    for all $k \ge i$.  Pictorially,
    \bqb
      \tilde D &=& \mf{ccccccccc}{
        \ddots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\\
        \cdots&{*}&0&0&0&0&0&0&\cdots\\
        \cdots&0&{*}&0&0&0&0&0&\cdots\\
        \cdots&0&0&{*}&0&0&0&0&\cdots\\
        \cdots&0&0&0&1&0&0&0&\cdots\\
        \cdots&0&0&0&0&1&0&0&\cdots\\
        \cdots&0&0&0&0&0&1&0&\cdots\\
        \cdots&0&0&0&0&0&0&1&\cdots\\
        &\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\ddots
      },\\
      \tilde L = L^{\{i-1\}} &=& \mf{ccccccccc}{
        \ddots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\\
        \cdots&1&0&0&0&0&0&0&\cdots\\
        \cdots&{*}&1&0&0&0&0&0&\cdots\\
        \cdots&{*}&{*}&1&0&0&0&0&\cdots\\
        \cdots&{*}&{*}&{*}&1&0&0&0&\cdots\\
        \cdots&{*}&{*}&{*}&0&1&0&0&\cdots\\
        \cdots&{*}&{*}&{*}&0&0&1&0&\cdots\\
        \cdots&{*}&{*}&{*}&0&0&0&1&\cdots\\
        &\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\ddots
      },\\
      \tilde I &=& \mf{ccccccccc}{
        \ddots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\\
        \cdots&1&{*}&{*}&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&1&{*}&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&1&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&{*}&{*}&{*}&{*}&\cdots\\
        &\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\ddots
      },
    \eqb
    where the $i$th row and $i$th column are depicted at center.
  \item \label{gjrank:341:s20}
    \index{Intel} \index{AMD}
    \index{x86-class computer processor}
    \index{computer processor}
    \index{processor, computer}
    \index{bit}
    \index{floating-point number}
    \index{mantissa}
    \index{exponent, floating-point}
    \index{exact arithmetic}
    \index{arithmetic!exact}
    Choose a nonzero element $\tdi_{pq} \neq 0$ on or below the pivot
    row, where $p \ge i$ and $q \ge i$.  (The easiest choice may simply
    be~$\tdi_{ii}$, where $p=q=i$, if $\tdi_{ii} \neq 0$; but any
    nonzero element from the $i$th row downward can in general be
    chosen.  Beginning students of the Gauss-Jordan or~$LU$ decomposition
    are conventionally taught to choose first the least possible~$q$
    and then the least possible~$p$.  When one has no reason to choose
    otherwise, that is as good a choice as any.  There is however no
    actual need to choose so.  In fact alternate choices can sometimes
    improve practical numerical accuracy.%
    \footnote{
      A typical Intel or AMD x86-class computer processor represents a
      C/C++ \texttt{double}-type floating-point number, $x=2^pb$,
      in~0x40 bits of computer memory.  Of the~0x40 bits,~0x34 are for
      the number's mantissa $2.0 \le b < 4.0$ (not $1.0 \le b < 2.0$ as
      one might expect),~0xB are for the number's exponent
      $-\mbox{0x3FF} \le p \le \mbox{0x3FE}$, and one is for the
      number's~$\pm$ sign.  (The mantissa's high-order bit, which is
      always~1, is implied not stored, being thus one neither of the~0x34
      nor of the~0x40 bits.)  The out-of-bounds exponents
      $p=-\mbox{0x400}$ and $p=\mbox{0x3FF}$ serve specially
      respectively to encode~0 and~$\infty$.  All this is standard
      computing practice.  Such a floating-point representation is
      easily accurate enough for most practical purposes, but of course
      it is not generally exact.\ \cite[\S~1-4.2.2]{Intel}
    }$\mbox{}^,$\footnote{
      The Gauss-Jordan's floating-point errors come mainly from
      dividing by small pivots.  Such errors are naturally avoided by
      avoiding small pivots, at least until as late in the algorithm as
      possible.  Smallness however is relative: a small pivot in a row
      and a column each populated by even smaller elements is unlikely
      to cause as much error as is a large pivot in a row and a column
      each populated by even larger elements.

      To choose a pivot, any of several heuristics is reasonable.  The
      following heuristic if programmed intelligently might not be too
      computationally expensive:  Define the pivot-smallness metric
      \settoheight\tla{\tiny$*$}
      \[
        \tilde\eta_{pq}^2 \equiv \frac{2\tdi_{pq}^{*}\tdi_{pq}^{\rule{0em}{\tla}}}{
          \sum_{p'=i}^m \tdi_{p'q}^{*}\tdi_{p'q}^{\rule{0em}{\tla}}
          + \sum_{q'=i}^n \tdi_{pq'}^{*}\tdi_{pq'}^{\rule{0em}{\tla}}
        }.
      \]
      Choose the~$p$ and~$q$ of least~$\tilde\eta_{pq}^2$.  If two are
      equally least, then choose first the lesser column index~$q$ and then
      if necessary the lesser row index~$p$.
    }
    Theoretically nonetheless, when doing exact arithmetic, the choice
    is quite arbitrary, so long as $\tdi_{pq} \neq 0$.)  If no nonzero
    element is available---if all remaining rows $p \ge i$ are now
    null---then skip directly to step~\ref{gjrank:341:s55}.
  \item \label{gjrank:341:s25}
    \index{row} \index{matrix!row of}
    \index{column} \index{matrix!column of}
    Observing that~(\ref{gjrank:341:05}) can be expanded to read
    \bqb
      A &=&
        \left( \tilde P T_{[p \lra i]} \right)
        \left( T_{[p \lra i]} \tilde D T_{[p \lra i]} \right)
        \left( T_{[p \lra i]} \tilde L T_{[p \lra i]} \right)
        \left( T_{[p \lra i]} \tilde U T_{[p \lra i]} \right)
        \\ && \ \ \mbox{} \times
        \left( T_{[p \lra i]} \tilde I T_{[i \lra q]} \right)
        \left( T_{[i \lra q]} \tilde K T_{[i \lra q]} \right)
        \left( T_{[i \lra q]} \tilde S \right)
      \\ &=&
        \left( \tilde P T_{[p \lra i]} \right) \tilde D
        \left( T_{[p \lra i]} \tilde L T_{[p \lra i]} \right) \tilde U
        \\ && \ \ \mbox{} \times
        \left( T_{[p \lra i]} \tilde I T_{[i \lra q]} \right) \tilde K
        \left( T_{[i \lra q]} \tilde S \right),
    \eqb
    let
    \[
      \begin{split}
        \tilde P &\la \tilde P T_{[p\lra i]}, \\
        \tilde L &\la T_{[p\lra i]} \tilde L T_{[p\lra i]}, \\
        \tilde I &\la T_{[p\lra i]} \tilde I T_{[i\lra q]}, \\
        \tilde S &\la T_{[i\lra q]} \tilde S,
      \end{split}
    \]
    thus interchanging the $p$th with the $i$th row and the $q$th with
    the $i$th column, to bring the chosen element to the pivot position.
    (Refer to Table~\ref{gjrank:337:table} for the similarity
    transformations.  The~$\tilde U$ and~$\tilde K$ transformations
    disappear because at this stage of the algorithm, still
    $\tilde U = \tilde K = I$.  The~$\tilde D$ transformation
    disappears because $p \ge i$ and because $\tilde d_{kk}=1$ for all $k \ge
    i$.  Regarding the~$\tilde L$ transformation, it does not disappear,
    but~$\tilde L$ has major partial unit triangular form $L^{\{i-1\}}$,
    which form according to Table~\ref{gjrank:337:table} it retains
    since $i-1 < i \le p$.)
  \item \label{gjrank:341:s40}
    Observing that~(\ref{gjrank:341:05}) can be expanded to read
    \bqb
      A &=& \tilde P
        \left( \tilde D T_{\tdi_{ii}[i]} \right)
        \left( T_{(1/\tdi_{ii})[i]} \tilde L T_{\tdi_{ii}[i]} \right)
        \left( T_{(1/\tdi_{ii})[i]} \tilde U T_{\tdi_{ii}[i]} \right)
        \\ && \ \ \mbox{} \times
        \left( T_{(1/\tdi_{ii})[i]} \tilde I \right)
        \tilde K \tilde S
      \\ &=& \tilde P
        \left( \tilde D T_{\tdi_{ii}[i]} \right)
        \left( T_{(1/\tdi_{ii})[i]} \tilde L T_{\tdi_{ii}[i]} \right)
        \tilde U
        \left( T_{(1/\tdi_{ii})[i]} \tilde I \right)
        \tilde K \tilde S,
    \eqb
    normalize the new~$\tdi_{ii}$ pivot by letting
    \[
      \begin{split}
        \tilde D &\la \tilde D T_{\tdi_{ii}[i]}, \\
        \tilde L &\la T_{(1/\tdi_{ii})[i]} \tilde L T_{\tdi_{ii}[i]}, \\
        \tilde I &\la T_{(1/\tdi_{ii})[i]} \tilde I.
      \end{split}
    \]
    This forces $\tdi_{ii} = 1$.  It also changes the value
    of~$\tilde d_{ii}$.  Pictorially after this step,
    \bqb
      \tilde D &=& \mf{ccccccccc}{
        \ddots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\\
        \cdots&{*}&0&0&0&0&0&0&\cdots\\
        \cdots&0&{*}&0&0&0&0&0&\cdots\\
        \cdots&0&0&{*}&0&0&0&0&\cdots\\
        \cdots&0&0&0&{*}&0&0&0&\cdots\\
        \cdots&0&0&0&0&1&0&0&\cdots\\
        \cdots&0&0&0&0&0&1&0&\cdots\\
        \cdots&0&0&0&0&0&0&1&\cdots\\
        &\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\ddots
      },\\
      \tilde I &=& \mf{ccccccccc}{
        \ddots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\\
        \cdots&1&{*}&{*}&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&1&{*}&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&1&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&1&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&{*}&{*}&{*}&{*}&\cdots\\
        &\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\ddots
      }.
    \eqb
    (Though the step changes~$\tilde L$, too, again it leaves~$\tilde L$
    in the major partial unit triangular form $L^{\{i-1\}}$, because $i-1<i$.
    Refer to Table~\ref{gjrank:337:table}.)
  \item \label{gjrank:341:s45}
    Observing that~(\ref{gjrank:341:05}) can be expanded to read
    \bqb
      A &=& \tilde P \tilde D
        \left( \tilde L T_{\tdi_{pi}[pi]} \right)
        \left( T_{-\tdi_{pi}[pi]} \tilde U T_{\tdi_{pi}[pi]} \right)
        \left( T_{-\tdi_{pi}[pi]} \tilde I \right)
        \tilde K \tilde S
      \\ &=& \tilde P \tilde D
        \left( \tilde L T_{\tdi_{pi}[pi]} \right)
        \tilde U
        \left( T_{-\tdi_{pi}[pi]} \tilde I \right)
        \tilde K \tilde S,
    \eqb
    clear $\tilde I$'s $i$th column below the pivot by letting
    \[
      \begin{split}
        \tilde L &\la \left( \tilde L \right)
          \left( \coprod_{p=i+1}^mT_{\tdi_{pi}[pi]} \right), \\
        \tilde I &\la \left( \prod_{p=i+1}^mT_{-\tdi_{pi}[pi]} \right)
          \left( \tilde I \right).
      \end{split}
    \]
    This forces $\tdi_{ip}=0$ for all $p>i$.  It also fills in $\tilde
    L$'s $i$th column below the pivot, advancing that matrix from
    the $L^{\{i-1\}}$ form to the $L^{\{i\}}$ form.  Pictorially,
    \bqb
      \tilde L = L^{\{i\}} &=& \mf{ccccccccc}{
        \ddots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\\
        \cdots&1&0&0&0&0&0&0&\cdots\\
        \cdots&{*}&1&0&0&0&0&0&\cdots\\
        \cdots&{*}&{*}&1&0&0&0&0&\cdots\\
        \cdots&{*}&{*}&{*}&1&0&0&0&\cdots\\
        \cdots&{*}&{*}&{*}&{*}&1&0&0&\cdots\\
        \cdots&{*}&{*}&{*}&{*}&0&1&0&\cdots\\
        \cdots&{*}&{*}&{*}&{*}&0&0&1&\cdots\\
        &\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\ddots
      },\\
      \tilde I &=& \mf{ccccccccc}{
        \ddots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\\
        \cdots&1&{*}&{*}&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&1&{*}&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&1&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&1&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&0&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&0&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&0&{*}&{*}&{*}&\cdots\\
        &\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\ddots
      }.
    \eqb
    (Note that it is not necessary actually to apply the addition
    elementaries here one by one.  Together they easily form an addition
    quasielementary $L_{[i]}$, and thus can be applied all at once.  See
    \S~\ref{matrix:325.30}.)
  \item \label{gjrank:341:s50}
    Increment
    \[
      i\la i+1.
    \]
    Go back to step~\ref{gjrank:341:s18}.
  \item \label{gjrank:341:s55}
    Decrement
    \[
      i\la i-1
    \]
    to undo the last instance of step~\ref{gjrank:341:s50} (even if
    there never was an instance of step~\ref{gjrank:341:s50}), thus
    letting~$i$ point to the matrix's last nonzero row.  After
    decrementing, let the rank
    \[
      r \equiv i.
    \]
    Notice that, certainly, $r \le m$ and $r \le n$.
  \item \label{gjrank:341:s60}
    (Besides arriving at this point from step~\ref{gjrank:341:s55} above, the
    algorithm also re\"enters here from step~\ref{gjrank:341:s70} below.)
    If $i=0$, then skip directly to step~\ref{gjrank:341:s75}.
  \item \label{gjrank:341:s65}
    Observing that~(\ref{gjrank:341:05}) can be expanded to read
    \[
      A = \tilde P \tilde D \tilde L
        \left( \tilde U T_{\tdi_{pi}[pi]} \right)
        \left( T_{-\tdi_{pi}[pi]} \tilde I \right)
        \tilde K \tilde S,
    \]
    clear $\tilde I$'s $i$th column above the pivot by letting
    \[
      \begin{split}
        \tilde U &\la \left( \tilde U \right)
          \left( \prod_{p=1}^{i-1}T_{\tdi_{pi}[pi]} \right), \\
        \tilde I &\la \left( \coprod_{p=1}^{i-1}T_{-\tdi_{pi}[pi]} \right)
          \left( \tilde I \right).
      \end{split}
    \]
    This forces $\tdi_{ip}=0$ for all $p \neq i$.  It also fills in $\tilde
    U$'s $i$th column above the pivot, advancing that matrix from the
    $U^{\{i+1\}}$ form to the $U^{\{i\}}$ form.  Pictorially,
    \bqb
      \tilde U = U^{\{i\}} &=& \mf{ccccccccc}{
        \ddots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\\
        \cdots&1&0&0&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&1&0&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&1&{*}&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&1&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&0&1&{*}&{*}&\cdots\\
        \cdots&0&0&0&0&0&1&{*}&\cdots\\
        \cdots&0&0&0&0&0&0&1&\cdots\\
        &\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\ddots
      },\\
      \tilde I &=& \mf{ccccccccc}{
        \ddots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\\
        \cdots&1&{*}&{*}&0&0&0&0&\cdots\\
        \cdots&0&1&{*}&0&0&0&0&\cdots\\
        \cdots&0&0&1&0&0&0&0&\cdots\\
        \cdots&0&0&0&1&0&0&0&\cdots\\
        \cdots&0&0&0&0&1&0&0&\cdots\\
        \cdots&0&0&0&0&0&1&0&\cdots\\
        \cdots&0&0&0&0&0&0&1&\cdots\\
        &\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\ddots
      }.
    \eqb
    (As in step~\ref{gjrank:341:s45}, here again it is not necessary
    actually to apply the addition elementaries one by one.  Together
    they easily form an addition quasielementary $U_{[i]}$.  See
    \S~\ref{matrix:325.30}.)
  \item \label{gjrank:341:s70}
    Decrement $i\la i-1$.  Go back to step~\ref{gjrank:341:s60}.
  \item \label{gjrank:341:s75}
    \index{extra column}
    \index{spare column}
    \index{column!spare}
    Notice that~$\tilde I$ now has the form of a rank-$r$ identity
    matrix, except with $n-r$ extra columns dressing its right edge
    (often $r=n$ however; then there are no extra columns).
    Pictorially,
    \[
      \tilde I = \mf{ccccccccc}{
        \ddots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\\
        \cdots&1&0&0&0&{*}&{*}&{*}&\cdots\\
        \cdots&0&1&0&0&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&1&0&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&1&{*}&{*}&{*}&\cdots\\
        \cdots&0&0&0&0&0&0&0&\cdots\\
        \cdots&0&0&0&0&0&0&0&\cdots\\
        \cdots&0&0&0&0&0&0&0&\cdots\\
        &\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\ddots
      }.
    \]
    Observing that~(\ref{gjrank:341:05}) can be expanded to read
    \[
      A = \tilde P \tilde D \tilde L \tilde U
        \left( \tilde I T_{-\tdi_{pq}[pq]} \right)
        \left( T_{\tdi_{pq}[pq]} \tilde K \right)
        \tilde S,
    \]
    use the now conveniently elementarized columns of $\tilde I$'s main
    body to suppress the extra columns on its right edge by
    \[
      \begin{split}
        \tilde I &\la \left( \tilde I \right)
          \left( \prod_{q=r+1}^n \, \coprod_{p=1}^r T_{-\tdi_{pq}[pq]} \right), \\
        \tilde K &\la
          \left( \coprod_{q=r+1}^n \, \prod_{p=1}^r T_{\tdi_{pq}[pq]} \right)
          \left( \tilde K \right).
      \end{split}
    \]
    (Actually, entering this step, it was that $\tilde K = I$, so in
    fact~$\tilde K$ becomes just the product above.  As in
    steps~\ref{gjrank:341:s45} and~\ref{gjrank:341:s65}, here again it
    is not necessary actually to apply the addition elementaries one by
    one.  Together they easily form a parallel unit upper---not
    lower---triangular matrix $L_\|^{\{r\}T}$.  See
    \S~\ref{matrix:330.50}.)
  \item \label{gjrank:341:s85}
    Notice now that $\tilde I=I_r$.  Let
    \[
      P \equiv \tilde P, \
      D \equiv \tilde D, \
      L \equiv \tilde L, \
      U \equiv \tilde U, \
      K \equiv \tilde K, \
      S \equiv \tilde S.
    \]
    End.
\end{enumerate}
Never stalling, the algorithm cannot fail to achieve $\tilde I=I_r$ and
thus a complete Gauss-Jordan decomposition of the
form~(\ref{gjrank:341:GJ}), though what value the rank~$r$ might turn
out to have is not normally known to us in advance.  (We have not yet
proven, but will in \S~\ref{gjrank:340}, that the algorithm always
produces the same~$I_r$, the same rank $r \ge 0$, regardless of which
pivots $\tdi_{pq}\neq 0$ one happens to choose in
step~\ref{gjrank:341:s20} along the way.  We can safely ignore this
unproven fact however for the immediate moment.)

\subsection{Rank and independent rows}
\label{gjrank:341.11}
\index{rank!and independent rows}
\index{rank!maximum}

Observe that the Gauss-Jordan algorithm of \S~\ref{gjrank:341.10}
operates always within the bounds of the original $m\times n$
matrix~$A$.  Therefore, necessarily,
\bq{gjrank:341:22}
  \begin{split}
    r &\le m, \\
    r &\le n.
  \end{split}
\eq
The rank~$r$ exceeds the number neither of the matrix's rows nor of
its columns.  This is unsurprising.  Indeed the narrative of the
algorithm's step~\ref{gjrank:341:s55} has already noticed the fact.

Observe also however that \emph{the rank always fully reaches $r=m$ if
the rows of the original matrix~$A$ are linearly independent.}  The
reason for this observation is that the rank can fall short,
$r<m$, only if step~\ref{gjrank:341:s20} finds a null row $i\le m$;
but step~\ref{gjrank:341:s20} can find such a null row only if
step~\ref{gjrank:341:s45} has created one (or if there were a null row
in the original matrix~$A$; but according to \S~\ref{gjrank:335}, such
null rows never were linearly independent in the first place).  How do
we know that step~\ref{gjrank:341:s45} can never create a null row?  We
know this because the action of step~\ref{gjrank:341:s45} is to add
multiples only of \emph{current and earlier} pivot rows to rows
in~$\tilde I$ which have not yet been on pivot.%
\footnote{
  If the truth of the sentence's assertion regarding the action of
  step~\ref{gjrank:341:s45} seems nonobvious, one can drown the
  assertion rigorously in symbols to prove it, but before going to that
  extreme consider: The action of steps~\ref{gjrank:341:s20}
  and~\ref{gjrank:341:s25} is to choose a pivot row $p \ge i$ and to
  shift it upward to the $i$th position.  The action of
  step~\ref{gjrank:341:s45} then is to add multiples of the chosen pivot
  row downward only---that is, only to rows which have not yet been on
  pivot.  This being so, steps~\ref{gjrank:341:s20}
  and~\ref{gjrank:341:s25} in the second iteration find no unmixed rows
  available to choose as second pivot, but find only rows which already
  include multiples of the first pivot row.  Step~\ref{gjrank:341:s45}
  in the second iteration therefore adds downward multiples of the second
  pivot row, \emph{which already includes a multiple of the first pivot
  row.}  Step~\ref{gjrank:341:s45} in the $i$th iteration adds downward
  multiples of the $i$th pivot row, which already includes multiples of
  the first through $(i-1)$th.  So it comes to pass that multiples only
  of current and earlier pivot rows are added to rows which have not yet
  been on pivot.  To no row is ever added, directly or indirectly, a
  multiple of itself---until step~\ref{gjrank:341:s65}, which does not
  belong to the algorithm's main loop and has nothing to do with the
  availability of nonzero rows to step~\ref{gjrank:341:s20}.
}
According to~(\ref{gjrank:335:10}), such action has no power to cancel
the independent rows it targets.

\subsection{Inverting the factors}
\label{gjrank:341.30}
\index{Gauss-Jordan decomposition!inverting the factors of}

Inverting the six Gauss-Jordan factors is easy.
Sections~\ref{matrix:325} and~\ref{matrix:330} have shown how.
One need not however go even to that much trouble.  Each of the six
factors---$P$, $D$, $L$, $U$, $K$ and~$S$---is composed of a sequence
$\prod T$ of elementary operators.  Each of the six inverse
factors---$P^{-1}$, $D^{-1}$, $L^{-1}$, $U^{-1}$, $K^{-1}$
and~$S^{-1}$---is therefore composed of the \emph{reverse} sequence
$\coprod T^{-1}$ of \emph{inverse} elementary operators.  Refer
to~(\ref{matrix:320:34}).  If one merely records the sequence of
elementaries used to build each of the six factors---if one reverses
each sequence, inverts each elementary, and multiplies---then the six
inverse factors result.

And, in fact, it isn't even that hard.  One actually need not record the
individual elementaries; one can invert, multiply and forget them in
stream.  This means starting the algorithm from
step~\ref{gjrank:341:s10} with six extra variable working matrices
(besides the seven already there):
\[
  \tilde P^{-1} \la I; \
  \tilde D^{-1} \la I; \
  \tilde L^{-1} \la I; \
  \tilde U^{-1} \la I; \
  \tilde K^{-1} \la I; \
  \tilde S^{-1} \la I.
\]
(There is no~$\tilde I^{-1}$, not because it would not be useful, but
because its initial value would be%
\footnote{
  Section~\ref{matrix:321} explains the notation.
}
$A^{-1(r)}$, unknown at algorithm's
start.)  Then, for each operation on any of~$\tilde P$, $\tilde D$,
$\tilde L$, $\tilde U$, $\tilde K$ or~$\tilde S$, one operates inversely
on the corresponding inverse matrix.  For example, in
step~\ref{gjrank:341:s40},
\[
  \setlength\arraycolsep{0.30\arraycolsep}
  \br{rclcrcl}
    \tilde D &\la& \tilde D T_{\tdi_{ii}[i]}, &\ \ \ \ &
    \tilde D^{-1} &\la& T_{(1/\tdi_{ii})[i]} \tilde D^{-1}, \\
    \tilde L &\la& T_{(1/\tdi_{ii})[i]} \tilde L T_{\tdi_{ii}[i]}, &&
    \tilde L^{-1} &\la& T_{(1/\tdi_{ii})[i]} \tilde L^{-1} T_{\tdi_{ii}[i]}, \\
    \tilde I &\la& T_{(1/\tdi_{ii})[i]} \tilde I. &&&&
  \er
\]
With this simple extension, the algorithm yields all the factors not
only of the Gauss-Jordan decomposition~(\ref{gjrank:341:GJ}) but
simultaneously also of the Gauss-Jordan's complementary
form~(\ref{gjrank:341:GJinv}).

\subsection{Truncating the factors}
\label{gjrank:341.40}
\index{truncation}
\index{factor!truncating}

None of the six factors of~(\ref{gjrank:341:GJ}) actually needs to
retain its entire extended-operational form (\S~\ref{matrix:180.35}).
The four factors on the left, row operators, act wholly by their $m
\times m$ squares; the two on the right, column operators, by their $n
\times n$.  Indeed, neither~$I_r$ nor~$A$ has anything but zeros outside
the $m \times n$ rectangle, anyway, so there is nothing for the six
operators to act upon beyond those bounds in any event.  We can truncate
all six operators to dimension-limited forms (\S~\ref{matrix:180.25})
for this reason if we want.

To truncate the six operators formally, we
left-multiply~(\ref{gjrank:341:GJ}) by~$I_m$ and right-multiply it
by~$I_n$, obtaining
\[
  I_mAI_n = I_mPDLUI_rKSI_n.
\]
According to \S~\ref{matrix:180.23}, the~$I_m$ and~$I_n$ respectively
truncate rows and columns, actions which have no effect on~$A$ since it
is already a dimension-limited $m \times n$ matrix.  By successive
steps, then,
\bqb
  A &=& I_mPDLUI_rKSI_n
  \\&=& I_m^7PDLUI_r^2KSI_n^3;
\eqb
and finally, by using~(\ref{matrix:180:35}) or~(\ref{matrix:320:39})
repeatedly,
\bq{gjrank:341:GJt}
  A = (I_mPI_m)(I_mDI_m)(I_mLI_m)(I_mUI_r)(I_rKI_n)(I_nSI_n),
\eq
where the dimensionalities of the six factors on the equation's right
side are respectively
  $m \times m$,
  $m \times m$,
  $m \times m$,
  $m \times r$,
  $r \times n$ and
  $n \times n$.
Equation~(\ref{gjrank:341:GJt}) expresses any dimension-limited
rectangular matrix~$A$ as the product of six particularly simple,
dimension-limited rectangular factors.

By similar reasoning from~(\ref{gjrank:341:GJ}),
\bq{gjrank:341:GJtC}
  A = (I_mG_>I_r)(I_rG_<I_n),
\eq
where the dimensionalities of the two factors are
  $m \times r$ and
  $r \times n$.

The book will seldom point it out again explicitly, but one can
straightforwardly truncate not only the Gauss-Jordan factors but most
other factors and operators, too, by the method of this subsection.%
\footnote{
  Comes the objection, ``Black, why do you make it more complicated
  than it needs to be?  For what reason must all your matrices have
  infinite dimensionality, anyway?  They don't do it that way in my
  other linear algebra book.''

  It is a fair question.  The answer is that this book is a book of
  applied mathematical theory; and theoretically in the author's view,
  infinite-dimensional matrices are significantly neater to handle.  To
  append a null row or a null column to a dimension-limited matrix is to
  alter the matrix in no essential way, nor is there any real difference
  between $T_{5[21]}$ when it row-operates on a $3 \times p$ matrix and
  the same elementary when it row-operates on a $4 \times p$.  The
  relevant theoretical constructs ought to reflect such insights.  Hence
  infinite dimensionality.

  Anyway, a matrix displaying an infinite field of zeros resembles a
  shipment delivering an infinite supply of nothing; one need not be too
  impressed with either.  The two matrix forms of \S~\ref{matrix:180}
  manifest the sense that a matrix can represent a linear
  transformation, whose rank matters; or a reversible row or column
  operation, whose rank does not.  The extended-operational form, having
  infinite rank, serves the latter case.  In either case, however, the
  dimensionality $m \times n$ of the matrix is a distraction.  It is the
  rank~$r$, if any, that counts.
}

\subsection{Properties of the factors}
\label{gjrank:341.43}
\index{Gauss-Jordan factors!properties of}

\index{schematic}
\index{pencil}
One would expect such neatly formed operators as the factors of the
Gauss-Jordan to enjoy some useful special properties.  Indeed they do.
Table~\ref{gjrank:341:t34} lists a few.
\begin{table}
  \caption{A few properties of the Gauss-Jordan factors.}
  \label{gjrank:341:t34}
  \[
    \renewcommand\arraystretch{1.3}
    \setlength\arraycolsep{0.30\arraycolsep}
    \br{rcccl}
      P^{*} &=& P^{-1} &=& P^{T} \\
      S^{*} &=& S^{-1} &=& S^{T} \\
      P^{-*} &=& P &=& P^{-T} \\
      S^{-*} &=& S &=& S^{-T}
    \er
  \]
  \[
      \frac{K + K^{-1}}{2} = I
  \]
  \[
    \renewcommand\arraystretch{1.3}
    \br{rcccl}
      I_r K (I_n-I_r) &=& K-I &=& I_r (K-I) (I_n-I_r) \\
      I_r K^{-1} (I_n-I_r) &=& K^{-1}-I &=& I_r (K^{-1}-I) (I_n-I_r) \\
      (I-I_n)(K-I) &=& 0 &=& (K-I)(I-I_n) \\
      (I-I_n)(K^{-1}-I) &=& 0 &=& (K^{-1}-I)(I-I_n)
    \er
  \]
\end{table}
The table's properties formally come from~(\ref{matrix:325:11}) and
Table~\ref{matrix:330:t18}; but, if one firmly grasps the matrix forms
involved and comprehends the notation (neither of which is trivial to
do), if one understands that the operator $(I_n-I_r)$ is a truncator that
selects columns $r+1$ through~$n$ of the matrix it operates leftward
upon, and if one sketches the relevant factors schematically with a
pencil, then the properties are plainly seen without reference to
chapter~\ref{matrix} as such.

The table's properties regarding~$P$ and~$S$ express a general advantage
all permutors share.  The table's properties regarding~$K$ are
admittedly less significant, included mostly only because
\S~\ref{mtxinv:245} will need them.  Still, even the~$K$ properties are
always true.  They might find other uses.

Further properties of the several Gauss-Jordan factors can be gleaned
from the respectively relevant subsections of \S\S~\ref{matrix:325}
and~\ref{matrix:330}.

\subsection{Marginalizing the factor~$I_n$}
\label{gjrank:341.55}

If~$A$ happens to be a square, $n \times n$ matrix and if it develops
that the rank $r=n$, then one can take advantage
of~(\ref{matrix:180:35}) to rewrite the Gauss-Jordan
decomposition~(\ref{gjrank:341:GJ}) in the form
\bq{gjrank:341:55}
  PDLUKSI_n = A = I_nPDLUKS,
\eq
thus marginalizing the factor~$I_n$.  This is to express the Gauss-Jordan
solely in row operations or solely in column operations.
It does not change the algorithm and it does not alter the factors; it
merely reorders the factors after the algorithm has determined them.  It
fails however if~$A$ is rectangular or $r<n$.

\subsection{Decomposing an extended operator}
\label{gjrank:341.45}
\index{extended operator!decomposing}

Sections~\ref{gjrank:340} and~\ref{mtxinv:220} will demonstrate that,
if a matrix~$A$ is a extended operator (\S~\ref{matrix:180.35}) with
an $n\times n$ active region and if the operation the matrix implements
is reversible, then the truncated operator $I_nA=I_nAI_n=AI_n$ necessarily
enjoys full rank $r=n$.  Complementarily, those sections will
demonstrate that, if such an extended operator is irreversible,
then the truncated operator $I_nA=I_nAI_n=AI_n$ must suffer $r<n$.  Full
rank is associated with reversibility.  This fact has many consequences,
among which is the following.
%(where per \S~\ref{matrix:180.35} $A$ outside the $n \times n$ active
%region resembles the infinite-dimensional identity matrix~$I$)

To extend the Gauss-Jordan decomposition of the present section to
decompose a reversible, $n \times n$ extended operator~$A$ is trivial.
One merely writes
\[
  A = PDLUKS,
\]
wherein the~$I_r$ has become an~$I$.  Or, equivalently, one decomposes the
$n \times n$ dimension-limited matrix $I_n A = I_n A I_n = A I_n$ as
\[
  A I_n = PDLUI_nKS = PDLUKSI_n,
\]
from which, inasmuch as all the factors present but~$I_n$ are $n \times
n$ extended operators, the preceding equation results.

One can decompose only reversible extended operators so.  The Gauss-Jordan
fails on irreversible extended operators.  Fortunately, as we have seen in
chapter~\ref{matrix}, every extended operator constructible as the
product of elementary, quasielementary,
% The style elsewhere in the book has been "unit triangular," but that
% does not seem to work here.  The hyphenated form is thus strictly
% irregular, but it should be all right.
unit-triangular and/or shift operators is indeed reversible, so a great
variety of extended operators are decomposable.  (Note incidentally that
shifts, \S~\ref{matrix:340}, generally prevent the extended operators
whose construction includes them from honoring any finite, $n\times n$
active region.  Therefore, in this subsection we are generally thinking
of extended operators constructible without shifts.)

This subsection's equations remain unnumbered because they
say little new.  Their only point, really, is that what an operator does
outside some appropriately delimited active region is seldom
interesting because the vector on which the operator ultimately acts is
probably null there in any event.  In such a context it may not matter
whether one truncates the operator.  Indeed, this was also the point of
\S~\ref{gjrank:341.40} and, if you like, of~(\ref{matrix:180:35}), too.%
\footnote{
  If ``it may not matter,'' as the narrative says, then one might just
  put all matrices in dimension-limited form.  Many books do.  To put
  them all in dimension-limited form however brings at least three
  effects the book you are reading prefers to avoid.  First, it leaves
  shift-and-truncate operations hard to express cleanly (refer to
  \S\S~\ref{matrix:180.23} and~\ref{matrix:340} and, as a typical
  example of the usage, eqn.~\ref{mtxinv:245:kernel}).  Second, it
  confuses the otherwise natural extension of discrete vectors into
  continuous functions.  Third, it leaves one to consider the ranks of
  reversible operators like $T_{[1\lra 2]}$ that naturally should have
  no rank.  The last of the three is arguably most significant: matrix
  rank is such an important attribute that one prefers to impute it only
  to those operators about which it actually says something interesting.

  Nevertheless, the extended-operational matrix form is hardly more than
  a formality.  All it says is that the extended operator unobtrusively
  leaves untouched anything it happens to find outside its operational
  domain, whereas a dimension-limited operator would have truncated
  whatever it found there.  Since what is found outside the operational
  domain is typically uninteresting, this may be a distinction without a
  difference, a distinction one can safely ignore.
}

Regarding the present section as a whole, the Gauss-Jordan decomposition
is a significant achievement.  It is not the only matrix
decomposition---further interesting decompositions include the
Gram-Schmidt of \S~\ref{mtxinv:460}, the diagonal of \S~\ref{eigen:423},
the Schur of \S~\ref{eigen:520} and the singular-value of
\S~\ref{eigen:600},
among others---but the Gauss-Jordan nonetheless reliably factors an
arbitrary $m \times n$ matrix~$A$, which we had not known how to handle
very well, into a product of unit triangular matrices and
quasielementaries, which we do.  We shall put the Gauss-Jordan to good
use in chapter~\ref{mtxinv}.  However, before closing the present chapter we
should like finally, squarely to define and to establish the concept of
matrix rank, not only for~$I_r$ but for all matrices.  To do that, we
shall first need one more preliminary: the technique of vector
replacement.

% ----------------------------------------------------------------------

\section{Vector replacement}
\label{gjrank:338}
\index{vector!replacement of}

\index{addressing a vector space}
\index{vector space}
\index{space}
\index{vector!arbitrary}
Consider a set of $m+1$ (not necessarily independent) vectors
\[
  \{\ve u,\ve a_1,\ve a_2,\ldots,\ve a_m\}.
\]
As a definition, the \emph{space} these vectors \emph{address} consists
of all linear combinations of the set's several vectors.  That is, the
space consists of all vectors~$\ve b$ formable as
\bq{gjrank:338:10}
  \beta_o\ve u+\beta_1\ve a_1+\beta_2\ve a_2+\cdots+\beta_m\ve a_m = \ve b.
\eq

Now consider a specific vector~$\ve v$ in the space,
\bq{gjrank:338:15}
  \psi_o\ve u+\psi_1\ve a_1+\psi_2\ve a_2+\cdots+\psi_m\ve a_m = \ve v,
\eq
for which
\[
  \psi_o \neq 0.
\]
Solving~(\ref{gjrank:338:15}) for~$\ve u$, we find that
\[
  \frac{1}{\psi_o}\ve v
  -\frac{\psi_1}{\psi_o}\ve a_1
  -\frac{\psi_2}{\psi_o}\ve a_2
  -\cdots
  -\frac{\psi_m}{\psi_o}\ve a_m
  = \ve u.
\]
With the change of variables
\bqb
  \phi_o &\la& \frac{1}{\psi_o}, \\
  \phi_1 &\la& -\frac{\psi_1}{\psi_o}, \\
  \phi_2 &\la& -\frac{\psi_2}{\psi_o}, \\
  &\vdots&\\
  \phi_m &\la& -\frac{\psi_m}{\psi_o},
\eqb
for which, quite symmetrically, it happens that
\bqb
  \psi_o &=& \frac{1}{\phi_o}, \\
  \psi_1 &=& -\frac{\phi_1}{\phi_o}, \\
  \psi_2 &=& -\frac{\phi_2}{\phi_o}, \\
  &\vdots&\\
  \psi_m &=& -\frac{\phi_m}{\phi_o},
\eqb
the solution is
\bq{gjrank:338:17}
  \phi_o\ve v+\phi_1\ve a_1+\phi_2\ve a_2+\cdots+\phi_m\ve a_m = \ve u.
\eq
Equation~(\ref{gjrank:338:17}) has identical form to~(\ref{gjrank:338:15}),
only with the symbols $\ve u \lra \ve v$ and $\psi \lra \phi$ swapped.
Since $\phi_o = 1/\psi_o$, assuming that~$\psi_o$ is finite it even
appears that
\[
  \phi_o \neq 0;
\]
so, the symmetry is complete.  Table~\ref{gjrank:338:table} summarizes.
\begin{table}
  \caption{The symmetrical equations of \S~\ref{gjrank:338}.}
  \label{gjrank:338:table}
  \index{symmetry}
  \bc
    \setlength\tla{3.5ex}
    \renewcommand\arraystretch{2.0}
    \[
      \br{rclcrcl}
        +\cdots+\psi_m\ve a_m
          \makebox[0em][r]{\raisebox{\tla}{$\psi_o\ve u+\psi_1\ve a_1+\psi_2\ve a_2$\ \ \ }}
          &=& \ve v&\ \ \ \ \ \ &
        +\cdots+\phi_m\ve a_m
          \makebox[0em][r]{\raisebox{\tla}{$\phi_o\ve v+\phi_1\ve a_1+\phi_2\ve a_2$\ \ \ }}
          &=& \ve u\\
        \ds 0 \makebox[\arraycolsep]{}\neq\makebox[\arraycolsep]{} \frac{1}{\psi_o} &=& \phi_o &&
        \ds 0 \makebox[\arraycolsep]{}\neq\makebox[\arraycolsep]{} \frac{1}{\phi_o} &=& \psi_o \\
        \ds-\frac{\psi_1}{\psi_o} &=& \phi_1 &&
        \ds-\frac{\phi_1}{\phi_o} &=& \psi_1 \\
        \ds-\frac{\psi_2}{\psi_o} &=& \phi_2 &&
        \ds-\frac{\phi_2}{\phi_o} &=& \psi_2 \\
        &\ds\vdots&&&&\ds\vdots&\\
        \ds-\frac{\psi_m}{\psi_o} &=& \phi_m &&
        \ds-\frac{\phi_m}{\phi_o} &=& \psi_m
      \er
    \]
  \ec
\end{table}

Now further consider an arbitrary vector~$\ve b$ which lies in the space
addressed by the vectors
\[
  \{\ve u,\ve a_1,\ve a_2,\ldots,\ve a_m\}.
\]
Does the same~$\ve b$ also lie in the space addressed by the vectors
\[
  \{\ve v,\ve a_1,\ve a_2,\ldots,\ve a_m\}?
\]
To show that it does, we substitute into~(\ref{gjrank:338:10}) the
expression for~$\ve u$ from
% bad break
(\ref{gjrank:338:17}), obtaining the form
\[
  (\beta_o)
  (\phi_o\ve v+\phi_1\ve a_1+\phi_2\ve a_2+\cdots+\phi_m\ve a_m)
  +\beta_1\ve a_1+\beta_2\ve a_2+\cdots+\beta_m\ve a_m = \ve b.
\]
Collecting terms, this is
\[
  \beta_o\phi_o\ve v
  +(\beta_o\phi_1+\beta_1)\ve a_1
  +(\beta_o\phi_2+\beta_2)\ve a_2
  +\cdots
  +(\beta_o\phi_m+\beta_m)\ve a_m
  = \ve b,
\]
in which we see that, yes,~$\ve b$ does indeed also lie in the latter
space.  Naturally the problem's $\ve u \lra \ve v$ symmetry then
guarantees the converse, that an arbitrary vector~$\ve b$ which lies in
the latter space also lies in the former.  Therefore, a vector~$\ve b$
must lie in both spaces or neither, never just in one or the other.  The
two spaces are, in fact, one and the same.

This leads to the following useful conclusion.  Given a set of vectors
\[
  \{\ve u,\ve a_1,\ve a_2,\ldots,\ve a_m\},
\]
one can safely replace the~$\ve u$ by a new vector~$\ve v$, obtaining
the new set
\[
  \{\ve v,\ve a_1,\ve a_2,\ldots,\ve a_m\},
\]
provided that the replacement vector~$\ve v$ includes at least a little
of the replaced vector~$\ve u$ ($\psi_o \neq 0$ in
eqn.~\ref{gjrank:338:15}) and that~$\ve v$ is otherwise an honest linear
combination of the several vectors of the original set, untainted by
foreign contribution.  \emph{Such vector replacement does not in any way
alter the space addressed.}  The new space is exactly the same as the
old.

As a corollary, \emph{if the vectors of the original set happen to be
linearly independent (\S~\ref{gjrank:335}), then the vectors of the new
set are linearly independent, too;} for, if it were that
\[
  \gamma_o\ve v+\gamma_1\ve a_1+\gamma_2\ve a_2+\cdots+\gamma_m\ve a_m = 0
\]
for nontrivial~$\gamma_o$ and~$\gamma_k$, then either $\gamma_o =
0$---impossible since that would make the several~$\ve a_k$ themselves
linearly dependent---or $\gamma_o \neq 0$, in which case~$\ve v$ would
be a linear combination of the several~$\ve a_k$ alone.  But if~$\ve v$
were a linear combination of the several~$\ve a_k$ alone,
then~(\ref{gjrank:338:15}) would still also explicitly make~$\ve v$ a
linear combination of the same~$\ve a_k$ plus a nonzero multiple
of~$\ve u$.  Yet both combinations cannot be, because according to
\S~\ref{gjrank:335}, two distinct combinations among a set of independent vectors
can never target the same~$\ve v$.  The contradiction proves false the
assumption which gave rise to it: that the vectors of the new set were
linearly dependent.  Hence the vectors of the new set are equally as
independent as the vectors of the old.

% ----------------------------------------------------------------------

\section{Rank}
\label{gjrank:340}
\index{rank}

Sections~\ref{matrix:180.22} and~\ref{matrix:180.23} have introduced the
rank-$r$ identity matrix~$I_r$, where the integer~$r$ is the number of
ones the matrix has along its main diagonal.  Other matrices have rank,
too.  Commonly, an $n\times n$ matrix has rank $r=n$, but consider the
matrix
\[
  \mf{ccc}{
    5&1&6\\
    3&6&9\\
    2&4&6
  }.
\]
The third column of this matrix is the sum of the first and second
columns.  Also, the third row is just two-thirds the second.
Either way, by columns or by rows, the matrix has only two independent
vectors.  The rank of this $3\times 3$ matrix is not $r=3$ but $r=2$.

This section establishes properly the important concept of matrix
\emph{rank.}  The section demonstrates that every matrix has a definite,
unambiguous rank, and shows how this rank can be calculated.

\index{exact matrix}
\index{exact number}
\index{matrix!exact}
\index{number!exact}
\index{quantity!exact}
\index{exact quantity}
\index{quantity!imprecise}
\index{imprecise quantity}
\index{quantity!inexact}
\index{inexact quantity}
\index{exact arithmetic}
\index{arithmetic!exact}
To forestall potential confusion in the matter, we should immediately
observe that---like the rest of this chapter but unlike some other parts
of the book---this section explicitly trades in exact numbers.  If a
matrix element here is~$5$, then it is exactly~$5$; if~$0$, then
exactly~$0$.  Many real-world matrices, of course---especially matrices
populated by measured data---can never truly be exact, but that is not
the point here.  Here, the numbers are exact.%
\footnote{
  It is false to suppose that because applied mathematics \emph{permits}
  inexact or imprecise quantities,
  like~$3.0\pm 0.1$ inches for the length of your
  thumb, it also requires them.  On the contrary, the length of your
  thumb may indeed be~$3.0\pm 0.1$ inches, but surely no triangle
  has~$3.0\pm 0.1$ sides!  A triangle has exactly three sides.  The
  ratio of a circle's circumference to its radius is exactly~$2\pi$.
  The author has exactly one brother.  A construction contract might
  require the builder to finish within exactly 180 days (though the
  actual construction time might be an inexact $t=172.6\pm 0.2$ days), and so
  on.  Exact quantities are every bit as valid in applied mathematics as
  inexact or imprecise ones are.
  Where the distinction matters, it is the applied
  mathematician's responsibility to distinguish between the two kinds of
  quantity.
}

% diagn:  The next couple of subsections were a nice idea, perhaps, but
% they may be too hard to follow.  A simpler presentation is probably
% possible and might be written and substituted.  Or maybe not.
% Maybe the point demonstrated is just a subtle one.
%
% Later consideration suggests that the subsections should remain,
% after all.  We'll see.

\subsection{A logical maneuver}
\label{gjrank:340.05}
\index{logic}
\index{maneuver, logical}

\index{end, justifying the means}
\index{means, justified by the end}
In \S~\ref{gjrank:340.10} we will execute a pretty logical maneuver,
which one might name, ``the end justifies the means.''%
\footnote{
  The maneuver's name rings a bit sinister, does it not?  However, the
  book is not here setting forth an ethical proposition, but is merely
  previewing an abstract logical form the mathematics of
  \S~\ref{gjrank:340.10} will use.%
}
When embedded within a larger logical construct as in
\S~\ref{gjrank:340.10}, the maneuver if unexpected can confuse, so this
subsection is to prepare the reader to expect the maneuver.

%\index{chess}
%The maneuver's name rings a bit sinister, does it not?  Perhaps we had
%better discuss that first.  In human affairs, it would be quite wrong to
%suppose that the end justified the means, and you and I should not
%lightly be excused if we supposed that it did.  If I believed, for
%example, that your health would improve if you walked to work each
%morning, I should nonetheless be wrong to take your car and sell it for
%you.  I must mind my own business.  Such sound morals are, however, at
%least in part, a matter of competence, responsibility and the burden of
%consequences: I am neither competent to assess your health nor
%responsible to dispose of your car, and the burden of consequences rests
%chiefly with you in any case.  Playing chess, by contrast, if I believed
%that my army's tactics would improve if the army achieved a certain
%position, then nothing would forbid me from sacrificing a piece to help
%the army achieve it.  Were I general of a real army instead, of course,
%considerations would differ, but in chess the competence, responsibility
%and burden of consequences all rest with me.  Mathematics is as chess:
%the competence, responsibility and (insofar as the math itself is
%concerned) burden of consequences rest with none but him who holds the
%pencil; so, without further philosophizing, let us allow the possibility
%that in mathematics the end could properly justify the means.

The logical maneuver follows this basic pattern.
\begin{quote}
  If~$P_1$ then~$Q$.
  If~$P_2$ then~$Q$.
  If~$P_3$ then~$Q$.
  Which of~$P_1$, $P_2$ and~$P_3$ are true is not known, but suppose
  that it is nevertheless known that \emph{at least one} of the three is
  true:~$P_1$ or~$P_2$ or~$P_3$.  If this is so, then---though one can
  draw no valid conclusion regarding any one of the three
  conditions~$P_1$, $P_2$ or~$P_3$---one can still conclude that their
  common object~$Q$ is true.
\end{quote}
One valid way to prove~$Q$, then, would be to suppose~$P_1$ and show
that it led to~$Q$, then alternately to suppose~$P_2$ and show that it
separately led to~$Q$, then again to suppose~$P_3$ and show that it also
led to~$Q$.  The final step would be to show somehow that~$P_1$, $P_2$
and~$P_3$ could not possibly all be false at once.  Herein, the
\emph{means} is to assert several individually suspicious claims, none
of which one actually means to prove.  The \emph{end} which justifies
the means is the conclusion~$Q$, which thereby one can and does prove.

It is a subtle maneuver.  Once the reader feels that he grasps its
logic, he can proceed to the next subsection where the maneuver is put
to use.

\subsection{The impossibility of identity-matrix promotion}
\label{gjrank:340.10}
\index{rank!impossibility to promote of}
\index{matrix!identity, impossibility to promote of}
\index{identity matrix!impossibility to promote of}
\index{\emph{reductio ad absurdum}}
\index{proof!by contradiction}
\index{contradiction, proof by}

Consider the matrix equation
\bq{gjrank:340:10}
  AI_rB = I_s.
\eq
If $r \ge s$, then it is trivial to find matrices~$A$ and~$B$ for
which~(\ref{gjrank:340:10}) holds: $A=I_s=B$, for instance.  If instead
\[
  r < s,
\]
however, it is not so easy to find such matrices~$A$ and~$B$.  In fact
it is impossible.  This subsection proves the impossibility.  It shows that
\emph{one cannot by any row and column operations, reversible or
otherwise, ever transform an identity matrix into another identity
matrix of greater rank (\S~\ref{matrix:180.22}).}

Equation~(\ref{gjrank:340:10}) can be written in the form
\bq{gjrank:340:11}
  (AI_r)B = I_s,
\eq
where, because~$I_r$ attacking from the right is the column
truncation operator (\S~\ref{matrix:180.23}), the product~$AI_r$ is a
matrix with an unspecified number of rows but only~$r$ columns---or,
more precisely, with no more than~$r$ nonzero columns.  Viewed this
way, per \S~\ref{matrix:120.27}, $B$ operates on the~$r$ columns
of~$AI_r$ to produce the~$s$ columns of~$I_s$.

\index{vector!column}
\index{column vector}
\index{vector!elementary}
\index{elementary vector}
The~$r$ columns of~$AI_r$ are nothing more than the first through $r$th
columns of~$A$.  Let the symbols $\ve a_1,\ve a_2,\ve a_3,\ve a_4,\ve
a_5,\ldots,\ve a_r$ denote these columns.  The~$s$ columns of~$I_s$,
then, are nothing more than the elementary vectors $\ve e_1,\ve e_2,\ve
e_3,\ve e_4,\ve e_5,\ldots,\ve e_s$ (\S~\ref{matrix:180.30}).  The
claim~(\ref{gjrank:340:11}) makes is thus that the several
vectors~$\ve a_k$ together address each of the several elementary
vectors~$\ve e_j$---that is, that a linear combination%
\footnote{
  Observe that unlike as in \S~\ref{gjrank:335}, here we have not
  necessarily assumed that the several~$\ve a_k$ are linearly
  independent.
}
\bq{gjrank:340:15}
    b_{1j} \ve a_1
  + b_{2j} \ve a_2
  + b_{3j} \ve a_3
  + \cdots
  + b_{rj} \ve a_r
  = \ve e_j
\eq
exists for each~$\ve e_j$, $1 \le j \le s$.

The claim~(\ref{gjrank:340:15}) will turn out to be false because there are
too many~$\ve e_j$, but to prove this, \emph{we shall assume for the
moment that the claim were true.}  The proof then is by contradiction,%
\footnote{
  As the reader will have observed by this point in the book,
  the technique---also called \emph{reductio ad absurdum}---is the usual
  mathematical technique to prove impossibility.  One assumes the
  falsehood to be true, then reasons toward a contradiction which proves
  the assumption false.  Section~\ref{noth:220.10} among others has
  already illustrated the technique, but the technique's use here is
  more sophisticated.
}
and it runs as follows.

Consider the elementary vector~$\ve e_1$.  For $j=1$,
(\ref{gjrank:340:15}) is
\[
    b_{11} \ve a_1
  + b_{21} \ve a_2
  + b_{31} \ve a_3
  + \cdots
  + b_{r1} \ve a_r
  = \ve e_1,
\]
which says that the elementary vector~$\ve e_1$ is a linear
combination of the several vectors
\[
  \{\ve a_1,\ve a_2,\ve a_3,\ve a_4,\ve a_5,\ldots,\ve a_r\}.
\]
Because~$\ve e_1$ is a linear combination, according to
\S~\ref{gjrank:338} one can safely replace any of the vectors in the set
by~$\ve e_1$ without altering the space addressed.
For example, replacing~$\ve a_1$ by~$\ve e_1$,
\[
  \{\ve e_1,\ve a_2,\ve a_3,\ve a_4,\ve a_5,\ldots,\ve a_r\}.
\]
The only restriction per \S~\ref{gjrank:338} is that~$\ve e_1$ contain
at least a little of the vector~$\ve a_k$ it replaces---that
$b_{k1} \neq 0$.  Of course there is no guarantee specifically that
$b_{11} \neq 0$, so for~$\ve e_1$ to replace~$\ve a_1$ might not be
allowed.  However, inasmuch as~$\ve e_1$ is nonzero, then according
to~(\ref{gjrank:340:15}) at least one of the several~$b_{k1}$ also is
nonzero; and if~$b_{k1}$ is nonzero then~$\ve e_1$ can replace~$\ve
a_k$.  Some of the~$\ve a_k$ might indeed be forbidden, but never all;
there is always at least one~$\ve a_k$ which~$\ve e_1$ can replace.
(For example, if~$\ve a_1$ were forbidden because $b_{11}=0$, then~$\ve
a_3$ might be available instead because $b_{31} \neq 0$.  In this case
the new set would be $\{\ve a_1,\ve a_2,\ve e_1,\ve a_4,\ve
a_5,\ldots,\ve a_r\}$.)

Here comes the hard part.
Here is where the logical maneuver of \S~\ref{gjrank:340.05} comes in.
The book to this point has established no general method to tell which
of the several~$\ve a_k$ the elementary vector~$\ve e_1$ actually
contains (\S~\ref{mtxinv:230}
gives the method, but that section depends
logically on this one, so we cannot properly appeal to it here).
According to~(\ref{gjrank:340:15}), the vector~$\ve e_1$ might contain
some of the several~$\ve a_k$ or all of them, \emph{but surely it contains at
least one of them.}  Therefore, even though it is illicit to replace
an~$\ve a_k$ by an~$\ve e_1$ which contains none of it, even though we
have no idea which of the several~$\ve a_k$ the vector~$\ve e_1$
contains, even though replacing the wrong~$\ve a_k$ logically
invalidates any conclusion which flows from the replacement, still we
can proceed with the proof---provided only that, in the end, we shall
find that the illicit choice of replacement and the licit choice had
led alike to the same, identical conclusion.  If we do so find then---in
the end---the logic will demand of us only an assurance that some licit
choice had existed at the time the choice was or might have been made.  The logic will
never ask, even in retrospect, which specific choice had been the licit
one, for only the complete absence of licit choices can threaten the
present maneuver.

The claim~(\ref{gjrank:340:15}) guarantees at least one licit choice.
Whether as the maneuver also demands, all the choices, licit and illicit,
lead ultimately alike to the same, identical conclusion remains to be
determined.

Now consider the elementary vector~$\ve e_2$.  According
to~(\ref{gjrank:340:15}), $\ve e_2$ lies in the space addressed by the
original set
\[
  \{\ve a_1,\ve a_2,\ve a_3,\ve a_4,\ve a_5,\ldots,\ve a_r\}.
\]
Therefore as we have seen,~$\ve e_2$ also lies in the space
addressed by the new set
\[
  \{\ve e_1,\ve a_2,\ve a_3,\ve a_4,\ve a_5,\ldots,\ve a_r\}
\]
(or $\{\ve a_1,\ve a_2,\ve e_1,\ve a_4,\ve a_5,\ldots,\ve a_r\}$, or
whatever the new set happens to be).  That is, not only do
coefficients~$b_{k2}$ exist such that
\[
    b_{12} \ve a_1
  + b_{22} \ve a_2
  + b_{32} \ve a_3
  + \cdots
  + b_{r2} \ve a_r
  = \ve e_2,
\]
but also coefficients~$\beta_{k2}$ exist such that
\[
    \beta_{12} \ve e_1
  + \beta_{22} \ve a_2
  + \beta_{32} \ve a_3
  + \cdots
  + \beta_{r2} \ve a_r
  = \ve e_2.
\]
Again it is impossible for all the coefficients~$\beta_{k2}$ to be zero
but, moreover, it is impossible for~$\beta_{12}$ to be the sole nonzero
coefficient, for (as should seem plain to the reader who grasps
the concept of the elementary vector, \S~\ref{matrix:180.30}) no
elementary vector can ever be a linear combination of other elementary
vectors alone!  The linear combination which forms~$\ve e_2$ evidently
includes a nonzero multiple \emph{of at least one of the remaining~$\ve
a_k$.}  At least one of the~$\beta_{k2}$ attached to an~$\ve a_k$
(not~$\beta_{12}$, which is attached to~$\ve e_1$) must be nonzero.
Therefore by the same reasoning as before, we now choose an~$\ve a_k$
with a nonzero coefficient $\beta_{k2}\neq 0$ and replace it by~$\ve
e_2$, obtaining an even newer set of vectors like
\[
  \{\ve e_1,\ve a_2,\ve a_3,\ve e_2,\ve a_5,\ldots,\ve a_r\}.
\]
This newer set addresses precisely the same space as the previous set,
and thus also as the original set.

\index{elementary vector!left-over}
\index{vector!elementary, left-over}
And so it goes, replacing one~$\ve a_k$ by an~$\ve e_j$ at a time, until
all the~$\ve a_k$ are gone and our set has become
\[
  \{\ve e_1,\ve e_2,\ve e_3,\ve e_4,\ve e_5,\ldots,\ve e_r\},
\]
which, as we have reasoned, addresses precisely the same space as did
the original set
\[
  \{\ve a_1,\ve a_2,\ve a_3,\ve a_4,\ve a_5,\ldots,\ve a_r\}.
\]
And this is the one, identical conclusion the maneuver of \S~\ref{gjrank:340.05}
has demanded.  All intermediate choices, by various paths licit and
illicit, ultimately have led alike to the single conclusion of this
paragraph, which thereby is properly established.

\index{blockade}
Admittedly, such subtle logic may not be easy to discern.  Here is
another, slightly different light by which to illuminate the question.
Suppose again that, by making exactly~$r$ replacements, we wish to
convert the set
\[
  \{\ve a_1,\ve a_2,\ve a_3,\ve a_4,\ve a_5,\ldots,\ve a_r\}
\]
into
\[
  \{\ve e_1,\ve e_2,\ve e_3,\ve e_4,\ve e_5,\ldots,\ve e_r\},
\]
assuming again per \S~\ref{gjrank:338} that the several vectors~$\ve
a_k$ of the original set, taken together, address each of the elementary
vectors~$\ve e_j$, $1 \le j \le s$, $r < s$.  Suppose further again that
we wish per \S~\ref{gjrank:338} to convert the set without altering the
space the set addresses.  To reach our goal, first we will put the
elementary vector~$\ve e_1$ in the place of one of the several
vectors~$\ve a_k$, then we will put~$\ve e_2$ in the place of \emph{one
of the remaining}~$\ve a_k$; and so on until, at last, we put~$\ve e_r$
in the place of \emph{the last remaining}~$\ve a_k$.  We will give the
several~$\ve e_j$, $1 \le j \le r$, in their proper order but we might
take the several~$\ve a_k$ in any of~$r!$ distinct sequences: for
instance, in the case that $r=3$, we might take the several~$\ve a_k$ in
any of the $3!=6$ distinct sequences
\[
  (\ve a_1,\ve a_2,\ve a_3);\
  (\ve a_1,\ve a_3,\ve a_2);\
  (\ve a_2,\ve a_1,\ve a_3);\
  (\ve a_2,\ve a_3,\ve a_1);\
  (\ve a_3,\ve a_1,\ve a_2);\
  (\ve a_3,\ve a_2,\ve a_1);
\]
except however that we might (or might not) find certain sequences
blockaded in the event.  Blockaded?  Well, consider for example the
sequence $(\ve a_2,\ve a_3,\ve a_1)$, and suppose that $\ve e_1 = 0\ve
a_1+4\ve a_2-2\ve a_3$ and that $\ve e_2 = 5\ve a_1-(1/2)\ve e_1+0\ve
a_3$ (noticing that the latter already has~$\ve e_1$ on the right
instead of~$\ve a_2$): in this case the sequence $(\ve a_2,\ve a_3,\ve
a_1)$ is blockaded---which is to say, forbidden---because, once~$\ve
e_1$ has replaced~$\ve a_2$, since~$\ve e_2$ then contains none of~$\ve
a_3$, $\ve e_2$ cannot according to \S~\ref{gjrank:338} replace~$\ve
a_3$.  [Actually, in this example, both sequences beginning $(\ve
a_1,\ldots)$ are blockaded, too, because the turn of~$\ve e_1$ comes
first and, at that time,~$\ve e_1$ contains none of~$\ve a_1$.]  Clear?
No?  Too many subscripts?  Well, there's nothing for it: if you wish to
understand then you will simply have to trace all the subscripts out
with your pencil; the example cannot be made any simpler.  Now, although
some sequences might be blockaded, no unblockaded sequence can run to a
dead end, so to speak.  After each unblockaded replacement another
replacement will always be possible.  The reason is as before: that,
according to \S~\ref{gjrank:338}, so long as each elementary vector~$\ve
e_j$ in turn contains some of the vector it replaces, the replacement
cannot alter the space the set addresses; that the space by initial
assumption includes all the elementary vectors; that each elementary
vector in turn must therefore be found to contain at least one of the
vectors then in the set; that no elementary vector can be composed
solely of other elementary vectors; and, consequently, that each
elementary vector in turn must be found to contain at least one of the
set's then remaining~$\ve a_k$.  The logic though slightly complicated
is nonethemore escapable.  The conclusion is that we can indeed convert
$
  \{\ve a_1,\ve a_2,\ve a_3,\ve a_4,\ve a_5,\ldots,\ve a_r\}
$
into
$
  \{\ve e_1,\ve e_2,\ve e_3,\ve e_4,\ve e_5,\ldots,\ve e_r\},
$
step by step, without altering the space addressed.

The conclusion leaves us with a problem, however.  There remain more
% bad break
$\ve e_j$, $1 \le j \le s$, than there are~$\ve a_k$, $1 \le k \le r$,
because, as we have stipulated, $r < s$.  Some elementary vectors~$\ve
e_j$, $r < j \le s$, are evidently left over.  Back at the beginning of
the section, the claim~(\ref{gjrank:340:15}) made was that
\[
  \{\ve a_1,\ve a_2,\ve a_3,\ve a_4,\ve a_5,\ldots,\ve a_r\}
\]
together addressed each of the several elementary vectors~$\ve e_j$.
But as we have seen, this amounts to a claim that
\[
  \{\ve e_1,\ve e_2,\ve e_3,\ve e_4,\ve e_5,\ldots,\ve e_r\}
\]
together addressed each of the several elementary vectors~$\ve e_j$.
Plainly this is impossible with respect to the left-over~$\ve e_j$, $r <
j \le s$.  The contradiction proves false the claim which gave rise to
it.  The false claim: that the several~$\ve a_k$, $1 \le k \le r$,
addressed all the~$\ve e_j$, $1 \le j \le s$, even when $r < s$.

\index{demotion}
\index{promotion}
Equation~(\ref{gjrank:340:11}), which is just~(\ref{gjrank:340:10})
written differently, asserts that~$B$ is a column operator which does
precisely what we have just shown impossible: to combine the~$r$ columns
of~$AI_r$ to yield the~$s$ columns of~$I_s$, the latter of which are
just the elementary vectors $\ve e_1,\ve e_2,\ve e_3,\ve e_4,\ve
e_5,\ldots,\ve e_s$.  Hence finally we conclude that no matrices~$A$
and~$B$ exist which satisfy~(\ref{gjrank:340:10}) when $r<s$.  In other
words, we conclude that \emph{although row and column operations can
demote identity matrices in rank, they can never promote them.}
The promotion of identity matrices is impossible.

\subsection{General matrix rank and its uniqueness}
\label{gjrank:340.20}
\index{rank}
\index{matrix!rank of}
\index{uniqueness!of matrix rank}
\index{rank!uniqueness of}

Step~\ref{gjrank:341:s55} of the Gauss-Jordan algorithm
(\S~\ref{gjrank:341.10}) discovers a rank~$r$ for any matrix~$A$.  One
should like to think that this rank~$r$ were a definite property of the
matrix itself rather than some unreliable artifact of the algorithm, but
until now we have lacked the background theory to prove it.  Now we have
the theory.  Here is the proof.

\index{reversibility}
The proof begins with a formal definition of the quantity whose
uniqueness we are trying to prove.
\bi
  \item
    The rank~$r$ of an identity matrix~$I_r$ is the number of ones along its
    main diagonal.  (This is from \S~\ref{matrix:180.22}.)
  \item
    The rank~$r$ of a general matrix~$A$ is the rank of an identity
    matrix~$I_r$ to which~$A$ can be reduced by \emph{reversible} row
    and column operations.
\ei
Let the symbols~$B_>$ and~$B_<$ respectively represent the
aforementioned reversible row and column operations:
\bq{gjrank:340:18}
  \begin{split}
    B_>^{-1}B_> &= I = B_>B_>^{-1}; \\
    B_<^{-1}B_< &= I = B_<B_<^{-1}.
  \end{split}
\eq
A matrix~$A$ has rank~$r$ if and only if row and column operations~$B_>$
and~$B_<$ exist such that
\bq{gjrank:340:20}
  \begin{split}
    B_>AB_< &= I_r, \\
    A &= B_>^{-1} I_r B_<^{-1}.
  \end{split}
\eq
The question is whether in~(\ref{gjrank:340:20}) only a single rank~$r$
is possible.

To answer the question, we suppose that another rank were possible,
that~$A$ had not only rank~$r$ but also rank~$s$.  Then,
\[
  \begin{split}
    A &= B_>^{-1} I_r B_<^{-1}, \\
    A &= G_>^{-1} I_s G_<^{-1}.
  \end{split}
\]
Combining these equations,
\[
  B_>^{-1} I_r B_<^{-1} = G_>^{-1} I_s G_<^{-1}.
\]
Solving first for~$I_r$, then for~$I_s$,
\[
  \begin{split}
    (B_>G_>^{-1}) I_s (G_<^{-1}B_<) &= I_r, \\
    (G_>B_>^{-1}) I_r (B_<^{-1}G_<) &= I_s.
  \end{split}
\]
Were it that $r \neq s$, then one of these two equations would
constitute the demotion of an identity matrix and the other, a
promotion.  But according to \S~\ref{gjrank:340.10} and
its~(\ref{gjrank:340:10}), promotion is impossible.  Therefore $r \neq
s$ is also impossible, and
\[
  r = s
\]
is guaranteed.  No matrix has two different ranks.  \emph{Matrix rank is
unique.}

This finding has two immediate implications:
\bi
  \item
    Reversible row and/or column operations exist to change any matrix
    of rank~$r$ to \emph{any other matrix} of the same rank.  The reason is
    that, according to~(\ref{gjrank:340:20}), (\ref{gjrank:341:GJ})
    and~(\ref{gjrank:341:GJinv}), reversible operations
    exist to change both matrices to~$I_r$ and back.
  \item
    No reversible operation can change a matrix's rank.
\ei
The finding further suggests a conjecture:
\bi
  \item
    The matrices~$A$, $A^T$ and~$A^{*}$ share the same rank~$r$.
\ei
The conjecture is proved by using~(\ref{matrix:120:30})
or~(\ref{matrix:120:33}) to compute the transpose or adjoint
of~(\ref{gjrank:340:20}).

The discovery that every matrix has a single, unambiguous rank and the
establishment of a failproof algorithm---the Gauss-Jordan---to ascertain
that rank have not been easy to achieve, but they are important
achievements nonetheless, worth the effort thereto.  The reason these
achievements matter is that the mere dimensionality of a matrix is a
chimerical measure of the matrix's true size---as for instance for the
$3 \times 3$ example matrix at the head of the section.  Matrix rank by
contrast is a solid, dependable measure.  We will rely on it often.

Section~\ref{gjrank:340.30} comments further.

\subsection{The full-rank matrix}
\label{gjrank:340.25}
\index{rank!full}
\index{full rank}
\index{matrix!full-rank}

\index{degenerate matrix}
\index{matrix!degenerate}
According to~(\ref{gjrank:341:22}), the rank~$r$ of a dimension-limited
matrix (\S~\ref{matrix:180.25})---let us refer to it as a \emph{matrix}
(just to reduce excess verbiage)---can exceed
the number neither of the matrix's rows nor of its columns.  The
greatest rank possible for an $m\times n$ matrix is the lesser of~$m$
and~$n$.  A \emph{full-rank} matrix, then, is defined to be an $m\times
n$ matrix with rank $r=m$ or $r=n$---or, if $m=n$, both.  A
matrix of less than full rank is a \emph{degenerate} matrix.

\index{linear combination}
\index{linear dependence}
Consider a tall $m\times n$ matrix~$C$, $m\ge n$, one of whose~$n$
columns is a linear combination (\S~\ref{gjrank:335}) of the others.
One could by definition target the dependent column with addition
elementaries, using multiples of the other columns to wipe the dependent
column out.  Having zeroed the dependent column, one could then
interchange it over to the matrix's extreme right, effectively throwing
the column away, shrinking the matrix to $m\times(n-1)$ dimensionality.
Shrinking the matrix necessarily also shrinks the bound on the matrix's
rank to $r\le n-1$---which is to say, to $r<n$.  But the shrinkage, done by
reversible column operations, is itself reversible, by which
\S~\ref{gjrank:340.20} binds the rank of the original, $m\times n$
matrix~$C$ likewise to $r<n$.  The matrix~$C$, one of whose columns is a
linear combination of the others, is necessarily degenerate for this
reason.

\index{linear independence}
\index{independence}
Now consider a tall matrix~$A$ with the same $m\times n$ dimensionality,
but with a full~$n$ independent columns.  The transpose~$A^{T}$ of such
a matrix has a full~$n$ independent \emph{rows.}  One of the
conclusions of \S~\ref{gjrank:341.11} was that a matrix of
independent rows always has rank equal to the number of rows.
Since~$A^{T}$ is such a matrix, its rank is a full $r=n$.  But
formally, what this says is that there exist operators~$B_<^{T}$
and~$B_>^{T}$ such that $I_n = B_<^{T}A^{T}B_>^{T}$, the transpose of
which equation is $B_>AB_< = I_n$---which in turn says that not
only~$A^T$, but also~$A$ itself, has full rank $r=n$.

Parallel reasoning rules the rows and columns of broad matrices, $m\le n$, of
course.  To square matrices, $m=n$, both lines of reasoning apply.

\index{matrix!tall}
\index{matrix!broad}
\index{matrix!square}
Gathering findings, we have that
\bi
  \item
    a tall $m\times n$ matrix, $m\ge n$, has full rank if and only if
    its columns are linearly independent;
  \item
    a broad $m\times n$ matrix, $m\le n$, has full rank if and only if
    its rows are linearly independent;
  \item
    a square $n\times n$ matrix, $m = n$, has full rank if and only if
    its columns and/or its rows are linearly independent; and
  \item
    a square matrix has both independent columns and independent rows,
    or neither; never just one or the other.
\ei

\index{full column rank}
\index{column rank!full}
\index{rank!column}
\index{full row rank}
\index{row rank!full}
\index{rank!row}
To say that a matrix has \emph{full column rank} is to say that it is
tall or square and has full rank $r=n \le m$.  To say that a matrix has
\emph{full row rank} is to say that it is broad or square and has full
rank $r=m \le n$.  Only a square matrix can have full column rank and
full row rank at the same time, because a tall or broad matrix cannot
but include, respectively, more columns or more rows than~$I_r$.

\index{rank!infinite}
\index{infinite rank}
Observe incidentally that extended operators, which per
\S~\ref{matrix:180.35} define their $m\times n$ active regions
differently, have infinite rank.

\subsection[Under- and overdetermined systems (introduction)]
  {Underdetermined and overdetermined linear systems (introduction)}
\label{gjrank:340.24}

\index{linear system!classification of}
\index{linear system!taxonomy of}
\index{taxonomy!of the linear system}
\index{linear system!underdetermined}
\index{linear system!overdetermined}
\index{linear system!degenerate}
\index{underdetermined linear system}
\index{overdetermined linear system}
\index{degenerate linear system}
The last paragraph of \S~\ref{gjrank:340.25}
% Thesaurus for provoke: awaken, call forth, call up, cultivate,
% enkindle, fire up, foment, inspire, instigate, motivate, move,
% promote, prompt, provoke, stir up.
provokes yet further terminology.
A linear system $A\ve x = \ve b$ is \emph{underdetermined} if~$A$ lacks
full column rank---that is, if $r<n$---because inasmuch as some of
$A$'s columns then depend linearly on the others such a system maps
multiple $n$-element vectors~$\ve x$ to the same $m$-element
vector~$\ve b$, meaning that knowledge of~$\ve b$
does not suffice to determine~$\ve x$ uniquely.  Complementarily, a
linear system $A\ve x = \ve b$ is \emph{overdetermined} if~$A$ lacks
full row rank---that is, if $r<m$.  If~$A$ lacks both, then the system
is paradoxically both underdetermined and overdetermined and is thereby
\emph{degenerate.} If~$A$ happily has both, then the system is
\emph{exactly determined.}

Section~\ref{mtxinv:230} solves the exactly determined linear system.
Section~\ref{mtxinv:240} solves the nonoverdetermined linear system.
Section~\ref{mtxinv:320} analyzes the unsolvable overdetermined linear
system among others.  Further generalities await chapter~\ref{mtxinv}; but, regarding
the overdetermined system specifically, the present subsection would
observe at least the few following facts.

\emph{An overdetermined linear system $A\ve x = \ve b$ cannot have a
solution for every possible $m$-element driving vector~$\ve b$.}  The
truth of this claim can be seen by decomposing the system's matrix~$A$
by Gauss-Jordan and then left-multiplying the decomposed system
by~$G_>^{-1}$ to reach the form
\[
  I_rG_<\ve x = G_>^{-1}\ve b.
\]
If the $m$-element vector $\ve c \equiv G_>^{-1}\ve b$, then $I_rG_<\ve
x = \ve c,$ which is impossible unless the last $m-r$ elements of~$\ve
c$ happen to be zero.  But since~$G_>$ is invertible, each~$\ve b$
corresponds to a unique~$\ve c$ and vice versa; so, if~$\ve b$ is an
unrestricted $m$-element vector then so also is~$\ve c$, which verifies
the claim.

Complementarily, \emph{a nonoverdetermined linear system $A\ve
x = \ve b$ does have a solution for every possible $m$-element driving
vector~$\ve b$.}  This is so because in this case the last $m-r$ elements
of~$\ve c$ do happen to be zero; or, better stated, because~$\ve c$ in
this case has no nonzeros among its last $m-r$ elements, because it
\emph{has} no last $m-r$ elements, for the trivial reason that $r=m$.

It is an analytical error, and an easy one innocently to commit, to
require that
\[
  A\ve x = \ve b
\]
for unrestricted~$\ve b$ when~$A$ lacks full row rank.  The error is
easy to commit because the equation looks right, because such an
equation is indeed valid over a broad domain of~$\ve b$ and might very
well have been written correctly in that context, only not in the
context of unrestricted~$\ve b$.  Analysis including such an error can
lead to subtly absurd conclusions.  It is never such an analytical error
however to require that
\[
  A\ve x = 0
\]
because, whatever other solutions such a system might have, it has at
least the solution $\ve x = 0$.

\subsection{The full-rank factorization}
\label{gjrank:340.26}
\index{full-rank factorization}
\index{factorization!full-rank}

One sometimes finds dimension-limited matrices of less than full rank
inconvenient to handle.  However, every dimension-limited, $m \times n$
matrix of rank~$r$ can be expressed as the product of two full-rank
matrices, one $m \times r$ and the other $r \times n$, both also of
rank~$r$:
\bq{gjrank:340:26}
  A = BC.
\eq
The truncated Gauss-Jordan~(\ref{gjrank:341:GJtC}) constitutes one such
\emph{full-rank factorization:} $B=I_mG_>I_r$, $C=I_rG_<I_n$, good for
any matrix.  Other full-rank factorizations are possible, however,
including among others the truncated Gram-Schmidt~(\ref{mtxinv:460:70}).
The full-rank factorization is not unique.%
\footnote{
  \cite[\S~3.3]{Beattie}%
  \cite[``Moore-Penrose generalized inverse'']{planetm}
}

Of course, if an $m \times n$ matrix already has full rank $r=m$ or
$r=n$, then the full-rank factorization is trivial: $A=I_mA$ or
$A=AI_n$.

Section~\ref{mtxinv:320.50} uses the full-rank factorization.

\subsection
  [Full column rank and the Gauss-Jordan's~$K$ and~$S$]
  {Full column rank and the Gauss-Jordan factors~$K$ and~$S$}
\label{gjrank:340.60}
\index{Gauss-Jordan decomposition!and full column rank}
\index{Gauss-Jordan decomposition!factors~$K$ and~$S$ of}
\index{full column rank}
\index{column rank!full}
\index{rank!column}

\index{tall matrix}
\index{square matrix}
\index{matrix!tall}
\index{matrix!square}
The Gauss-Jordan decomposition~(\ref{gjrank:341:GJ}),
\[
  A = P D L U I_r K S,
\]
of a tall or square $m \times n$ matrix~$A$ of full column rank $r=n \le
m$ always finds the factor $K=I$, regardless of the pivots one chooses
during the Gauss-Jordan algorithm's step~\ref{gjrank:341:s20}.  If one
happens always to choose $q=i$ as pivot column then not only $K=I$ but
$S=I$, too.

\index{extra column}
\index{spare column}
\index{column!spare}
That $K=I$ is seen by the algorithm's step~\ref{gjrank:341:s75}, which
creates~$K$.  Step~\ref{gjrank:341:s75} nulls the spare columns $q>r$
that dress~$\tilde I$'s right, but in this case~$\tilde I$ has only~$r$
columns and therefore has no spare columns to null.  Hence
step~\ref{gjrank:341:s75} does nothing and $K=I$.

\index{interchange!refusing an}
That $S=I$ comes immediately of choosing $q=i$ for pivot column during
each iterative instance of the algorithm's step~\ref{gjrank:341:s20}.
But, one must ask, can one choose so?  What if column $q=i$ were
unusable?  That is, what if the only nonzero elements remaining
in~$\tilde I$'s $i$th column stood above the main diagonal, unavailable
for step~\ref{gjrank:341:s25} to bring to pivot?  Well, \emph{were} it
so, then one would indeed have to choose $q \neq i$ to swap the unusable
column away rightward, but see: nothing in the algorithm later fills
such a column's zeros with anything else---they remain zeros---so
swapping the column away rightward could only delay the crisis.  The
column would remain unusable.  Eventually the column would reappear on
pivot when no usable column rightward remained available to swap it
with, which contrary to our assumption would mean precisely that $r<n$.
Such contradiction can only imply that if $r=n$ then no unusable column
can ever appear.  One need not swap.  We conclude that though one might
voluntarily choose $q \neq i$ during the algorithm's
step~\ref{gjrank:341:s20}, the algorithm cannot force one to do so if
$r=n$.  Yet if one always does choose $q=i$, as the full-column-rank
matrix~$A$ evidently leaves one free to do, then indeed $S=I$.

\index{exact arithmetic}
\index{arithmetic!exact}
Theoretically, the Gauss-Jordan decomposition~(\ref{gjrank:341:GJ})
includes the factors~$K$ and~$S$ precisely to handle matrices with more
columns than rank.  Matrices of full column rank $r=n$, common in
applications, by definition have no such problem.  Therefore, the
Gauss-Jordan decomposition theoretically needs no~$K$ or~$S$ for such
matrices, which fact lets us abbreviate the decomposition for such
matrices to read
\bq{gjrank:340:60}
  A = P D L U I_n.
\eq
Observe however that just because one theoretically can set $S=I$ does
not mean that one actually should.  The column permutor~$S$ exists to be
used, after all---especially numerically to avoid small pivots during
early invocations of the algorithm's step~\ref{gjrank:341:s40}.
Equation~(\ref{gjrank:340:60}) is not mandatory but optional for a
matrix~$A$ of full column rank (though still $r=n$ and thus $K=I$ for
such a matrix, even when the unabbreviated eqn.~\ref{gjrank:341:GJ} is
used).  There are however times when it is nice to know that one
theoretically could, if doing exact arithmetic, set $S=I$ if one wanted
to.

\index{vector space!address of}
\index{space!address of}
Since $PDLU$ acts as a row operator,~(\ref{gjrank:340:60}) implies that
each row of the square, $n\times n$ matrix~$A$ whose rank $r=n$ is full
lies in the space the rows of~$I_n$ address.
% A later review rediscovers the phrase on the next line,
% which seems like pretty good prose.
% I should try to write like that more.
This is obvious and boring,
but interesting is the converse implication
of~(\ref{gjrank:340:60})'s complementary form,
\[
  U^{-1} L^{-1} D^{-1} P^{-1} A = I_n,
\]
that each row of~$I_n$ lies in the space the rows of~$A$ address.  The
rows of~$I_n$ and the rows of~$A$ evidently address the same space.
One can moreover say the same of~$A$'s columns since, according to
\S~\ref{gjrank:340.20}, $A^T$ has full
rank just as~$A$ does.  In the whole, \emph{if a matrix~$A$ is square
and has full rank $r=n$, then $A$'s columns together, $A$'s rows
together, $I_n$'s columns together and $I_n$'s rows together each
address the same, complete $n$-dimensional space.}

\subsection{The significance of rank uniqueness}
\label{gjrank:340.30}

\index{dimensionality}
\index{matrix!dimensionality of}
The result of \S~\ref{gjrank:340.20}, that matrix rank is unique, is an
extremely important matrix theorem.  It constitutes the chapter's chief
result, which we have spent so many pages to attain.  Without this
theorem, the very concept of matrix rank must remain in doubt, along
with all that attends to the concept.  The theorem is the rock upon
which the general theory of the matrix is built.

The concept underlying the theorem promotes the useful sensibility that
a matrix's rank, much more than its mere dimensionality or the extent of
its active region, represents the matrix's true size.  Dimensionality
can deceive, after all.  For example, the honest $2\times 2$ matrix
\[
  \mf{cc}{
    5&1\\
    3&6
  }
\]
has two independent rows or, alternately, two independent columns, and,
hence, rank $r=2$.  One can easily construct a phony $3\times 3$ matrix
from the honest $2\times 2$, however, simply by applying some $3\times
3$ row and column elementaries:
\[
  T_{(2/3)[32]}
  \mf{cc}{
    5&1\\
    3&6
  }
  T_{1[13]}
  T_{1[23]}
  = \mf{ccc}{
      5&1&6\\
      3&6&9\\
      2&4&6
  }.
\]
The $3 \times 3$ matrix on the equation's right is the one we met at the
head of the section.  It \emph{looks} like a rank-three matrix, but
really has only two independent columns and two independent rows.  Its
true rank is $r=2$.  We have here caught a matrix impostor pretending to
be bigger than it really is.%
\footnote{
  An applied mathematician with some matrix experience
  actually probably recognizes this particular $3\times 3$ matrix as a
  fraud on sight, but it is a very simple example.  No one can just look
  at some arbitrary matrix and instantly perceive its true rank.
  Consider for instance the $5\times 5$ matrix (in hexadecimal notation)
  \[
    \mf{rrrrr}{
       12 & 9 & 3 & 1 & 0 \\
       \frac 3 2 & \frac{\mathrm F}2 & \frac{15}2 & 2 & 12 \\
       \mbox D & 9 & -19 & -\frac{\mathrm E}3 & -6 \\
       -2 & 0 & 6 & 1 & 5 \\
       1 & -4 & 4 & 1 & -8
    }.
  \]
  As the reader can verify by the Gauss-Jordan algorithm, the matrix's
  rank is not $r=5$ but $r=4$.
  %
  %The particular instance given is interesting, yet typical, in that a
  %small change in \emph{any one} of the matrix's many elements would
  %restore full rank $r=5$.
}

Now, admittedly, adjectives like ``honest'' and ``phony,'' terms like
``imposter,'' are a bit hyperbolic.  The last paragraph has used them to
convey the subjective sense of the matter, but of course there is
nothing mathematically improper or illegal about a matrix of less than
full rank so long as the true rank is correctly recognized.  When one
models a physical phenomenon by a set of equations, one sometimes is
dismayed to discover that one of the equations, thought to be
independent, is really just a useless combination of the others.  This
can happen in matrix work, too.  The rank of a matrix helps one to
recognize how many truly independent vectors, dimensions or equations
one actually has available to work with, rather than how many seem
available at first glance.
% In an earlier draft, the next line read, "of a matrix's rank," which
% may have been more grammatical; but the prose reads better as now
% written.
Such is the sense of matrix rank.

%The reader was promised a tedious chapter, and it seems fair to say that
%the promise has been kept.  The matrix is an odd animal.  At first
%meeting it seems so simple, so dull, compared to exciting, dangerous,
%carnivorous mathematical beasts like the integral, that one somehow
%cannot imagine much theory to it.  Tracking the matrix nevertheless, one
%soon finds its lair, not the orderly orchard of rows and columns one
%would expect, but an unexpectedly thick theoretical bramble, whose
%untangling demands such a tedious chapter as this.  The uniqueness of
%matrix rank is the fruit of the bramble, worth the harvest, as we shall
%taste in chapters~\ref{mtxinv} and~\ref{eigen}.  Those chapters begin
%to harness the matrix to more interesting work.

