% ----------------------------------------------------------------------

\chapter{The derivative and its incidents}
\label{drvtv}
\index{calculus}
\index{calculus!the two complementary questions of}
\index{derivative}
\index{rate}
\index{change, rate of}

\index{Newton, Sir Isaac (1642--1727)}
\index{Leibnitz, Gottfried Wilhelm (1646--1716)}
The mathematics of \emph{calculus} concerns a complementary pair of
questions:%
\footnote{
  Although once grasped the concept is relatively simple, to understand
  this pair of questions---so briefly stated---is no trivial thing.
  They are the pair which eluded or confounded the most brilliant
  mathematical minds of the ancient world.

  The greatest conceptual hurdle---the stroke of brilliance---probably
  lies in simply stating the pair of questions clearly.  Sir Isaac
  Newton and G.~W.\ Leibnitz cleared this hurdle for us in the
  seventeenth century, so now at least we know the right pair of
  questions to ask.  With the pair in hand, the calculus beginner's
  first task is quantitatively to understand the pair's
  interrelationship, generality and significance.  Such an understanding
  constitutes the basic calculus concept.

  It cannot be the role of a book like this to lead the beginner gently
  toward an apprehension of the basic calculus concept.  Once grasped,
  the concept is simple and briefly stated.  Therefore, in this book, we
  shall take the concept as simple, briefly state it, and then move along;
  whereas, of course, you would find it hard first to learn calculus
  like that.

  Many instructional textbooks have been written to lead the beginner
  gently.  Worthy examples of such a textbook include~\cite{Hamming}.
}
\bi
  \item Given some function $f(t)$, what is the function's instantaneous
    rate of change, or \emph{derivative,} $f'(t)$?
  \item Interpreting some function $f'(t)$ as an instantaneous rate of
    change, what is the corresponding accretion, or \emph{integral,}
    $f(t)$?
\ei
This chapter builds toward a basic understanding of the first question.

% ----------------------------------------------------------------------

\section{Infinitesimals and limits}
\label{drvtv:210}

Calculus systematically treats numbers so large and so small, they lie
beyond the reach of our mundane number system.

\subsection{The infinitesimal}
\label{drvtv:210.001}
\index{infinitesimal}
\index{infinity}
\index{number!very large or very small}

A number~$\ep$ is an \emph{infinitesimal} if it is so small that
\[
  0 < \left|\ep\right| < a
\]
for all possible, mundane positive numbers~$a$.

This is somewhat a difficult concept, so if it is not immediately clear
then let us approach the matter colloquially.  Let me propose to you
that I have an infinitesimal.

``How big is your infinitesimal?'' you ask.

``Very, very small,'' I reply.

``How small?''

``Very small.''

``Smaller than 0x0.01?''

``Smaller than what?''

``Than~$2^{-8}$.  You said that we should use
hexadecimal notation in this book, remember?''

``Sorry.  Yes, right, smaller than 0x0.01.''

``What about 0x0.0001?  Is it smaller than that?''

``Much smaller.''

``Smaller than $\mr{0x0.0000\,0000\,0000\,0001}$?''

``Smaller.''

``Smaller than $2^{-\mr{0x1\,0000\,0000\,0000\,0000}}$?''

``Now \emph{that} is an impressively small number.  Nevertheless, my
infinitesimal is smaller still.''

``Zero, then.''

``Oh, no.  Bigger than that.  My infinitesimal is definitely bigger than
zero.''

This is the idea of the infinitesimal.  It is a definite number of a
certain nonzero magnitude, but its smallness conceptually lies beyond
the reach of our mundane number system.

If~$\ep$ is an infinitesimal, then $1/\ep$ can be regarded as an
\emph{infinity:} a very large number much larger than any mundane
number one can name.\footnote{\label{drvtv:210.001-fn}%
  %\index{Berkeley, George (1685--1753), Bishop of Cloyne}
  %\index{Robinson, Abraham (1918--1974)}
  Some professional mathematicians have deprecated talk of this sort.
  However, their reasons are abstruse and appear to bear little on
  applications.  See \S~\ref{intro:284}.

  The literature in this and related matters is vast, deep and
  inconclusive.  \hspace{\stretch{1}} It includes
  % superdiagn: bad break, or bad spacing. The several \stretch below
  % and just above should be unnecessary
  \cite{Berkeley}\hspace{\stretch{1}}%
  \cite{Courant/Hilbert}\hspace{\stretch{1}}%
  \cite{Dasgupta}\hspace{\stretch{1}}%
  \cite{Fisher}\hspace{\stretch{1}}%
  \cite{Frege}\hspace{\stretch{1}}%
  \cite{Hardy}\hspace{\stretch{1}}%
  \cite{Hersh}\hspace{\stretch{1}}%
  \cite{Hersh-book}\hspace{\stretch{1}}%
  \cite{Hilbert:geometrie}\hspace{\stretch{1}}%
  \cite{Pinter}\hspace{\stretch{1}}%
  \cite{Robinson}\hspace{\stretch{1}}%
  \cite{Russell:1903}\hspace{\stretch{1}}%
  \cite{Shilov}\hspace{\stretch{1}}%
  \cite{Sieg/Schlimm}\hspace{\stretch{1}}%
  \cite{Toader}\hspace{\stretch{1}}%
  \cite{Tymoczko}\hspace{\stretch{1}}%
  \cite{Weierstrass:werke}\hspace{\stretch{1}}%
  \cite{Weiner}\hspace{\stretch{1}}%
  \cite{Weyl:1918}\hspace{\stretch{1}}%
  \cite{Weyl:1950}\hspace{\stretch{1}}%
  \cite{Wilson}
  among others.%
}

\index{$\delta$}
\index{$\epsilon$}
\index{dividing by zero}
\index{zero!dividing by}
\index{$0$ (zero)!dividing by}
\index{infinitesimal!practical size of}
The principal advantage of using symbols like~$\ep$ rather than~$0$ for
infinitesimals is in that it permits us conveniently to compare one
infinitesimal against another, to add them together, to divide them,
etc.  For instance, if $\delta = 3\ep$ is another infinitesimal, then
the quotient $\delta/\ep$ is not some unfathomable~$0/0$; rather it is
$\delta/\ep = 3$.  In physical applications, the infinitesimals often
are not true mathematical infinitesimals but rather relatively very
small quantities such as the mass of a wood screw compared to the mass
of a wooden house frame, or the audio power of your voice compared to
that of a jet engine.  The additional cost of inviting one more guest to
the wedding may or may not be infinitesimal, depending upon your point of
view.  The key point is that the infinitesimal quantity be negligible by
comparison, whatever ``negligible'' might mean in the context.%
\footnote{
  Among scientists and engineers who study wave phenomena, there is an
  old rule of thumb that sinusoidal waveforms be discretized not less
  finely than ten points per wavelength.  In keeping with this book's
  adecimal theme (appendix~\ref{hex}) and the concept of the hour of arc
  (\S~\ref{trig:260}), we should probably render the rule as
  \emph{twelve} points per wavelength here.  In any case, even very
  roughly speaking, a quantity greater than $1/\mbox{0xC}$ of the
  principal to which it compares probably cannot rightly be regarded as
  infinitesimal.  On the other hand, a quantity less than
  $1/\mbox{0x10000}$ of the principal is indeed infinitesimal for most
  practical purposes (but not all: for example, positions of spacecraft
  and concentrations of chemical impurities must sometimes be accounted
  more precisely).  For quantities between $1/\mbox{0xC}$ and
  $1/\mbox{0x10000}$, it depends on the accuracy one seeks.
}

\index{infinitesimal!second- and higher-order}
The second-order infinitesimal~$\ep^2=(\ep)(\ep)$ is so small on the
scale of the common, first-order infinitesimal~$\ep$ that the even
latter cannot measure it.  The~$\ep^2$ is an infinitesimal to the
infinitesimals.  Third- and higher-order infinitesimals likewise are
possible.

\index{$\ll$ and~$\gg$}
The notation $u \ll v$, or $v \gg u$, indicates that~$u$ is much less than~$v$,
typically such that one can regard the quantity $u/v$ to be an
infinitesimal.  In fact, one common way to specify that~$\ep$ be
infinitesimal is to write that $\ep \ll 1$.

\subsection{Limits}
\label{drvtv:210.20}
\index{limit}

The notation
  $\lim_{z\rightarrow z_o}$
indicates that~$z$ draws as near to~$z_o$ as it possibly can.
When written as
  $\lim_{z\rightarrow z_o^+}$,
the implication is that~$z$ draws toward~$z_o$ from the positive side
such that $z > z_o$.  Similarly, when written as
  $\lim_{z\rightarrow z_o^-}$,
the implication is that~$z$ draws toward~$z_o$ from the negative side.

\index{|}
The reason for the notation is to provide an orderly way to handle
expressions like
\[
  \frac{f(z)}{g(z)}
\]
as~$z$ vanishes or approaches some otherwise difficult value.  For
example, if $f(z)\equiv 3z+5z^3$ and $g(z)\equiv 2z+z^2$, then
\[
    \lim_{z\rightarrow 0}\frac{f(z)}{g(z)}
    = \lim_{z\rightarrow 0}\frac{3z+5z^3}{2z+z^2}
    = \lim_{z\rightarrow 0}\frac{3+5z^2}{2+z}
    = \frac{3+0}{2+0}
    = \frac{3}{2},
\]
which is preferable to writing na\"ively that $f(z)/g(z)|_{z=0} =
0/0$ (the ``$|_{z=0}$'' meaning, ``given that, or evaluated when,
$z=0$'').  The symbol ``$\lim_Q$'' is short for ``in the limit as~$Q$,''
so ``$\lim_{z\rightarrow 0}$'' says, ``in the limit as~$z$
approaches~0.''

Observe that $\lim$ is not a function like $\log$ or $\sin$.  Rather, it
is a mere reminder.  It is a reminder that a quantity like~$z$ approaches
some value, used when saying that the quantity \emph{equaled}
the value would be inconvenient or confusing.

% ----------------------------------------------------------------------

\section{Combinatorics}
\label{drvtv:220}
\index{combinatorics}

In its general form, the problem of selecting~$k$ specific items out of
a set of~$n$ available items belongs to probability theory (chapter~\ref{prob}).
In its basic form however, the same problem also applies to the
handling of polynomials or power series.  This section treats the
problem in its basic form.%
\footnote{\cite{Hamming}}

\subsection{Combinations and permutations}
\label{drvtv:220.20}
\index{combination}
\index{permutation}
\index{choice of wooden blocks}
\index{selection from among wooden blocks}
\index{block, wooden}
\index{wooden block}

Consider the following scenario.  I have several small,
wooden blocks of various shapes and sizes, painted different colors so
that you can readily tell each block from the others.  If I offer you
the blocks and you are free to take all, some or none of them at your
option, if you can take whichever blocks you like, then how many
distinct choices of blocks confront you?  Answer:  the symbol~$n$
representing the number of blocks I have, a total of~$2^n$ distinct
choices confront you, for you can accept or reject the first block, then
accept or reject the second, then the third, and so on.

Now, suppose that what you want are exactly~$k$ blocks, neither more nor
fewer.  Desiring exactly~$k$ blocks, you select your favorite block
first: there are~$n$ options for this.  Then you select your second
favorite: for this, there are $n-1$ options (why not~$n$ options?
because you have already taken one block from me; I have only $n-1$
blocks left).  Then you select your third favorite---for this there are
$n-2$ options---and so on until you have~$k$ blocks.  There are
evidently
\bq{drvtv:220:20}
  P\cmb{n}{k} \equiv n!/(n-k)!
\eq
ordered ways, or \emph{permutations,} available for you to select
exactly~$k$ blocks.

However, some of these distinct permutations would put exactly
the same \emph{combination} of blocks in your hand; for instance,
the permutations red-green-blue and green-red-blue constitute the same
combination, whereas red-white-blue is a different combination
entirely.  For a single combination of~$k$ blocks (red, green, blue),
evidently~$k!$ permutations are possible (%
  red-green-blue,
  red-blue-green,
  green-red-blue,
  green-blue-red,
  blue-red-green,
  blue-green-red%
).  Thus, dividing the number of permutations~(\ref{drvtv:220:20})
by~$k!$ yields the number of combinations
\bq{drvtv:220:30}
  \cmb{n}{k} \equiv \frac{n!/(n-k)!}{k!}.
\eq

\index{combination!properties of}
\index{combinatorics!properties of}
\index{Pascal's triangle!neighbors in}
Table~\ref{drvtv:220:tbl} repeats the definitions~(\ref{drvtv:220:20})
and~(\ref{drvtv:220:30}), and then proceeds to list several properties of
the number {$\cmbl{n}{k}$} of combinations.
\begin{table}
  \caption{Combinatorical properties.}
  \index{combination!properties of}
  \index{combinatorics!properties of}
  \label{drvtv:220:tbl}
  \bc
    \bqb
      P\cmb{n}{k} &\equiv& n!/(n-k)!\\
      \cmb{n}{k} &\equiv& \frac{n!/(n-k)!}{k!}
              =  \frac{1}{k!}P\cmb{n}{k}\\
             &=& \cmb{n}{n-k}\\
             &=& \cmb{n-1}{k-1} + \cmb{n-1}{k}\\
             &=& \frac{n-k+1}{k}\cmb{n}{k-1}\\
             &=& \frac{k+1}{n-k}\cmb{n}{k+1}\\
             &=& \frac{n}{k}\cmb{n-1}{k-1}\\
             &=& \frac{n}{n-k}\cmb{n-1}{k}\\
      \sum_{k=0}^{n} \cmb{n}{k} &=& 2^n
    \eqb
  \ec
\end{table}
Among the several properties, the property of the table's third line
results from changing the variable $k\la n-k$ in~(\ref{drvtv:220:30}).
The property of the table's fourth line is seen when an $n$th
block---let us say that it is a black block---is added to an existing
set of $n-1$ blocks:  to choose~$k$ blocks then, you can choose
either~$k$ from the original set, or the black block plus $k-1$ from the
original set.  The next four lines come directly from the
definition~(\ref{drvtv:220:30}); they relate combinatoric coefficients
to their neighbors in Pascal's triangle (\S~\ref{drvtv:220.30}).  The
last line merely observes, again as at the head of this section,
that~$2^n$ total combinations are possible if any~$k$ is allowed.

Because one can choose neither fewer than zero nor more than~$n$ from~$n$
blocks,
\bq{drvtv:220:48}
  \cmb n k = 0 \ \ \mbox{unless}\ 0 \le k \le n.
\eq
For $\cmbl n k$ when $n<0$, there is no obvious definition.\footnote{%
  \index{Silverman, Richard~A.}%
  \index{faux rigor}%
  \index{rigor!faux}%
  So, does that mean that $\cmbl n k$ is not allowed when $n<0$?  Answer:
  probably.  After all, it seems hard to imagine how one could allow such a
  quantity while retaining internal consistency within
  Table~\ref{drvtv:220:tbl}, for a division by zero seems to be implied.
  However, the question may not be the sort of question the
  applied mathematician is even likely to ask.  He is likely to ask, rather,
  what $\cmbl n k$, $n<0$, would mean---if anything---\emph{in light of a
  particular physical problem of interest.} Only once the latter question has
  been answered will the applied mathematician consider whether or how to treat
  the quantity.
  %
  % The below may be interesting but is not included in the book for now.
  %
  %The prospect of one single, absolute, formal logic to comprise all
  %mathematical circumstances is deeply alluring to many mathematicians (though
  %admittedly less deeply to this writer).  However, a book of applied
  %mathematics would not be the place to look for such a logic.  Indeed, not
  %even the professionals have discovered such a logic (see
  %footnote~\ref{drvtv:210.001-fn}, \S~\ref{intro:284.4}
  %and~\cite[Preface]{Shilov}); and yet, even if they had, the applied
  %mathematician would hardly think along such lines, anyway.
  %
  %The book includes several footnotes like the one you are
  %reading.  It includes them in part because, in the Western world in
  %the writer's day, an irksome, indefinable social expectation has grown
  %among us applied mathematicians that the applied mathematician should
  %sporadically engage in what one might call ``faux rigor.''  The
  %results of such engagement are
  %almost always unsatisfactory.  Such engagement clarifies nothing and
  %incidentally, moreover, for what it's worth, as one suspects, leaves
  %professional mathematicians unimpressed.  Such engagement is species
  %of cant.  We should stop doing it.  Many of us have stopped doing it.
  %It does not help.
  %
  %During the Soviet era of the 20th century, Soviet mathematicians
  %seemed to suffer less from the social problem in
  %question, which may be the chief reason English-speaking engineers so valued
  %Richard~A.\ Silverman's excellent series of translations (\cite{Lebedev}
  %and~\cite{Shilov}, for instance) of mathematical books from the Russian.%
}

\subsection{Pascal's triangle}
\label{drvtv:220.30}
\index{Pascal's triangle}
\index{Pascal, Blaise (1623--1662)}

Consider the triangular layout in Fig.~\ref{drvtv:pasc0} of the various
possible $\cmbl{n}{k}$.
\begin{figure}
  \caption{The plan for Pascal's triangle.}
  \label{drvtv:pasc0}
  \bc
    \[
      \br{c}
        \cmbl{0}{0} \\
        \cmbl{1}{0}\cmbl{1}{1} \\
        \cmbl{2}{0}\cmbl{2}{1}\cmbl{2}{2} \\
        \cmbl{3}{0}\cmbl{3}{1}\cmbl{3}{2}\cmbl{3}{3} \\
        \cmbl{4}{0}\cmbl{4}{1}\cmbl{4}{2}\cmbl{4}{3}\cmbl{4}{4} \\
        \cmbl{5}{0}\cmbl{5}{1}\cmbl{5}{2}\cmbl{5}{3}\cmbl{5}{4}\cmbl{5}{5} \\
        \vdots
      \er
    \]
  \ec
\end{figure}
% bad break
Eval\-u\-ated, this yields Fig.~\ref{drvtv:pasc}, \emph{Pascal's triangle.}
Notice that each entry in the triangle is the sum of the two entries
immediately above, as Table~\ref{drvtv:220:tbl} predicts.  (In fact, this is
the easy way to fill out Pascal's triangle: for each entry, just add the
two entries above.)
\begin{figure}
  \nc\di[1]{\makebox[\wdig][c]{#1}}
  \caption{Pascal's triangle (in hexadecimal notation).}
  \label{drvtv:pasc}
  \bc
    \[
      \br{c}
        1 \\
        1\ \ \ 1 \\
        1\ \ \ 2\ \ \ 1 \\
        1\ \ \ 3\ \ \ 3\ \ \ 1 \\
        1\ \ \ 4\ \ \ 6\ \ \ 4\ \ \ 1 \\
        1\ \ \ 5\ \ \ \di A\ \ \ \di A\ \ \ 5\ \ \ 1 \\
        1\ \ \ 6\ \ \ \di F\ \ \ \di{14}\ \ \ \di F\ \ \ 6\ \ \ 1 \\
        1\ \ \ 7\ \ \ \di{15}\ \ \ \di{23}\ \ \ \di{23}\ \ \ \di{15}\ \ \ 7\ \ \ 1 \\
        1\ \ \ 8\ \ \ \di{1C}\ \ \ \di{38}\ \ \ \di{46}\ \ \ \di{38}\ \ \ \di{1C}\ \ \ 8\ \ \ 1 \\
        1\ \ \ 9\ \ \ \di{24}\ \ \ \di{54}\ \ \ \di{7E}\ \ \ \di{7E}\ \ \ \di{54}\ \ \ \di{24}\ \ \ 9\ \ \ 1 \\
        \vdots
      \er
    \]
  \ec
\end{figure}
\begin{figure}
  \nc\di[1]{\makebox[\wdig][c]{#1}}
  \caption{Pascal's triangle (in decimal notation).}
  \label{drvtv:pascdec}
  \setlength\tla{0.13em}
  \newcommand\tta{0.6}
  \bc
    \[
      \br{c}
        1 \\
        1\ \ \ 1 \\
        1\ \ \ 2\ \ \ 1 \\
        1\ \ \ 3\ \ \ 3\ \ \ 1 \\
        1\ \ \ 4\ \ \ 6\ \ \ 4\ \ \ 1 \\
        1\ \ \ 5\ \ \ \di{10}\ \ \ \di{10}\ \ \ 5\ \ \ 1 \\
        1\ \ \ 6\ \ \ \di{15}\ \ \ \di{20}\ \ \ \di{15}\ \ \ 6\ \ \ 1 \\
        1\ \ \ 7\ \ \ \di{21}\ \ \ \di{35}\ \ \ \di{35}\ \ \ \di{21}\ \ \ 7\ \ \ 1 \\
        1\ \ \ 8\ \ \ \di{28}\ \ \ \di{56}\ \ \ \di{70}\ \ \ \di{56}\ \ \ \di{28}\ \ \ 8\ \ \ 1 \\
        1\ \ \ 9\ \ \ \di{36}\ \ \ \di{\hspace{-\tta\tla}84}\ \ \ \di{\hspace{-\tla}126}%
          \ \ \ \di{126\hspace{-\tla}}\ \ \ \di{84\hspace{-\tta\tla}}\ \ \ \di{36}\ \ \ 9\ \ \ 1 \\
        \vdots
      \er
    \]
  \ec
\end{figure}

% ----------------------------------------------------------------------

\section{The binomial theorem}
\label{drvtv:230}
\index{binomial theorem}

This section presents the binomial theorem and one of its significant
consequences.

\subsection{Expanding the binomial}
\label{drvtv:230.20}

The \emph{binomial theorem} holds that%
\footnote{
  The author is given to understand that, by an heroic derivational
  effort,~(\ref{drvtv:230:binth}) can be extended directly to
  nonintegral~$n$.  However, we shall have no immediate need for such an
  extension.   Later, in Table~\ref{taylor:315:tbl}, we will compute
  the Taylor series for $(1+z)^{a-1}$, anyway, which indirectly amounts
  to much the same thing as the extension, and has a more elegant form
  to boot, and moreover (at least in the author's experience) arises
  much more often in applications.%
}
\bq{drvtv:230:binth}
  (a+b)^n = \sum_{k=0}^n \cmb{n}{k} a^{n-k}b^k.
\eq
In the common case that $a=1$, $b=\ep$, $\left| \ep \right| \ll 1$,
this is that
\bq{drvtv:230:binthe}
  (1+\ep)^n = \sum_{k=0}^n \cmb{n}{k} \ep^k
\eq
(actually, eqn.~\ref{drvtv:230:binthe} holds for any~$\ep$, small or
large; but the typical case of interest has that $|\ep| \ll 1$).  In
either form, the binomial theorem is a direct consequence of the
combinatorics of \S~\ref{drvtv:220}.  Since
\[
  (a+b)^n = (a+b)(a+b)\cdots(a+b)(a+b),
\]
each $(a+b)$ factor corresponds to one of the ``wooden blocks,'' where~$a$
means rejecting the block and~$b$, accepting it.

\subsection{Powers of numbers near unity}
\label{drvtv:230.30}

\index{$\approx$}
Since $\cmbl{n}{0} = 1$ and $\cmbl{n}{1} = n$, it follows
from~(\ref{drvtv:230:binthe}) for
\[
  (m,n) \in \mathbb Z, \ m > 0, \ n \ge 0,
  \ \left|\delta\right| \ll 1,
  \ \left|\ep\right| \ll 1,
  \ \left|\ep_o\right| \ll 1,
\]
that%
\footnote{
  The symbol~$\approx$ means ``approximately equals.''
}
\[
  1 + m\ep_o \approx (1+\ep_o)^m
\]
to arbitrary precision as long as~$\ep_o$ is small enough.
Furthermore, raising the equation to the $1/m$ power then changing
$\delta \la m\ep_o$, we have that
\[
  (1 + \delta)^{1/m} \approx 1+\frac{\delta}{m}.
\]
Changing $1+\delta \la (1+\ep)^n$ and observing from the $(1+\ep_o)^m$
equation above that this implies that $\delta \approx n\ep$, we have that
\[
  (1 + \ep)^{n/m} \approx 1+\frac{n}{m}\ep.
\]
Inverting this equation yields that
\[
  (1+\ep)^{-n/m} \approx \frac{1}{1 + (n/m)\ep} =
  \frac{[1-(n/m)\ep]}{[1-(n/m)\ep][1+(n/m)\ep]} \approx 1-\frac{n}{m}\ep.
\]
Taken together, the last two equations imply that
\bq{drvtv:230:apxex}
  (1+\ep)^x \approx 1 + x\ep
\eq
for any real~$x$.

\index{Taylor expansion, first-order}
\index{first-order Taylor expansion}
Obscurely,~(\ref{drvtv:230:apxex}) is called the \emph{first-order Taylor
expansion.}  The reason the equation is called by such an unwieldy name
will be explained in chapter~\ref{taylor}, but howsoever the equation
may be called, it is an important result.  The equation offers a
simple, accurate way of approximating any real power of numbers in the
near neighborhood of~1.

\subsection{Complex powers of numbers near unity}
\label{drvtv:230.35}

\index{power!complex}
\index{complex power}
\index{definition}
Equation~(\ref{drvtv:230:apxex}) is fine as far as it goes, but its very
form suggests the question: what if~$\ep$ or~$x$, or both, is complex?
Changing the symbol $z\la x$ and observing that the infinitesimal~$\ep$
also may be complex, one wants to know whether
\bq{drvtv:230:apxe}
  (1+\ep)^z \approx 1 + z\ep
\eq
still holds.  No work we have yet done in the book answers the question,
because though a complex infinitesimal~$\ep$ poses no particular
problem, the action of a complex power~$z$ remains undefined.  Still, for
consistency's sake, one would like~(\ref{drvtv:230:apxe}) to hold.  In
fact nothing prevents us from \emph{defining} the action of a complex power
such that~(\ref{drvtv:230:apxe}) does hold, which we now do, logically
extending the known result~(\ref{drvtv:230:apxex}) into the new domain.

But we cannot just define that, can we?  Surely we cannot glibly assert
that ``nothing prevents us'' and then go to define whatever we like!

Can we?

Actually, yes, in this case we can.  Consider that, insofar
as~(\ref{drvtv:230:apxe}) holds,
{
  \newcommand\tta{\ensuremath(1+\ep)^{z_1+z_2}}
  \newcommand\ttb{\ensuremath(1+\ep)^{z_1z_2}}
  \settowidth\tla{$\tta$}
  \bqb
    \lefteqn{\makebox[\tla][l]{$\tta$}
      = (1+\ep)^{z_1}(1+\ep)^{z_2} \approx (1+z_1\ep)(1+z_2\ep)}
    &&\\&&\makebox[\tla][l]{}
    = 1+z_1\ep+z_2\ep+z_1z_2\ep^2 \approx 1+(z_1+z_2)\ep;
    \\
    \lefteqn{\makebox[\tla][l]{$\ttb$} = \left[(1+\ep)^{z_1}\right]^{z_2}
    \approx
    [1+z_1\ep]^{z_2} \approx 1+z_1z_2\ep;}
  \eqb
}%
and so on.  These alone do not of course conclusively prove that our new
definition is destined to behave well in every circumstance of future interest.
Experience will tell.  Notwithstanding, in the meantime, since we seem unable
for the moment to identify a relevant circumstance in which our new definition
misbehaves, since our definition does seem a natural extension
of~(\ref{drvtv:230:apxex}), since it does not seem to contradict
anything we already know, and since no obvious alternative presents
itself, let us provisionally accept the definition and find out to what
results it leads.

Section~\ref{cexp:230} will investigate the extremely interesting
effects which arise when $\Re(\ep)=0$ and the power~$z$
in~(\ref{drvtv:230:apxe}) grows large, but for the moment we shall use
the equation in a more ordinary manner to develop the concept and basic
application of the derivative, as follows.

% ----------------------------------------------------------------------

\section{The derivative}
\label{drvtv:240}
\index{derivative!definition of}
\index{derivative!Newton notation for}
\index{Newton, Sir Isaac (1642--1727)}

\index{derivative!balanced form}
\index{derivative!unbalanced form}
With~(\ref{drvtv:230:apxe}) at least provisionally in hand, we can now turn to
the chapter's subject proper, the derivative.

What is the derivative?  The \emph{derivative} is the instantaneous rate
or slope of a function.  In mathematical symbols and for the moment
using real numbers,\footnote{%
  Professional mathematicians tend to prefer another, more
  self-contained definition.  Section~\ref{drvtv:240.38} will briefly
  present it.  See too
  % The word "eqns." should normally be omitted, but in this instance it
  % disambiguates compared to the last sentence.
  eqns.~(\ref{drvtv:defz}) and~(\ref{drvtv:defzunbal}).%
}
\bq{drvtv:def}
  f'(t) \equiv \lim_{\ep\rightarrow 0^+} \frac{f(t+\ep/2)-f(t-\ep/2)}{\ep}.
\eq
Alternately,
\bq{drvtv:defunbal}
  f'(t) \equiv \lim_{\ep\rightarrow 0^+} \frac{f(t+\ep)-f(t)}{\ep}.
\eq
Because~$\ep$ is infinitesimal, either the balanced
definition~(\ref{drvtv:def}) or the unbalanced
definition~(\ref{drvtv:defunbal}) should in theory yield the same result
(where it does not, you have a problem: the derivative does not exist at
that value of~$t$; for example, given $f[t]=1/t$, $f'[t]_{t=0}$ does not
exist despite that it exists at other values of~$t$).  Both definitions
have their uses but applied mathematicians tend to prefer the
balanced~(\ref{drvtv:def}) because it yields comparatively accurate
results in practical approximations in which~$\ep$, though small, is not
actually infinitesimal.\footnote{\cite[\S\S~I:9.6
and~I:9.7]{Feynman}\cite[\S~4.3.4]{Cunningham}}
Except where otherwise
stated, this book will prefer the balanced~(\ref{drvtv:def})---or
rather, as we shall eventually see, will prefer its generalized form,
the balanced~(\ref{drvtv:defz}).

(Note:  from this section through \S~\ref{drvtv:260}, the mathematical
notation necessarily grows a little thick.  This cannot be helped, so if
you are reading straight through, be prepared for a bit of a hard slog.)

\subsection{The derivative of the power series}
\label{drvtv:240.20}
\index{power series!derivative of}

In the very common case that $f(t)$ is the power series
\bq{drvtv:240:30}
  f(t) = \sum_{k=-\infty}^{\infty} c_k t^k,
\eq
where the~$c_k$ are in general complex coefficients,~(\ref{drvtv:def})
says that
\bqb
  f'(t) &=& \sum_{k=-\infty}^{\infty} \lim_{\ep\rightarrow 0^+} \frac{
    (c_k)(t+\ep/2)^k - (c_k)(t-\ep/2)^k
  }{\ep} \\
  &=& \sum_{k=-\infty}^{\infty} \lim_{\ep\rightarrow 0^+}
  c_kt^k
  \frac{
    (1+\ep/2t)^k - (1-\ep/2t)^k
  }{\ep}.
\eqb
Applying~(\ref{drvtv:230:apxe}),
\[
  f'(t)
  = \sum_{k=-\infty}^{\infty} \lim_{\ep\rightarrow 0^+}
  c_kt^k
  \frac{
    (1+k\ep/2t) - (1-k\ep/2t)
  }{\ep},
\]
which simplifies to
\bq{drvtv:240:polyderiv}
  f'(t) = \sum_{k=-\infty}^{\infty} c_kkt^{k-1},
\eq
assuming of course that the sum converges.%
\footnote{%
  The book will seldom again draw attention to such caveats of abstract
  rigor, even in passing.  For most \emph{concrete} series to which one
  is likely to apply~(\ref{drvtv:240:polyderiv}) in practice, the
  series' convergence or nonconvergence will be plain enough on its
  face, as abstract considerations of theoretical sumworthiness fade
  into an expedient irrelevance.  (For a closer applied consideration of
  sumworthiness nevertheless, see~\cite{Andrews}.)
}
Equation~(\ref{drvtv:240:polyderiv}) gives the general derivative of the
power series.%
\footnote{%
  Equation~(\ref{drvtv:240:polyderiv}) has not admittedly, explicitly
  considered what happens when the real~$t$ becomes the complex~$z$, but
  \S~\ref{drvtv:240.50} will remedy the oversight.
}

\subsection{The Leibnitz notation}
\label{drvtv:240.25}
\index{Leibnitz, Gottfried Wilhelm (1646--1716)}
\index{Leibnitz notation}
\index{derivative!Leibnitz notation for}
\index{infinitesimal!and the Leibnitz notation}
\index{dependent variable}
\index{independent variable}
\index{variable!dependent}
\index{variable!independent}

The $f'(t)$ notation used above for the derivative is due to Sir Isaac
Newton, and is easier to start with.  Usually better on the whole,
however (but see appendix~\ref{purec}), is G.~W.\ Leibnitz's notation,
\bqb
  dt &=& \ep,\\
  df &=& f(t+dt/2)-f(t-dt/2),
\eqb
such that per~(\ref{drvtv:def}),
\bq{drvtv:240:50}
  f'(t) = \frac{df}{dt}.
\eq
Here,~$dt$ is the infinitesimal, and~$df$ is a dependent infinitesimal
whose size \emph{relative to~$dt$} depends on the independent
variable~$t$.

\index{mental image of the infinitesimal}
\index{infinitesimal!mental image of}
Conceptually, one can choose any sufficiently small size~$\ep$ for the
independent infinitesimal~$dt$; and, actually, though we have
called~$dt$ ``independent,'' what we really mean is that the
variable~$t$ with which~$dt$ is associated is independent.  The size
of~$dt$ may be constant (this is typically easiest) but may instead
depend on~$t$ as $dt=\ep(t)$.  Fortunately, one seldom needs to say, or
care, what the size of an independent infinitesimal like~$dt$ is.  All
one normally needs to worry about are the sizes of other infinitesimals
in proportion to~$dt$.

As an example of the infinitesimal's use,
if $f(t) \equiv 3t^3 - 5$, then $f(t \pm dt/2) = 3(t \pm
dt/2)^3 - 5 = 3t^3 \pm (9/2)t^2\,dt + (9/4)t\,dt^2 \pm (3/8)dt^3 - 5$,
whence $df = f(t+dt/2)-f(t-dt/2) = 9t^2\,dt + (3/4)dt^3$, and thus
$df/dt = 9t^2 + (3/4)dt^2$---which has that $df/dt = 9t^2$ in the limit
as~$dt$ tends to vanish.  The example is easier
if~(\ref{drvtv:230:apxe}) is used to approximate that $f[(t)(1 \pm
dt/2t)] \approx 3t^3 \pm (9/2)t^2\,dt - 5$, the details of which are
left as an exercise.

Where two or more independent variables are simultaneously in play,
say~$s$ and~$t$, the mathematician can have two, distinct independent
infinitesimals~$ds$ and~$dt$---or, as one often styles them in such
cases,~$\partial s$ and~$\partial t$.  The size of~$\partial s$ may be
constant but may depend on~$s$, $t$, or both, as $\partial
s=\delta(s,t)$ where the~$\delta$ is like~$\ep$ an infinitesimal;
and, likewise, the size of~$\partial t$ may be constant but may depend
on~$s$, $t$, or both, as $\partial t=\ep(s,t)$.  Fortunately, as before,
one seldom needs to say or care what the sizes are.

An applied mathematician ought to acquire, develop and retain a clear,
lively, flexible mental image of Leibnitz's infinitesimal.

\subsection{Considerations of the Leibnitz notation}
\label{drvtv:240.26}
\index{Leibnitz, Gottfried Wilhelm (1646--1716)}
\index{Leibnitz notation}
\index{$d$ and~$\partial$}
\index{$\partial$ and~$d$}

The precise meaning of Leibnitz's letter~$d$ subtly depends on its
context.  In~(\ref{drvtv:240:50}), the meaning is clear enough:
$d(\cdot)$ signifies the amount by which~$(\cdot)$ changes while the
independent variable~$t$ is increasing by~$dt$.  Indeed, so
essential is this point to the calculus concept that it bears repeating
for emphasis!
\begin{quote}
  {\scshape Insofar as~$(\cdot)$ depends on~$t$, the notation
  $d(\cdot)$ signifies the amount by which~$(\cdot)$ changes while~$t$
  is increasing by~$dt$.}
\end{quote}

\noindent%
\index{independent variable!multiple}%
\index{variable!independent, multiple}%
\index{Jacobi, Carl Gustav Jacob (1804--1851)}%
The following notational anomaly intrudes to complicate the matter.
Where two or more independent variables are at work in the same
equation or model, for instance~$s$ and~$t$, convention warps Leibnitz's
letter~$d$ into the shape of Carl Jacobi's letter~$\partial$ (already
seen in \S~\ref{drvtv:240.25}).  Convention warps the letter, not for
any especially logical reason, but as a visual reminder that multiple
independents are in play.  For example, if $f(s,t) \equiv s^2 + 3st^2 +
t^4$, then $\partial_s f = (2s+3t^2)\partial s$ [which represents the
change~$f$ undergoes while~$s$ is increasing by an increment~$\partial
s$ and~$t$ is held constant] but $\partial_t f = (6st+4t^3)\partial t$
[which represents the change~$f$ undergoes while~$t$ is increasing by an
increment~$\partial t$ and~$s$ is held constant].

In practice, the style of~$\partial_s f$ and~$\partial_t f$ is usually
overelaborate.  Usually, one abbreviates each as~$\partial f$.  Context
normally clarifies.
%\footnote{\index{Cantor, Georg (1845--1918)}\index{Weierstrass, Karl Wilhelm Theodor (1815--1897)}%
%  % This footnote is a bit sharp, even impudent, maybe (though I still
%  % kinda like it).  --THB--
%  So, did Cantor not make a good point after all when he
%  execrated~\cite{Bell:Cantor} the very concept of the infinitesimal?
%  Answer: no, in your author's opinion, he did not.
%
%  If Cantor execrates the infinitesimal because he feels that the
%  institution of the infinitesimal frustrates his program, yet the
%  program nonetheless fails after the execrable device has been excluded
%  from it, then Cantor's distaste can hardly oblige you and me to
%  abandon the device.%
%}

\index{partial derivative}
\index{derivative!partial}
\index{differential equation}
\index{differential equation!ordinary}
\index{differential equation!partial}
\index{ordinary differential equation}
\index{partial differential equation}
A derivative like $\partial f/\partial s$ or $\partial f/\partial t$
(that is, like $\partial_s f/\partial s$ or $\partial_t f/\partial t$) is
called a \emph{partial derivative} because in it, only one of two or
more independent variables is varying.  An equation containing
derivatives (whether partial or otherwise), or containing infinitesimals
like~$df$ or~$\partial f$ that represent the change a dependent variable
like~$f$ undergoes, is called a \emph{differential equation.}  A
differential equation whose derivatives or infinitesimals track more
than one independent variable is called a \emph{partial differential
equation.}\footnote{%
  Chapter~\ref{vcalc} gives many examples of partial differential
  equations, for instance~(\ref{vcalc:440.cyl:grad}).%
}
A differential equation whose derivatives or
infinitesimals track only one independent variable is called an
\emph{ordinary differential equation.}

\index{nonstandard notation}
\index{notation!nonstandard}
Observe incidentally that the notation~$\partial_s f$ is nonstandard.
For obscure reasons (\S\S~\ref{drvtv:240.405} and~\ref{drvtv:240.407}),
the style usually instead seen in print is that of~$(\partial f/\partial
s)ds$, rather.

\index{partial~$d$}
\index{warped~$d$}
\index{$d,$ warped or partial}
The symbol~$\partial$ is merely a warped letter~$d$.
Chapter~\ref{integ} will use the warped letter a little, as will \S\S~\ref{taylor:370}
and~\ref{mtxinv:420}.  Chapter~\ref{vcalc} will use the warped letter a lot.
% diagn:  Should a reference be given here to the cylindrical solution
% of the wave eqn., maybe among others, once these have been added to
% the book?

\index{formal parameter}
\index{parameter!formal}
\index{energy!potential}
\index{energy!kinetic}
\index{potential energy}
\index{kinetic energy}
\index{independent variable!lack of}
\index{variable!independent, lack of}
We have mentioned equations with two or more independent variables.
However, some equations with infinitesimals, such as the
potential-kinetic energy equation that $ma\,dx=mv\,dv$, do not
explicitly include or refer to any independent variable at all.%
\footnote{%
  The~$m$ stands for mass, the~$x$ for position,
  the~$v$ for speed, and the~$a$ for acceleration.  The model's
  independent variable would probably be~$t$ for time but that variable
  does not happen to appear in this equation.%
}
Context can sometimes supply an independent the equation does not
mention, like~$t$, upon which~$x$ and~$v$ both depend; but it may be
that the equation speaks only to how~$x$ and~$v$ change conjointly,
without suggesting that either change caused the other and without
explicit reference to an independent of any kind.  Another example of
the sort would be the economist's demand-elasticity equation,
$e\,dP/P=dQ/Q$, which speaks to how~$P$ and~$Q$ change conjointly.%
\footnote{%
  The~$e$ (stated as a unitless negative number) stands for demand
  elasticity, the~$P$ for price, and the~$Q$ for quantity demanded.
  Refer to~\cite[chapter~4]{Jothi/Srin}.%
}
This is all right.  Moreover, even in the rare application in which the
lack of an independent does pose some trouble, one can often remedy
the trouble by introducing a purely formal parameter to serve as it were
an independent.

\index{factor of integration}
\index{infinitesimal factor of integration}
\index{integration!infinitesimal factor of}
Convention sports at least one other notational wrinkle we should
mention, a wrinkle that
comes into view from chapter~\ref{integ}.  Convention writes $\int_0^t
f(\tau)\,d\tau$ rather than $\int_0^t f(\tau)\,\partial\tau$, which is
to say that it eschews the
\linebreak % bad break
warped~$\partial$ when writing chapter~\ref{integ}'s
infinitesimal factor of integration.  One could explain this by
observing that the true independent~$t$ acts as a constant within the
dummy's scope, and thus that the dummy sees itself as a lone
independent within that scope; but whatever the explanation, that is how
mathematicians will write the thing.
%  The notational distinction between~$\partial$ and~$d$ is ubiquitous,
%  entrenched and harmless; and, besides, the distinction lends a
%  certain, classic look to equations in which the symbol~$\partial$
%  appears, the rounded~$\partial$ visually balancing the straight
%  division bar over and under which it typically appears; whereas the
%  straight-backed~$d$ puts a firm stop to the sinuous~$\int$.  Handsome
%  equations, after all, are for that not all to the bad.

\subsection{Remarks on the Leibnitz notation}
\label{drvtv:240.405}

A deep mystery is implicated.  None has wholly plumbed it.  Perhaps none
ever will.

\index{Thales (fl.~585~B.C.)}
\index{Anaximander (fl.~570~B.C.)}
\index{Anaximenes (fl.~550~B.C.)}
\index{Heraclitus (fl.~500~B.C.)}
\index{Parmenides (b.~c.~515~B.C.)}
\index{Zeno of Elea (fl.~460~B.C.)}
\index{Melissus (fl.~440~B.C.)}
\index{Anaxagoras (500--428~B.C.)}
\index{Leucippus (fl.~440~B.C.)}
\index{Democritus (b.~c.~458 B.C.)}
\index{Eudoxus (408--355~B.C.)}
\index{Euclid (325--265~B.C.)}
\index{Archimedes (287--212~B.C.)}
\index{Plato (428--348~B.C.)}
\index{Aristotle (384--322~B.C.)}
\index{Epicurus (341--271~B.C.)}
\index{Zeno of Cition (fl.~250~B.C.)}
\index{Chrysippus (280--206~B.C.)}
\index{antiquity}
\index{arithmetic}
\index{geometry}
\index{continuity}
\index{discreteness}
During antiquity,
Thales,
Anaximander,
Anaximenes,
Heraclitus,
Parmenides,
Zeno of Elea,
Melissus,
Anaxagoras,
Leucippus,
Democritus,
Eudoxus,
Euclid,
Archimedes,
Epicurus,
Zeno of Cition,
Chrysippus,
Plato and
Aristotle\footnote{These names and others are marshaled and accounted by~\cite{Bell:2006}.}
long debated---under various forms---whether material reality and, more
pertinently, immaterial reality\footnote{%
  An influential school of thought asserts that immaterial reality does
  not exist, or that it might as well not exist.  The school is
  acknowledged but the writer makes no further comment except that that
  is not what this paragraph is about.

  Meanwhile, mathematical ontology is something we can discuss but
  general ontology lies beyond the writer's expertise.%
}
are essentially \emph{continuous,} as geometry; or essentially
\emph{discrete,} as arithmetic.  (Here we use ``arithmetic'' in the
ancient sense of the word\@.)  We still do not know.  Indeed, we do not
even know whether this is the right question to ask.

\index{profundity}
\index{question!tailoring}
\index{tailoring the question}
One feels obliged to salute the erudition of the
professional mathematician's ongoing effort to find the question and
give the answer; and yet, after twenty-five centuries, when
the best efforts to give the answer seem to have succeeded chiefly to
the extent to which they have \emph{tailored the question}\footnote{%
  \index{Hercules}%
  Is this adverse criticism?  No.  Indeed, one can hardly see what else
  the best efforts might have done, given the Herculean task those
  efforts had set for themselves.  A task may, however, be too large, or
  be inherently impossible, even for Hercules.%
}
to suit whatever answer is currently known and deemed best, why, the
applicationist's interest in the matter may waver.

What the applicationist knows or believes is this: that the continuous
and the discrete---whether each separately or both together---appeal
directly to the mathematical intuition.  If the mind's eye already sees
both, and indeed sees them together in the same mental scene, then one
may feel little need further to deconstruct the two.\footnote{%
  \index{mind's eye}%
  \index{eye of the mind}%
  The mind's eye may be deceived where pure analysis does not err, of
  course, insofar as pure analysis relies upon disciplined formalisms.
  This is not denied.  What the mind's eye \emph{is,} and how it is able
  to perceive the abstract, are great questions of epistemology
  beyond the book's scope.%
}

One might, of course, inadvertently obscure the mental scene by
elaborating certain definitions, innocently having meant to place the
whole scene upon an indisputably unified basis;
but, if one does elaborate certain definitions \emph{and this act indeed
obscures the scene,} then one might ask:
is it not the definitions which are suspect?
Nowhere after all is it written that any indisputably unified basis
shall, even in principle, be accessible to the mind of man.\footnote{%
  ``But~ZFC is such a basis!'' comes the objection.

  \index{Wittgenstein, Ludwig (1889--1951)}
  However, whether~ZFC is truly a basis or is rather a clever
  contraption recently bolted onto the side of pre\"existing, even
  primordial mathematics is a question one can debate.  Wittgenstein
  debated it.  See \S~\ref{intro:284.4}.%
}
Such a basis might be accessible, or it might not.  The search is
honorable and worthwhile, but it can utterly fail.  We are not required
to abandon mathematics if it does.

To the applicationist meanwhile, during the modeling of a given
physical system, the choice of whether to employ the continuous or the
discrete will chiefly depend not on abstract considerations but rather
on the idiosyncratic demands of the problem at hand.

Such is the applicationist's creed.

\subsection{The professional's creed; further remarks}
\label{drvtv:240.407}

Yet, what of the \emph{professional's} creed?  May the professional not
also be heard?  And, anyway, what do any of these things have to do with
the Leibnitz notation?

\index{Brouwer, Luitzen Egbertus Jan (1881--1966)}
\index{truths of mathematics, the}
\index{stone}
\index{mountain, barren}
\index{disconsolate infinity}
\index{infinity!disconsolate}
\index{immovability}
\index{toleration}
\index{indulgence}
The professional may indeed be heard, and more eloquently elsewhere than
in this book; but, for this book's purpose, the answers are a matter
of perspective.
The professional mathematician L.~E.~J.\ Brouwer has memorably
described the truths of mathematics as ``fascinating by their
immovability, but horrifying by their lifelessness, like stones from
barren mountains of disconsolate
infinity.''\footnote{\label{drvtv:240.407fn1}Source:
\cite{vanAtten}\@.  Brouwer later changed his mind.}
Brouwer's stones have not always appeared to the professional
to countenance Leibnitz's infinitesimal.\footnote{\cite{Bell:Cantor}}
%of \S\S~\ref{drvtv:240.25} and~\ref{drvtv:240.26}
Though the professional may tolerate, though the professional may even
indulge, the applicationist's use of Leibnitz's infinitesimal as an
expedient, the professional may find himself unable to summon greater
enthusiasm for the infinitesimal than this.

\index{Bertrand Russell (1872--1970)}
Indeed, he may be even less enthusiastic.  As the professional
mathematician Bertrand Russell succinctly judges,
``[I]{n}{f}{i}{n}{i}{t}{e}{s}{i}{m}{a}{l}{s} as explaining continuity
must be regarded as unnecessary, erroneous, and
self-contradictory.''\footnote{\cite{Bell:2006}}

\index{Cantor, Georg (1845--1918)}
\index{Aquinas, St.~Thomas (1225-1274)}
The professional mathematician Georg Cantor recounts:
\begin{quote}
  [A] great quarrel arose among the philosophers, of whom some
  followed Aristotle, others Epicurus; still others, in order to remain
  aloof from this quarrel, declared with Thomas Aquinas that the
  continuum consisted neither of infinitely many nor of a finite number
  of parts, but of absolutely no parts. This last opinion seems to me to
  contain less an explanation of the facts than a tacit confession that
  one has not got to the bottom of the matter and prefers to get
  genteelly out of its way.\footnote{\emph{Ibid.}}
\end{quote}
(The present writer has no opinion on St.~Thomas' declaration or on Cantor's
interpretation thereof but, were it possible, would concur in Thomas'
preference to get out of the way.  Alas!)

\index{Weyl, Hermann (1885--1955)}
\index{Robinson, Abraham (1918--1974)}
\index{bridge}
\index{weather forecast}
\index{integrated circuit}
\index{space shot}
On the opposite side, you have professional mathematicians like
Hermann Weyl (as quoted in \S\S~\ref{intro:284.4} and~\ref{intro:284.45})
and Abraham Robinson, along maybe with Brouwer himself,\footnote{See
footnote~\ref{drvtv:240.407fn1}.} who have seemed to suggest that
professional mathematics might rearrange---or at least search out a more
congenial perspective upon---Brouwer's immovable stones to countenance
the infinitesimal
nevertheless.\footnote{\cite{Weyl:1918}\cite{Scholz}\cite{Robinson}\cite{Bell:2006}}
All the while, scientists and engineers have cheerfully kept plying the
infinitesimal.  Scientists and engineers appear to have been obtaining
good results, too: bridges; weather forecasts; integrated circuits;
space shots; etc.  It seems that whether one approves the infinitesimal
depends chiefly on whether one's focus is in applications or in
foundations.

The present book's focus is of course in applications.  Fortunately,
if you are a Platonist as the author is, or even if you are an
intuitionist as Brouwer was, then this particular collision of
foundations against applications need not much disturb you.  See
\S~\ref{intro:284.4}.

So, what of the infinitesimal, this concept which has exercised the
great philosophical minds of the ages?  After all this, how shall the
student of applied mathematics now approach it?

\index{consensus!vacillating}
\index{vacillating consensus}
Following twenty-five centuries of spirited debate and vacillating
consensus, the prudent student will remain skeptical of \emph{whatever}
philosophy the latest consensus (or this book) might press him to adopt.
%
Otherwise, at the undergraduate level at any rate, many students though
having learned a little calculus have not quite learned how to approach
the infinitesimal at all.  That is, they have never developed a clear
intuition as to what Leibnitz elements like~$df$, $dt$, $\partial g$
and~$\partial x$ individually might mean---especially when these
students have seen such elements heretofore only in certain, specific
combinations like $df/dt$, $\partial g/\partial x$ and $\int f(t)\,dt$.
Often, these students have developed positive misunderstandings
regarding such elements.  The vacillation of consensus may be blamed for
this.

% This paragraph was interesting in its initial conception but, as the
% surrounding text has evolved, the paragraph has proved unable to
% evolve with it.
%\index{Zermelo-Fraenkel and Choice set theory~(ZFC)}%
%\index{ZFC}%
%Teachers must take some of the blame for that, cautiously hesitating
%between the one school and the other, yet it is
%hard to see what else the teacher of our present era might done.  A
%teacher---simultaneously confronted by Leibnitz's infinitesimal and,
%contrarily, by Cantor's (and Russell's) anti-in\-finites\-i\-mal
%program---wants to turn both ways at once.  After all, Cantor's
%program in the form of~ZFC techniques (\S~\ref{intro:284.45}) has
%remarkably succeeded at representing or paralleling many kinds of
%mathematical logic;\footnote{The writer takes the professional's word
%for this.} whereas Leibnitz's infinitesimal has
%midwifed the modern industrialized world.  Still, to hesitate between
%the one and the other when first introducing students to calculus has
%arguably been a mistake.

Therefore, in this book, having acknowledged that an opposing,
meritorious school of thought exists, let us not hesitate between the
two schools.  Though the writer acknowledges the cleverness of certain
of Cantor's results (as for example in~\cite[\S\S~1.8 and~2.4]{Shilov})
and allows Cantor's ambition regarding the continuum its due, the author
is an applicationist and thus his book (you are reading it) esteems
Leibnitz more highly than it does Cantor.  The charge could be leveled
that the author does not grasp Cantor and there would be some truth in
this charge, but the book's esteem for Leibnitz is more than a mere
matter of preferred style.  A mystery of mathematical philosophy is
involved and deeply entwined, touched upon in \S~\ref{intro:284} and
again in this section, as Cantor himself would undoubtedly have been
the first to insist.  Also, remember, Weyl disputed Cantor, too.

%\index{Weierstrass, Karl Wilhelm Theodor (1815--1897)}%
%\index{Dedekind, Richard (1831--1916)}%
Even setting aside foundational mysteries and the formidable Cantor,
there is at any rate significant practical benefit in learning how to
handle the Leibnitz notation correctly.
Physicists, chemists, engineers and
economists have long been used to handling Leibnitz elements
individually.  For all these reasons among others, the present
section seeks to present each Leibnitz element in its proper,
individual light.

\index{Robinson, Abraham (1918--1974)}
The chief recent source to which professional mathematicians seem to
turn to bridge the questions this section ponders is Robinson's~1966
book \emph{Nonstandard Analysis}~\cite{Robinson}\@.
%\footnote{The author of the book you are now
%reading, an engineer, does not pretend to have digested Robinson's
%book despite that Robinson is a good writer; but to quote from
%Robinson's first two pages is nonetheless entertaining.}
To conclude the subsection, therefore, we may hear Robinson's words:
\begin{quotation}
  \noindent
  Suppose that we ask a well-trained mathematician for the meaning of
  [the derivative]
  \[
    \lim_{x\rightarrow x_o} \frac{f(x)-f(x_o)}{x-x_o} = a.
  \]
  %Then we may rely on it that \ldots\ his explanation will be thus:
  Then we may rely on it that [he will explain it as
  \S~\ref{drvtv:240.38} below].

  Let us now ask our mathematician whether he would not accept the
  following more direct interpretation\mdots

  For any~$x$ in the interval of definition of $f(x)$ such that
  $dx=x-x_o$ is \emph{infinitely close} to~$0$ but not equal to~$0$,
  the ratio $df/dx$, where
  \[
    df=f(x)-f(x_o),
  \]
  is \emph{infinitely close} to~$a$.

  To this question we may expect the answer that our definition may be
  simpler in appearance but unfortunately it is also meaningless.  If
  we then try to explain that two numbers are infinitely close to one
  another if their distance \ldots\ is \emph{infinitely small},\,\ldots\
  we shall probably be faced with the rejoinder that this is possible
  only if the numbers co\"incide.  And, so we may be told charitably,
  this obviously is not what we meant since it would make our
  explanation trivially wrong.

  However, in spite of this shattering rebuttal, the idea of
  infinitely small or \emph{infinitesimal} quantities seems to appeal
  naturally to our intuition.  At any rate, the use of infinitesimals
  was widespread during the formative stages of the Differential and
  Integral Calculus\mdots~\cite[\S~1.1]{Robinson}.
\end{quotation}
\index{Derbyshire, John (1945--)}%
\index{rugby}%
Your author is an engineer.  John Derbyshire
quips\footnote{Derbyshire is author of~\cite{Derbyshire} but this quip
came, not written, but spoken by him in~2015.} that we
``[e]{n}{g}{i}{n}{e}{e}{r}{s} were a semi-civilized tribe on an adjacent
island, beery oafs who played hard rugby and never listened to concert
music.'' Thus it seems fitting---whether your author be an oaf or
not!---that in the book you are reading the ``shattering rebuttal''
Robinson's hypothetical interlocutor has delivered shall not unduly
discomfit us.
%\footnote{%
%  If the writer may write in the first person for a moment:
%
%  I should have preferred not to speak of such philosophical matters at
%  all, did mathematical convention not leave one caught between the two
%  horns of a dilemma.  On the one horn, modern styles of mathematical
%  rigor impede the communication of the main point of a mathematical
%  idea to a scientist or engineer.  On the other horn, to avoid such
%  styles in formal writing seems to confess a lack of knowledge.
%  \emph{The style does not help but one must use it anyway,} especially
%  if one has not earned latitude by gaining an advanced degree in
%  mathematics!
%
%  [The footnote is abandoned from here.  I had thought that I was going
%  somewhere with it, but now am unsure to where.]
%}

The book will henceforth ply the Leibnitz notation and its
infinitesimals vigorously, with little further hedge, cavil or
equivocation.

\subsection{Higher-order derivatives}
\label{dvrtv:240.41}
\index{derivative!higher-order}
\index{higher-order derivative}

\index{derivative!second}
\index{second derivative}
\index{third derivative}
Conventional shorthand for $d(df)$ is~$d^2\!f$; for $(dt)^2$, $dt^2$; so
\[
  \frac{d(df/dt)}{dt} = \frac{d^2\!f}{dt^2}
\]
is a derivative of a derivative, or \emph{second derivative.}  By
extension, the notation
\[
  \frac{d^k\!f}{dt^k}
\]
represents the $k$th derivative.  For example, if $k=3$, then
$d[d(df)]=d^3\!f$, by which one writes $d^3\!f/dt^3$ for the derivative of
a derivative of a derivative, or third derivative.  So, if $f(t)\equiv t^4/8$,
then $df/dt = t^3/2$, $d^2\!f/dt^2 = 3t^2/2$ and $d^3\!f/dt^3 = 3t$.

\subsection{The derivative of a function of a complex variable}
\label{drvtv:240.50}
\index{derivative!of a function of a complex variable}
\index{complex variable}
\index{variable!complex}
\index{function!of a complex variable}

\index{referential infinitesimal}
\index{infinitesimal!referential}
For~(\ref{drvtv:def}) to be robust, one should like its ratio to
approach a single, common value for all sufficiently small~$\ep$, for
only when~$\ep$ grows beyond infinitesimal size should the ratio
of~(\ref{drvtv:def}) become inexact.  However,~(\ref{drvtv:def})
considers only real, positive~$\ep$.  What if~$\ep$ were not positive?
Indeed, what if~$\ep$ were not even real?

\index{derivative!balanced form}
\index{derivative!unbalanced form}
This turns out to be an important question, so let us now
revise~(\ref{drvtv:def}) to establish the slightly more general form
\bq{drvtv:defz}
  \frac{df}{dz} \equiv \lim_{\ep\rightarrow 0} \frac{f(z+\ep/2)-f(z-\ep/2)}{\ep}
\eq
and let us incidentally revise~(\ref{drvtv:defunbal}), also, to
establish the corresponding unbalanced form
\bq{drvtv:defzunbal}
  \frac{df}{dz} \equiv \lim_{\ep\rightarrow 0}
  \frac{f(z+\ep)-f(z)}{\ep},
\eq
where as in the section's introduction so here too applications tend to
prefer the balanced~(\ref{drvtv:defz}) over the
unbalanced~(\ref{drvtv:defzunbal}).

\index{derivative!nonexistent}
As in~(\ref{drvtv:def}), so too in~(\ref{drvtv:defz}) one should like the
ratio to approach a single, common value\footnote{%
  One can construct apparent exceptions like $f(z) = \sin(1/z)$.  If
  feeling obstreperous, one can construct far more unreasonable
  exceptions such as the one found toward the end of
  \S~\ref{taylor:320}.
  % diagn: check that the above reference continues to hold as the draft
  % changes
  The applied mathematician can hardly be asked to
  expand his definitions to accommodate all such mischief!  He hasn't
  the time.

  \index{Hadamard, Jacques (1865--1953)}
  When an apparent exception of the less unreasonable kinds arises in
  the context of a particular physical model, rather than attempting to
  accommodate the exception under the roof of an abstruse, universal
  rule, the applicationist is more likely to cook up a way to work
  around the exception in the specific context of the physical model at
  hand (as for example in the so-called Hadamard finite part
  of~\cite{WTAng}).%
  %
  %To the applicationist, the overgeneralization of methods is a vice,
  %for the most general possible method may also be the most useless
  %possible method.  A prudent degree of specialization (even if the
  %specialization is not explicitly delineated) can be preferable to an
  %attempt to accommodate all abstractly conceivable cases under the roof
  %of a single rule.
  %
  %You could write a whole book on how to expand definitions to
  %accommodate abstractly conceivable cases.  Some have done.  By the
  %time you had finished writing the book, though, the rest of us may
  %have forgotten the question!  No, the applicationist cannot afford the
  %time, but must be a bit more practical than that.  The applicationist
  %wants to keep his focus from straying too far from the collection of
  %physical models with which he might actually have to work.%
}
for all sufficiently small~$\ep$.  However, in~(\ref{drvtv:defz}) one
must consider not only the magnitude~$|\ep|$ of the referential
infinitesimal but also its phase $\arg \ep$ (\S~\ref{alggeo:225}).
%\footnote{
%  This book however prefers the balanced~(\ref{drvtv:defz}), among other
%  reasons because~(\ref{drvtv:defz}) more reliably approximates the
%  derivative of a function for which only discrete samples are
%  available~\cite[\S\S~I:9.6
%  and~I:9.7]{Feynman}\cite[\S~4.3.4]{Cunningham}\@.  Moreover, for this
%  writer at least the balanced definition just better captures the
%  subjective sense of the thing.%
%}
For example, supposing that the symbol~$\delta$ represented some
positive, real infinitesimal, it should be equally valid to let
$\ep=\delta$, $\ep=-\delta$, $\ep=i\delta$, $\ep=-i\delta$,
$\ep=(4-i3)\delta$, or any other infinitesimal value.
The ratio $df/dt$ of~(\ref{drvtv:defz}) ought to come out the same
for all these.
In fact, for the sake of robustness, one normally demands that the ratio
does come out the same; and~(\ref{drvtv:defz}) rather
than~(\ref{drvtv:def}) is the definition we normally use for the
derivative for this reason.  Specifically, where the
limit~(\ref{drvtv:defz}) or even~(\ref{drvtv:defzunbal}) is sensitive to
$\arg\ep$, there we normally say that the derivative does not exist.

\index{differentiability}
Where the derivative~(\ref{drvtv:defz}) does exist---that is, where the
derivative is finite and is insensitive to our choice of a complex,
infinitesimal value for~$\ep$---there we say that the function $f(z)$ is
\emph{differentiable.}

Excepting the nonanalytic parts of complex numbers ($\left|\cdot\right|$,
$\arg[\cdot]$, $[\cdot]^{*}$, $\Re[\cdot]$ and $\Im[\cdot]$; see
\S~\ref{alggeo:225.3}), plus the Heaviside unit step $u(t)$ and the
Dirac delta $\delta(t)$ (\S~\ref{integ:670}), most functions encountered
in applications do meet the criterion~(\ref{drvtv:defz}) except at
isolated nonanalytic points (like $z=0$ in $h[z]\equiv 1/z$ or
$g[z]\equiv\sqrt z$).  Meeting the criterion, such functions are fully
differentiable except at their poles (where the derivative goes infinite
in any case) and other nonanalytic points.  Particularly, the key
formula~(\ref{drvtv:230:apxe}), written here as
\[
  (1+\ep)^w \approx 1 + w\ep,
\]
works without modification when~$\ep$ is complex; so the
derivative~(\ref{drvtv:240:polyderiv}) of the general power series,
\bq{drvtv:240:polyderivz}
  \frac{d}{dz} \sum_{k=-\infty}^{\infty} c_kz^k
  = \sum_{k=-\infty}^{\infty} c_kkz^{k-1}
\eq
holds equally well for complex~$z$ as for real (but see also the next
subsection).

\subsection{The derivative of~$z^a$}
\label{drvtv:240.30}
\index{derivative!of~$z^a$}

Inspection of the logic of \S~\ref{drvtv:240.20} in light
of~(\ref{drvtv:230:apxe}) reveals that nothing prevents us from
replacing the real~$t$, real~$\ep$ and integral~$k$ of that section with
arbitrary, complex~$z$, $\ep$ and~$a$.  That is,
\bqb
  \frac{d(z^a)}{dz}
  &=& \lim_{\ep\rightarrow 0} \frac{
    (z+\ep/2)^a - (z-\ep/2)^a
  }{\ep} \\
  &=& \lim_{\ep\rightarrow 0} z^a
  \frac{
    (1+\ep/2z)^a - (1-\ep/2z)^a
  }{\ep} \\
  &=& \lim_{\ep\rightarrow 0} z^a
  \frac{
    (1+a\ep/2z) - (1-a\ep/2z)
  }{\ep},
\eqb
which simplifies to
\bq{drvtv:240.30:10}
  \frac{d(z^a)}{dz} = az^{a-1}
\eq
for any complex~$z$ and~$a$.

How exactly to evaluate~$z^a$ or $z^{a-1}$ when~$a$ is complex
is another matter, treated in \S~\ref{cexp:230} and
its~(\ref{cexp:230:33}); but in any case you can
use~(\ref{drvtv:240.30:10}) for real~$a$ right now.

\subsection{An alternate definition of the derivative}
\label{drvtv:240.38}

Professional mathematicians tend to prefer a less
picturesque, alternate definition of the derivative~(\ref{drvtv:defz})
or~(\ref{drvtv:defzunbal}):
``For any positive number~$\ep$ there exists a positive
number~$\delta$ such that
\bq{drvtv:defp}
  \left|\frac{f(z)-f(z_o)}{z-z_o} - a\right| < \ep
\eq
for all~$z$ \ldots\ for which
\bq{drvtv:defpd}
  0<|z-z_o|<\delta,
\eq
[the quantity~$a$ being the \emph{derivative} $df/dz$].''\footnote{%
  The quoted original is~\cite[\S~1.1]{Robinson}, from which the
  notation has been adapted to this book's usage.%
}%

Equations~(\ref{drvtv:defp}) and~(\ref{drvtv:defpd}) bring few practical
advantages to applications but are at least more self-contained
than~(\ref{drvtv:defz}) or~(\ref{drvtv:defzunbal}) is.  However that may
be, the derivative is such a pillar of mathematics that it behooves the
applied mathematician to learn at least to recognize the professional's
preferred definition of it.  See also \S\S~\ref{drvtv:240.405}
and~\ref{drvtv:240.407}.
% I believe that the following is wrong.
%Amusingly,
%unlike~(\ref{drvtv:defz}), (\ref{drvtv:defp}) automatically notices
%that, if $f(x) = 1/x$ (for example), then the derivative
%$df/dx|_{x=0}=a|_{x=0}$ does not exist.  How practically useful this
%kind of automation is in applications, and whether it pays for the extra
%esotericity~(\ref{drvtv:defp}) and~(\ref{drvtv:defpd}) bring, can be
%debated.

\subsection{The logarithmic derivative}
\label{drvtv:240.40}
\index{logarithmic derivative}
\index{derivative!logarithmic}
\index{rate!relative}
\index{relative rate}
\index{rate!proportional}
\index{proportional rate}

\index{bond}
\index{interest}
\index{rate!absolute}
\index{rate!of interest}
\index{absolute rate}
Sometimes one is more interested to know the rate of $f(t)$
\emph{in proportion to the value of $f(t)$} than to know the absolute rate
itself.  For example, if you inform me that you earn $\$\:1000$ a year
on a bond you hold, then I may commend you vaguely for your thrift but
otherwise the information does not tell me much.  However, if you inform
me instead that you earn~10 percent a year on the same bond, then I
might want to invest.  The latter figure is a \emph{proportional rate}
or \emph{logarithmic derivative,}
\bq{drvtv:240.40:10}
  \frac{df/dt}{f(t)} = \frac{d}{dt}\ln f(t).
\eq
The investment principal grows at the absolute rate $df/dt$ but the
bond's proportional rate, also called (in the case of a bond) its
\emph{interest rate,} is $(df/dt)/f(t)$.

The natural logarithmic notation $\ln f(t)$ may not mean much to you yet,
for we'll not introduce it formally until \S~\ref{cexp:225}, so you can
ignore the right side of~(\ref{drvtv:240.40:10}) for the moment; but the
equation's left side at least should make sense to you.  It expresses
the significant concept of a proportional rate, like~10 percent annual
interest on a bond.

% ----------------------------------------------------------------------

\section{Basic manipulation of the derivative}
\label{drvtv:250}
\index{derivative!manipulation of}

This section introduces the derivative chain and product rules.

\subsection{The derivative chain rule}
\label{drvtv:250.20}
\index{derivative!chain rule for}
\index{chain rule, derivative}

If~$f$ is a function of~$w$, which itself is a function of~$z$, then%
\footnote{
  For example, one can rewrite
  \[
    f(z) = \sqrt{3z^2 - 1}
  \]
  in the form
  \bqb
    f(w) &=& w^{1/2}, \\
    w(z) &=& 3z^2 - 1.
  \eqb
  Then
  \bqb
    \frac{df}{dw} &=& \frac{1}{2w^{1/2}} = \frac{1}{2\sqrt{3z^2 - 1}}, \\
    \frac{dw}{dz} &=& 6z,
  \eqb
  so by~(\ref{drvtv:chain}),
  \[
    \frac{df}{dz} = \left(\frac{df}{dw}\right) \left(\frac{dw}{dz}\right)
    = \frac{ 6z }{ 2\sqrt{3z^2 - 1} }
    = \frac{ 3z }{ \sqrt{3z^2 - 1} }.
  \]
}
\bq{drvtv:chain}
  \frac{df}{dz} = \left(\frac{df}{dw}\right) \left(\frac{dw}{dz}\right).
\eq
Equation~(\ref{drvtv:chain}) is the \emph{derivative chain rule.}%
\footnote{
  It bears emphasizing to readers who may inadvertently have picked up
  unhelpful ideas about the Leibnitz notation in the past:  the~$dw$
  factor in the denominator cancels the~$dw$ factor in the numerator,
  and a thing divided by itself is~1.  On an applied level, this more or
  less is all there is to it (but see \S~\ref{drvtv:240}).  Other than
  maybe in degenerate cases like~$dw=0$, cases the applied mathematician
  will treat individually as they come, there is hardly more to the
  applied proof of the derivative chain rule than this (but
  see~\cite[Prob.~3.39]{Spiegel}).%
  %
  % Am not sure that the rest of this footnote helps.  Let's try
  % commenting it out.
  %Actually, it is not to $dw=0$ as
  %such that the objection is raised, but the very notion of the
  %infinitesimal~$dw$ troubles some Cantorian mathematicians.  They have
  %their reasons, and they tend to prefer to regard the
  %infinitesimal~$dw$ merely as a convenient informality---as a shorthand
  %to an implied limit, which of course it is---but this book will not
  %worry overmuch about such as these.
}

\subsection{The derivative product rule}
\label{drvtv:250.30}
\index{derivative!product rule for}
\index{product rule, derivative}

In general per~(\ref{drvtv:defz}),
\[
  d\left[\prod_j f_j(z)\right] =
      \prod_j f_j\left(z+\frac{dz}{2}\right)
    - \prod_j f_j\left(z-\frac{dz}{2}\right).
\]
But to first order,
\[
  f_j\left(z\pm\frac{dz}{2}\right)
  \approx f_j(z) \pm \left(\frac{df_j}{dz}\right)\left(\frac{dz}{2}\right)
  = f_j(z) \pm \frac{df_j}{2};
\]
so, in the limit,
\[
  d\left[\prod_j f_j(z)\right] =
      \prod_j \left(f_j(z) + \frac{df_j}{2}\right)
    - \prod_j \left(f_j(z) - \frac{df_j}{2}\right).
\]
Since the product of two or more~$df_j$ is negligible
compared to the first-order infinitesimals to which they are here added,%
\footnote{Unless~$df_j \approx 0$ to first order, in which case it
  contributes nothing to the derivative, anyway.}
this simplifies to
\[
  d\left[\prod_j f_j(z)\right] =
      \left[ \prod_j f_j(z) \right] \left[\sum_k \frac{df_k}{2f_k(z)} \right]
    - \left[ \prod_j f_j(z) \right] \left[\sum_k \frac{-df_k}{2f_k(z)} \right],
\]
or in other words
\bq{drvtv:prod}
  d\prod_j f_j =
      \left[ \prod_j f_j \right]\left[ \sum_k \frac{df_k}{f_k} \right].
\eq
In the common case of only two~$f_j$, this comes to
\bq{drvtv:prod2}
  d(f_1f_2) = f_2\,df_1 + f_1\,df_2.
\eq
On the other hand, if $f_1(z) = f(z)$ and $f_2(z) = 1/g(z)$, then by the
derivative chain rule~(\ref{drvtv:chain}), $df_2 = -dg/g^2$; so,
\bq{drvtv:proddiv}
  d\left(\frac f g\right) = \frac{g\,df - f\,dg}{g^2},
\eq
and indeed
\bq{drvtv:proddiv2}
  d\left(\frac{f^a}{g^b}\right)
    = \frac{f^{a-1}}{g^{b+1}}(ag\,df-bf\,dg).
\eq
Similarly,
\bq{drvtv:prod4}
  \begin{split}
    d\left(f_1^{a_1}f_2^{a_2}\right) &=
    \Big(f_1^{a_1-1}f_2^{a_2-1}\Big)(a_1f_2\,df_1+a_2f_1\,df_2)\\
    &=
    \Big(f_1^{a_1}f_2^{a_2}\Big)\left(\frac{a_1\,df_1}{f_1}+\frac{a_2\,df_2}{f_2}\right).
  \end{split}
\eq
Equation~(\ref{drvtv:prod}) is the \emph{derivative product rule.}

After studying the complex exponential in chapter~\ref{cexp}, we shall stand
in a position to write~(\ref{drvtv:prod}) in the slightly specialized but often
useful form%
\footnote{
  This paragraph is extra.  You can skip it for now if you prefer.
}
\bqa
  \lefteqn{d\left[\prod_j g_j^{a_j} \prod_j e^{b_jh_j} \prod_j \ln c_jp_j \right]} &&
  \xn\\&=& \left[ \prod_j g_j^{a_j} \prod_j e^{b_jh_j} \prod_j \ln c_jp_j \right]
  \xn\\&& \ \ \mbox{}\times
      \left[ \sum_k a_k\frac{dg_k}{g_k} + \sum_k b_k\,dh_k
        + \sum_k \frac{dp_k}{p_k\ln c_kp_k}\right].
  \label{drvtv:prod3}
\eqa
where the~$a_k$, $b_k$ and~$c_k$ are arbitrary complex coefficients and
the~$g_k$, $h_k$ and~$p_k$ are arbitrary functions.%
\footnote{
  The subsection is sufficiently abstract that it is a little hard to
  understand unless one already knows what it means.  An example may
  help:
  \[
    d\left[\frac{u^2v^3}{z} e^{-5t} \ln 7s \right]
    = \left[\frac{u^2v^3}{z} e^{-5t} \ln 7s \right]
    \left[
      2\frac{du}{u} + 3\frac{dv}{v} - \frac{dz}{z} - 5\,dt
      + \frac{ds}{s\ln 7s}
    \right].
  \]
}

\subsection{A derivative product pattern}
\label{drvtv:250.35}
\index{pattern!derivative product}
\index{product rule, derivative!a pattern of}
\index{derivative!product rule for}
\index{derivative product!a pattern of}
\index{derivative pattern}

According to~(\ref{drvtv:prod2}) and~(\ref{drvtv:240.30:10}), the
derivative of the product $z^af(z)$ with respect to its independent
variable~$z$ is
\[
  \frac{d}{dz}[z^af(z)] = z^a\frac{df}{dz} + az^{a-1}f(z).
\]
Swapping the equation's left and right sides then dividing through
by~$z^a$ yields that
\bq{drvtv:250:35}
  \frac{df}{dz} + a\frac{f}{z} = \frac{d(z^af)}{z^a \,dz},
\eq
a pattern worth committing to memory, emerging among other places in
\S~\ref{vcalc:440}.%
\footnote{\label{drvtv:250:fn1}%
  This section completes the forward reference of
  \S~\ref{alggeo:228.40}.  See chapter~\ref{alggeo}'s
  footnote~\ref{alggeo:228:fn2}.
}

% ----------------------------------------------------------------------

\section{Extrema and higher derivatives}
\label{drvtv:255}
\index{extremum}
\index{minimum}
\index{maximum}
\index{derivative!higher}
\index{function!extremum of}

\index{evaluation}
One problem which arises very frequently in applied mathematics is the
problem of finding a local \emph{extremum}---that is, a local minimum or
max\-imum---of a real-valued function $f(x)$.  Refer to
Fig.~\ref{drvtv:255:fig1}.
\begin{figure}
  \caption{A local extremum.}
  \label{drvtv:255:fig1}
  \bc
    \begin{pspicture}(-2,-1.5)(7.5,4)
      {
        \small
        \psline[linewidth=0.5pt](-1,0)(6,0)
        \psline[linewidth=0.5pt](0,-0.5)(0,3)
        \psplot[linewidth=2.0pt,plotpoints=200]{0.5}{5.0}{
          x 2.5 sub dup mul -0.15 mul 2 add
        }
        \psdot[linewidth=2.0pt](2.5,2.0)
        \psline[linestyle=dashed,linewidth=0.5pt](0,2.0)(4.5,2.0)
        \psline[linestyle=dashed,linewidth=0.5pt](2.5,0)(2.5,2.0)
        \rput(2.5,-0.3){$x_o$}
        \rput[r](-0.15,2.0){$f(x_o)$}
        \rput(5.4,0.9){$f(x)$}
        \rput(6.3,0){$x$}
        \rput(0,3.3){$y$}
      }
    \end{pspicture}
  \ec
\end{figure}
The almost distinctive characteristic of the extremum $f(x_o)$ is that%
\footnote{
  The notation $P|_Q$ means ``$P$ when $Q$,'' ``$P$, given $Q$,'' or
  ``$P$ evaluated at $Q$.''  Sometimes it is alternately
  written $P|Q$ or $[P]_Q$.
}
\bq{drvtv:255:10}
  \left.\frac{df}{dx}\right|_{x=x_o} = 0.
\eq
At the extremum, the slope is zero.  The curve momentarily runs level
there.  One solves~(\ref{drvtv:255:10}) to find the extremum.

\index{slope}
\index{derivative!second}
\index{second derivative}
\index{inflection}
Whether the extremum be a minimum or a maximum depends on wheth\-er the
curve turn from a downward slope to an upward, or from an upward slope
to a downward, respectively.  If from downward to upward, then
the derivative of the slope is evidently positive; if from upward to
downward, then negative.  But the derivative of the slope is just the
derivative of the derivative, or second derivative.  Hence if $df/dx=0$
at $x=x_o$, then
\bqb
  \left.\frac{d^2\!f}{dx^2}\right|_{x=x_o} &>& 0
    \ \ \mbox{implies a local minimum at~$x_o$;} \\
  \left.\frac{d^2\!f}{dx^2}\right|_{x=x_o} &<& 0
    \ \ \mbox{implies a local maximum at~$x_o$.}
\eqb
Regarding the case
\[
  \left.\frac{d^2\!f}{dx^2}\right|_{x=x_o} = 0,
\]
this might be either a minimum or a maximum but more probably is
neither, being rather a \emph{level inflection point} as depicted in
Fig.~\ref{drvtv:255:fig2}.%
\footnote{
  Of course if the first and second derivatives are zero not just at
  $x=x_o$ but everywhere, then $f(x) = y_o$ is just a level straight
  line, but you knew that already.  Whether one chooses to call some
  arbitrary point on a level straight line an inflection point or an
  extremum, or both or neither, would be a matter of definition, best
  established not by prescription but rather by the needs of the model
  at hand.
}
\begin{figure}
  \caption{A level inflection.}
  \label{drvtv:255:fig2}
  \bc
    \begin{pspicture}(-2,-1.5)(7.5,4)
      {
        \small
        \psline[linewidth=0.5pt](-1,0)(6,0)
        \psline[linewidth=0.5pt](0,-0.5)(0,3)
        \psplot[linewidth=2.0pt,plotpoints=200]{0.5}{5.0}{
          x 2.5 sub dup dup mul mul -0.15 mul 2 add
        }
        \psdot[linewidth=2.0pt](2.5,2.0)
        \psline[linestyle=dashed,linewidth=0.5pt](0,2.0)(4.5,2.0)
        \psline[linestyle=dashed,linewidth=0.5pt](2.5,0)(2.5,2.0)
        \rput(2.5,-0.3){$x_o$}
        \rput[r](-0.15,2.0){$f(x_o)$}
        \rput(5.1,0.9){$f(x)$}
        \rput(6.3,0){$x$}
        \rput(0,3.3){$y$}
      }
    \end{pspicture}
  \ec
\end{figure}
(In general the term \emph{inflection point} signifies a point at which
the second derivative is zero.  The inflection point of
Fig.~\ref{drvtv:255:fig2} is \emph{level} because its first derivative
is zero, too.)

% ----------------------------------------------------------------------

\section{L'H\^opital's rule}
\label{drvtv:260}
\index{l'H\^opital's rule}
\index{l'H\^opital, Guillaume de (1661--1704)}
\index{root}
\index{pole}

If $z=z_o$ is a root of both $f(z)$ and $g(z)$, or alternately if
$z=z_o$ is a pole of both functions---that is, if both functions go to
zero or infinity together at $z=z_o$---then \emph{l'H\^opital's rule} holds
that
\bq{drvtv:260:lhopital}
  \lim_{z\ra z_o} \frac{f(z)}{g(z)}
  = \left. \frac{df/dz}{dg/dz} \right|_{z=z_o}.
\eq
In the case in which $z=z_o$ is a root, l'H\^opital's rule is
proved by reasoning%
\footnote{
  Partly with reference to
  \cite[``L'Hopital's rule,'' 03:40, 5 April 2006]{wikip}.
}
\bqb
  \lefteqn{
    \lim_{z\ra z_o} \frac{f(z)}{g(z)}
    = \lim_{z\ra z_o} \frac{f(z)-0}{g(z)-0}
  } &&\\&&
  = \lim_{z\ra z_o} \frac{f(z)-f(z_o)}{g(z)-g(z_o)}
  = \lim_{z\ra z_o} \frac{df}{dg}
  = \lim_{z\ra z_o} \frac{df/dz}{dg/dz}.
\eqb
In the case in which $z=z_o$ is a pole, new functions $F(z)\equiv 1/f(z)$
and $G(z)\equiv 1/g(z)$ of which $z=z_o$ is a root are defined, with
which
\[
  \lim_{z\ra z_o} \frac{f(z)}{g(z)}
  = \lim_{z\ra z_o} \frac{G(z)}{F(z)}
  = \lim_{z\ra z_o} \frac{dG}{dF}
  = \lim_{z\ra z_o} \frac{-dg/g^2}{-df/f^2},
\]
where we have used the fact from~(\ref{drvtv:240.30:10}) that
$d(1/u)=-du/u^2$ for any~$u$.  Canceling the minus signs and
multiplying by $g^2/f^2$, we have that
\[
  \lim_{z\ra z_o} \frac{g(z)}{f(z)}
  = \lim_{z\ra z_o} \frac{dg}{df}.
\]
Inverting,
\[
  \lim_{z\ra z_o} \frac{f(z)}{g(z)}
  = \lim_{z\ra z_o} \frac{df}{dg}
  = \lim_{z\ra z_o} \frac{df/dz}{dg/dz}.
\]
And if~$z_o$ itself is infinite?  Then, whether it represents a root or
a pole, we define the new variable $Z\equiv 1/z$ and the new functions
$\Phi(Z)\equiv f(1/Z)=f(z)$ and $\Gamma(Z)\equiv g(1/Z)=g(z)$, with which we
apply l'H\^opital's rule for $Z\ra 0$ to obtain
\bqb
  \lefteqn{
    \lim_{z\ra \infty} \frac{f(z)}{g(z)}
    = \lim_{Z\ra 0} \frac{\Phi(Z)}{\Gamma(Z)}
    = \lim_{Z\ra 0} \frac{d\Phi/dZ}{d\Gamma/dZ}
    = \lim_{Z\ra 0} \frac
      {df/dZ}
      {dg/dZ}
  } &&\\&&
  = \lim_{
      \stackindexdecl{
        z &\ra& \infty, \\
        Z &\ra& 0
      }
    } \frac{(df/dz)(dz/dZ)}{(dg/dz)(dz/dZ)}
  = \lim_{z\ra \infty} \frac{(df/dz)(-z^2)}{(dg/dz)(-z^2)}
  = \lim_{z\ra \infty} \frac{df/dz}{dg/dz}.
\eqb

Nothing in the derivation requires that~$z$ or~$z_o$ be real.  Nothing
prevents one from applying l'H\^opital's rule recursively,
should the occasion arise.%
\footnote{
  Consider for example the ratio $\lim_{x\ra 0} (x^3+x)^2/x^2$, which is
  $0/0$.  The easier way to resolve this particular ratio would
  naturally be to cancel a factor of~$x^2$ from it; but just to make the
  point let us apply l'H\^opital's rule instead, reducing the ratio
  to $\lim_{x\ra 0} 2(x^3+x)(3x^2+1)/2x$, which is still $0/0$.
  Applying l'H\^opital's rule again to the result yields $\lim_{x\ra 0}
  2[(3x^2+1)^2+(x^3+x)(6x)]/2 = 2/2 = 1$.
  % diagn
  %Where expressions involving trigonometric or special functions
  %(chapters~\ref{trig}, \ref{cexp} and [not yet written])
  Where expressions involving trigonometric functions
  (chapters~\ref{trig} and \ref{cexp})
  or special functions
  (mentioned in part~\ref{part30})
  appear in ratio, a recursive application of l'H\^opital's rule can be
  just the thing one needs.

  Observe that one must stop applying l'H\^opital's rule once the
  ratio is no longer $0/0$ or $\infty/\infty$.  In the example, applying
  the rule a third time would have ruined the result.
}

\index{indeterminate form}
L'H\^opital's rule is used in evaluating indeterminate forms of the kinds
$0/0$ and $\infty/\infty$, plus related forms like $(0)(\infty)$
which can be recast into either of the two main forms.  Good examples of
the use require mathematics from chapter~\ref{cexp} and later, but if we may borrow
from~(\ref{cexp:225:dln}) the natural logarithmic function and its
derivative,%
\footnote{
  This paragraph is optional reading for the moment.  You can read
  chapter~\ref{cexp} first, then come back here and read the paragraph if
  you prefer.
}
\[
  \frac{d}{dx}\ln x = \frac{1}{x},
\]
then a typical l'H\^opital example is%
\footnote{\cite[\S~10-2]{Shenk}}
\[
  \lim_{x\ra\infty}\frac{\ln x}{\sqrt x}
  = \lim_{x\ra\infty}\frac{1/x}{1/2\sqrt x}
  = \lim_{x\ra\infty}\frac{2}{\sqrt x} = 0.
\]
The example incidentally shows that natural logarithms grow slower than
square roots, an instance of a more general principle we shall meet in
\S~\ref{cexp:228}.

Section~\ref{cexp:228} will put l'H\^opital's rule to work.

% ----------------------------------------------------------------------

\section{The Newton-Raphson iteration}
\label{drvtv:270}
\index{Newton-Raphson iteration}
\index{Raphson, Joseph (1648--1715)}
\index{Newton, Sir Isaac (1642--1727)}
\index{root!finding of numerically}

\index{iteration}
The \emph{Newton-Raphson iteration} is a powerful, fast converging,
broadly applicable method for finding roots numerically.  Given a
function $f(z)$ of which the root is desired, the Newton-Raphson
iteration is
\bq{drvtv:NR}
  z_{k+1} = \left[ z - \frac{f(z)}{\frac{d}{dz}f(z)}\right]_{z=z_k}.
\eq
One begins the iteration by guessing the root and calling the guess~$z_0$.
Then~$z_1$, $z_2$, $z_3$, etc., calculated in turn by the
iteration~(\ref{drvtv:NR}), give successively better estimates of the
true root~$z_\infty$.

\index{tangent line}
To understand the Newton-Raphson iteration, consider the function $y=f(x)$
of Fig~\ref{drvtv:270:fig1}.
\begin{figure}
  \caption{The Newton-Raphson iteration.}
  \label{drvtv:270:fig1}
  \bc
    \begin{pspicture}(-2,-1.5)(7.5,4)
      {
        \small
        \psline[linewidth=0.5pt](-1,0)(6,0)
        \psline[linewidth=0.5pt](0,-0.5)(0,3)
        \psplot[linewidth=2.0pt,plotpoints=200]{1.0}{5.0}{
          x 6.0 sub dup mul 0.15 mul -1 add
        }
        \psdot[linewidth=2.0pt](1.35,2.2434)
        \psline[linestyle=dashed,linewidth=1.0pt](1.35,2.2434)(3.3000,-0.47687)
        \psline[linestyle=dashed,linewidth=0.5pt](1.35,0)(1.35,2.0)
        \psline[linewidth=0.5pt](1.35,-0.2)(1.35,0)
        \psline[linewidth=0.5pt](2.9582,-0.2)(2.9582,0)
        \rput(1.35,-0.4){$x_k$}
        \rput(2.8,-0.4){$x_{k+1}$}
        \rput(4.50,-1.0){$f(x)$}
        \rput(6.3,0){$x$}
        \rput(0,3.3){$y$}
      }
    \end{pspicture}
  \ec
\end{figure}
The iteration approximates the curve $f(x)$ by its tangent line%
\footnote{
  A \emph{tangent} line, also just called a \emph{tangent,} is the line
  which most nearly approximates a curve at a given point.  The tangent
  touches the curve at the point, and in the neighborhood of the point
  it goes in the same direction the curve goes.  The dashed line in
  Fig.~\ref{drvtv:270:fig1} is a good example of a tangent line.

  The relationship between the tangent line and the trigonometric
  tangent function of chapter~\ref{trig} is slightly obscure, maybe more of
  linguistic interest than of mathematical.  The trigonometric tangent
  function is named from a variation on Fig.~\ref{trig:226:f1} in which
  the triangle's bottom leg is extended to unit length, leaving the
  rightward leg tangent to the circle.
}
(shown as the dashed line in the figure):
\[
  \tilde f_k(x) = f(x_k) + \left[\frac{d}{dx}f(x)\right]_{x=x_k} (x-x_k).
\]
It then approximates the root $x_{k+1}$ as the point at which $\tilde
f_k(x_{k+1}) = 0$:
\[
  \tilde f_k(x_{k+1}) = 0
  = f(x_k) + \left[\frac{d}{dx}f(x)\right]_{x=x_k} (x_{k+1}-x_k).
\]
Solving for $x_{k+1}$, we have that
\[
  x_{k+1} = \left[ x_k - \frac{f(x)}{\frac{d}{dx}f(x)}\right]_{x=x_k},
\]
which is~(\ref{drvtv:NR}) with $x\la z$.

Although the illustration uses real numbers, nothing forbids
complex~$z$ and $f(z)$.  The Newton-Raphson iteration works just as well
for these.

The principal limitation of the Newton-Raphson arises when the function
has more than one root, as most interesting functions do.  The iteration
often converges on the root nearest the initial guess~$z_o$ but does not
always, and in any case there is no guarantee that the root
it finds is the one you wanted.  The most straightforward way to
beat this problem is to find \emph{all} the roots: first you find some
root~$\alpha$, then you remove that root (without affecting any of the
other roots) by dividing $f(z)/(z-\alpha)$, then you find the next root
by iterating on the new function $f(z)/(z-\alpha)$, and so on until you
have found all the roots.  If this procedure is not practical (perhaps
because the function has a large or infinite number of roots), then you
should probably take care to make a sufficiently accurate initial guess
if you can.

A second limitation of the Newton-Raphson is that, if you happen to
guess~$z_0$ especially unfortunately, then the iteration might never
converge at all.  For example, the roots of $f(z)=z^2+2$ are $z=\pm
i\sqrt 2$, but if you guess that $z_0 = 1$ then the iteration has no way
to leave the real number line, so it never converges%
\footnote{
  It is entertaining to try this on a computer.  Then try again with
  $z_0 = 1+i2^{-\mr{0x10}}$.
}
(and if you guess that $z_0 = \sqrt{2}$---well, try it with your pencil
and see what~$z_2$ comes out to be).  You can fix the problem with a
different, possibly complex initial guess.

A third limitation arises where there is a multiple root.  In this case,
the Newton-Raphson normally still converges, but relatively slowly.
For instance, the Newton-Raphson converges relatively slowly on the
triple root of $f(z) = z^3$.  However, even the relatively slow
convergence is still pretty fast and is usually adequate, even for
calculations by hand.

Usually in practice, the Newton-Raphson iteration works very well.  For
most functions, once the Newton-Raphson finds the root's neighborhood,
it converges on the actual root remarkably quickly.
Figure~\ref{drvtv:270:fig1} shows why: in the neighborhood, the curve
hardly departs from the straight line.

\index{square root!calculation of by Newton-Raphson}
\index{$n$th root!calculation of by Newton-Raphson}
The Newton-Raphson iteration is a champion square-root calculator,
incidentally.  Consider
\[
  f(x) = x^2 - p,
\]
whose roots are
\[
  x=\pm\sqrt p.
\]
Per~(\ref{drvtv:NR}), the Newton-Raphson iteration for this is
\bq{drvtv:270:30}
  x_{k+1} = \frac 1 2 \left[ x_k + \frac{p}{x_k} \right].
\eq
If you start by guessing
\[
  x_0=1
\]
and iterate several times, the iteration~(\ref{drvtv:270:30}) converges
on $x_\infty = \sqrt p$ fast.  To calculate the $n$th root $x=p^{1/n}$,
let
\[
  f(x) = x^n - p
\]
and iterate%
\footnote{
  Equations~(\ref{drvtv:270:30}) and~(\ref{drvtv:270:35}) work not only
  for real~$p$ but also usually for complex.  Given $x_0 = 1$, however,
  they converge reliably and orderly only for real, nonnegative~$p$.
  (To see why, sketch $f[x]$ in the fashion of
  Fig.~\ref{drvtv:270:fig1}.)

  If reliable, orderly convergence is
  needed for complex $p = u + iv = \sigma \cis \psi$, $\sigma \ge 0$,
  you can decompose $p^{1/n}$ per de~Moivre's
  theorem~(\ref{trig:280:24}) as $p^{1/n} = \sigma^{1/n} \cis(\psi/n)$,
  in which $\cis(\psi/n) = \cos(\psi/n) + i\sin(\psi/n)$ is calculated
  by the Taylor series of Table~\ref{taylor:315:tbl}.  Then~$\sigma$ is
  real and nonnegative, upon which~(\ref{drvtv:270:35}) reliably,
  orderly computes~$\sigma^{1/n}$.

  The Newton-Raphson iteration however excels as a \emph{practical}
  root-finding technique, so it often pays to be a little less
  theoretically rigid in applying it.  If so, then don't bother to
  decompose; seek $p^{1/n}$ directly, using complex~$z_k$ in place of
  the real~$x_k$.  In the uncommon event that the direct iteration does
  not seem to converge, start over again with some randomly chosen
  complex $z_0$.  This saves effort and usually works.
}$\mbox{}^,$\footnote{\cite[\S~4-9]{Shenk}\cite[\S~6.1.1]{Nayfeh/Bal}\cite{EWW}}
\bq{drvtv:270:35}
  x_{k+1} = \frac 1 n \left[ (n-1)x_k + \frac{p}{x_k^{n-1}} \right].
\eq

Section~\ref{mtxinv:420} generalizes the Newton-Raphson iteration to
handle vector-valued functions.

% ----------------------------------------------------------------------

This concludes the chapter.  Chapter~\ref{taylor}, treating the Taylor
series, will continue the general discussion of the derivative.

