<a name="Intro"></a>
<h1>Introduction</h1>

This tutorial program attempts to show how to use $hp$ finite element methods
with deal.II. It solves the Laplace equation and so builds only on the first
few tutorial programs, in particular on step-4 for dimension
independent programming and step-6 for adaptive mesh
refinement.

The $hp$ finite element method was proposed in the early 1980s by
Babuska and Guo as an alternative to either
(i) mesh refinement (i.e. decreasing the mesh parameter $h$ in a finite
element computation) or (ii) increasing the polynomial degree $p$ used for
shape functions. It is based on the observation that increasing the polynomial
degree of the shape functions reduces the approximation error if the solution
is sufficiently smooth.  On the other hand, it is well known
that even for the generally well-behaved class of elliptic problems, higher
degrees of regularity can not be guaranteed in the vicinity of boundaries,
corners, or where coefficients are discontinuous; consequently, the
approximation can not be improved in these areas by increasing the polynomial
degree $p$ but only by refining the mesh, i.e. by reducing the mesh size
$h$. These differing means to reduce the
error have led to the notion of $hp$ finite elements, where the approximating
finite element spaces are adapted to have a high polynomial degree $p$
wherever the solution is sufficiently smooth, while the mesh width $h$ is
reduced at places wherever the solution lacks regularity. It was
already realized in the first papers on this method that $hp$ finite elements
can be a powerful tool that can guarantee that the error is reduced not only
with some negative power of the number of degrees of freedom, but in fact
exponentially.

In order to implement this method, we need several things above and beyond
what a usual finite element program needs, and in particular above what we
have introduced in the tutorial programs leading up to step-6. In particular,
we will have to discuss the following aspects:
<ul>
  <li>Instead of using the same finite element on all cells, we now will want
  a collection of finite element objects, and associate each cell with one
  of these objects in this collection.

  <li>Degrees of freedom will then have to be allocated on each cell depending
  on what finite element is associated with this particular cell. Constraints
  will have to generated in the same way as for hanging nodes, but now also
  including the case where two neighboring cells.

  <li>We will need to be able to assemble cell and face contributions
  to global matrices and right hand side vectors.

  <li>After solving the resulting linear system, we will want to
  analyze the solution. In particular, we will want to compute error
  indicators that tell us whether a given cell should be refined
  and/or whether the polynomial degree of the shape functions used on
  it should be increased.
</ul>

We will discuss all these aspects in the following subsections of this
introduction. It will not come as a big surprise that most of these
tasks are already well supported by functionality provided by the
deal.II libraries, and that we will only have to provide the logic of
what the program should do, not exactly how all this is going to
happen.

In deal.II, the $hp$ functionality is largely packaged into
the hp namespace. This namespace provides classes that handle
$hp$ discretizations, assembling matrices and vectors, and other
tasks. We will get to know many of them further down below. In
addition, many of the functions in the DoFTools, and VectorTools
namespaces accept $hp$ objects in addition to the non-$hp$ ones. Much of
the $hp$ implementation is also discussed in the @ref hp documentation
module and the links found there.

It may be worth giving a slightly larger perspective at the end of
this first part of the introduction. $hp$ functionality has been
implemented in a number of different finite element packages (see, for
example, the list of references cited in the @ref hp_paper "hp paper").
However, by and large, most of these packages have implemented it only
for the (i) the 2d case, and/or (ii) the discontinuous Galerkin
method. The latter is a significant simplification because
discontinuous finite elements by definition do not require continuity
across faces between cells and therefore do not require the special
treatment otherwise necessary whenever finite elements of different
polynomial degree meet at a common face. In contrast, deal.II
implements the most general case, i.e. it allows for continuous and
discontinuous elements in 1d, 2d, and 3d, and automatically handles
the resulting complexity. In particular, it handles computing the
constraints (similar to hanging node constraints) of elements of
different degree meeting at a face or edge. The many algorithmic and
data structure techniques necessary for this are described in the
@ref hp_paper "hp paper" for those interested in such detail.

We hope that providing such a general implementation will help explore
the potential of $hp$ methods further.


<h3>Finite element collections</h3>

Now on again to the details of how to use the $hp$ functionality in
deal.II. The first aspect we have to deal with is that now we do not
have only a single finite element any more that is used on all cells,
but a number of different elements that cells can choose to use. For
this, deal.II introduces the concept of a <i>finite element
collection</i>, implemented in the class hp::FECollection. In essence,
such a collection acts like an object of type
<code>std::vector@<FiniteElement@></code>, but with a few more bells
and whistles and a memory management better suited to the task at
hand. As we will later see, we will also use similar quadrature
collections, and &mdash; although we don't use them here &mdash; there
is also the concept of mapping collections. All of these classes are
described in the @ref hpcollection overview.

In this tutorial program, we will use continuous Lagrange elements of
orders 2 through 7 (in 2d) or 2 through 5 (in 3d). The collection of
used elements can then be created as follows:
@code
  hp::FECollection<dim> fe_collection;
  for (unsigned int degree=2; degree<=max_degree; ++degree)
    fe_collection.push_back (FE_Q<dim>(degree));
@endcode



<h3>The hp::DoFHandler class, associating cells with finite elements, and constraints</h3>

The next task we have to consider is what to do with the list of
finite element objects we want to use. In previous tutorial programs,
starting with step-2, we have seen that the DoFHandler
class is responsible for making the connection between a mesh
(described by a Triangulation object) and a finite element, by
allocating the correct number of degrees of freedom for each vertex,
face, edge, and cell of the mesh.

The situation here is a bit more complicated since we do not just have
a single finite element object, but rather may want to use different
elements on different cells. We therefore need two things: (i) a
version of the DoFHandler class that can deal with this situation, and
(ii) a way to tell the DoF handler which element to use on which cell.

The first of these two things is implemented in the hp::DoFHandler
class: rather than associating it with a triangulation and a single
finite element object, it is associated with a triangulation and a
finite element collection. The second part is achieved by a loop over
all cells of this hp::DoFHandler and for each cell setting the index
of the finite element within the collection that shall be used on this
cell. We call the index of the finite element object within the
collection that shall be used on a cell the cell's <i>active FE
index</i> to indicate that this is the finite element that is active
on this cell, whereas all the other elements of the collection are
inactive on it. The general outline of this reads like this:

@code
  hp::DoFHandler<dim> dof_handler (triangulation);
  for (typename hp::DoFHandler<dim>::active_cell_iterator
         cell = dof_handler.begin_active();
       cell != dof_handler.end(); ++cell)
    cell->set_active_fe_index (...);
  dof_handler.distribute_dofs (fe_collection);
@endcode

Dots in the call to <code>set_active_fe_index()</code> indicate that
we will have to have some sort of strategy later on to decide which
element to use on which cell; we will come back to this later. The
main point here is that the first and last line of this code snippet
is pretty much exactly the same as for the non-$hp$ case.

Another complication arises from the fact that this time we do not
simply have hanging nodes from local mesh refinement, but we also have
to deal with the case that if there are two cells with different
active finite element indices meeting at a face (for example a Q2 and
a Q3 element) then we have to compute additional constraints on the
finite element field to ensure that it is continuous. This is
conceptually very similar to how we compute hanging node constraints,
and in fact the code looks exactly the same:
@code
  ConstraintMatrix constraints;
  DoFTools::make_hanging_node_constraints (dof_handler,
					   constraints);
@endcode
In other words, the DoFTools::make_hanging_node_constraints deals not
only with hanging node constraints, but also with $hp$ constraints at
the same time.



<h3>Assembling matrices and vectors with hp objects</h3>

Following this, we have to set up matrices and vectors for the linear system
of the correct size and assemble them. Setting them up works in exactly the
same way as for the non-$hp$ case. Assembling requires a bit more thought.

The main idea is of course unchanged: we have to loop over all cells, assemble
local contributions, and then copy them into the global objects. As discussed
in some detail first in step-3, deal.II has the FEValues class
that pulls finite element description, mapping, and quadrature formula
together and aids in evaluating values and gradients of shape functions as
well as other information on each of the quadrature points mapped to the real
location of a cell. Every time we move on to a new cell we re-initialize this
FEValues object, thereby asking it to re-compute that part of the information
that changes from cell to cell. It can then be used to sum up local
contributions to bilinear form and right hand side.

In the context of $hp$ finite element methods, we have to deal with the fact
that we do not use the same finite element object on each cell. In fact, we
should not even use the same quadrature object for all cells, but rather
higher order quadrature formulas for cells where we use higher order finite
elements. Similarly, we may want to use higher order mappings on such cells as
well.

To facilitate these considerations, deal.II has a class hp::FEValues that does
what we need in the current context. The difference is that instead of a
single finite element, quadrature formula, and mapping, it takes collections
of these objects. It's use is very much like the regular FEValues class,
i.e. the interesting part of the loop over all cells would look like this:

@code
  hp::FEValues<dim> hp_fe_values (mapping_collection,
                                  fe_collection,
				  quadrature_collection,
				  update_values    |  update_gradients |
				  update_q_points  |  update_JxW_values);

  typename hp::DoFHandler<dim>::active_cell_iterator
    cell = dof_handler.begin_active(),
    endc = dof_handler.end();
  for (; cell!=endc; ++cell)
    {
      hp_fe_values.reinit (cell,
                           cell->active_fe_index(),
                           cell->active_fe_index(),
                           cell->active_fe_index());

      const FEValues<dim> &fe_values = hp_fe_values.get_present_fe_values ();

      ...  // assemble local contributions and copy them into global object
    }
@endcode

In this tutorial program, we will always use a Q1 mapping, so the mapping
collection argument to the hp::FEValues construction will be omitted. Inside
the loop, we first initialize the hp::FEValues object for the current
cell. The second, third and fourth arguments denote the index within their
respective collections of the quadrature, mapping, and finite element objects
we wish to use on this cell. These arguments can be omitted (and are in the
program below), in which case <code>cell-@>active_fe_index()</code> is used
for this index. The order of these arguments is chosen in this way because one
may sometimes want to pick a different quadrature or mapping object from their
respective collections, but hardly ever a different finite element than the
one in use on this cell, i.e. one with an index different from
<code>cell-@>active_fe_index()</code>. The finite element collection index is
therefore the last default argument so that it can be conveniently omitted.

What this <code>reinit</code> call does is the following: the
hp::FEValues class checks whether it has previously already allocated a
non-$hp$ FEValues object for this combination of finite element, quadrature,
and mapping objects. If not, it allocates one. It then re-initializes this
object for the current cell, after which there is now a FEValues object for
the selected finite element, quadrature and mapping usable on the current
cell. A reference to this object is then obtained using the call
<code>hp_fe_values.get_present_fe_values()</code>, and will be used in the
usual fashion to assemble local contributions.



<h3>A simple indicator for hp refinement and estimating smoothness</h3>

One of the central pieces of the adaptive finite element method is that we
inspect the computed solution (a posteriori) with an indicator that tells us
which are the cells where the error is largest, and then refine them. In many
of the other tutorial programs, we use the KellyErrorEstimator class to get an
indication of the size of the error on a cell, although we also discuss more
complicated strategies in some programs, most importantly in step-14.

In any case, as long as the decision is only "refine this cell" or "do not
refine this cell", the actual refinement step is not particularly
challenging. However, here we have a code that is capable of hp refinement,
i.e. we suddenly have two choices whenever we detect that the error on a
certain cell is too large for our liking: we can refine the cell by splitting
it into several smaller ones, or we can increase the polynomial degree of the
shape functions used on it. How do we know which is the more promising
strategy? Answering this question is the central problem in $hp$ finite
element research at the time of this writing.

In short, the question does not appear to be settled in the literature at this
time. There are a number of more or less complicated schemes that address it,
but there is nothing like the KellyErrorEstimator that is universally accepted
as a good, even if not optimal, indicator of the error. Most proposals use the
fact that it is beneficial to increase the polynomial degree whenever the
solution is locally smooth whereas it is better to refine the mesh wherever it
is rough. However, the questions of how to determine the local smoothness of
the solution as well as the decision when a solution is smooth enough to allow
for an increase in $p$ are certainly big and important ones.

In the following, we propose a simple estimator of the local smoothness of a
solution. As we will see in the results section, this estimator has flaws, in
particular as far as cells with local hanging nodes are concerned. We
therefore do not intend to present the following ideas as a complete solution
to the problem. Rather, it is intended as an idea to approach it that merits
further research and investigation. In other words, we do not intend to enter
a sophisticated proposal into the fray about answers to the general
question. However, to demonstrate our approach to $hp$ finite elements, we
need a simple indicator that does generate some useful information that is
able to drive the simple calculations this tutorial program will perform.


<h4>The idea</h4>

Our approach here is simple: for a function $u({\bf x})$ to be in the
Sobolev space $H^s(K)$ on a cell $K$, it has to satisfy the condition
@f[
	\int_K |\nabla^s u({\bf x})|^2 \; d{\bf x} < \infty.
@f]
Assuming that the cell $K$ is not degenerate, i.e. that the mapping from the
unit cell to cell $K$ is sufficiently regular, above condition is of course
equivalent to
@f[
	\int_{\hat K} |\nabla^s \hat u(\hat{\bf x})|^2 \; d\hat{\bf x} < \infty
@f]
where $\hat u(\hat{\bf x})$ is the function $u({\bf x})$ mapped back onto the unit cell
$\hat K$. From here, we can do the following: first, let us define the
Fourier series of $\hat u$ as
@f[
	\hat U_{\bf k}
	= \frac 1{(2\pi)^{d/2}} \int_{\hat K} e^{i {\bf k}\cdot \hat{\bf x}} \hat u(\hat{\bf x}) d\hat{\bf x}
@f]
with Fourier vectors ${\bf k}=(k_x,k_y)$ in 2d, ${\bf k}=(k_x,k_y,k_z)$
in 3d, etc, and $k_x,k_y,k_z=0,\pi,2\pi,3\pi,\ldots$. If we re-compose $\hat u$
from $\hat U$ using the formula
@f[
	\hat u(\hat{\bf x})
	= \frac 1{(2\pi)^{d/2}} \sum_{\bf k} e^{-i {\bf k}\cdot \hat{\bf x}} \hat U_{\bf k},
@f]
then it becomes clear that we can write the $H^s$ norm of $\hat u$ as
@f[
	\int_{\hat K} |\nabla^s \hat u(\hat{\bf x})|^2 \; d\hat{\bf x}
	=
	\frac 1{(2\pi)^d}
	\int_{\hat K}
	\left|
	  \sum_{\bf k} |{\bf k}|^s e^{-i{\bf k}\cdot \hat{\bf x}} \hat U_{\bf k}
        \right|^2 \; d\hat{\bf x}
	=
	\sum_{\bf k}
	  |{\bf k}|^{2s}
	  |\hat U_{\bf k}|^2.
@f]
In other words, if this norm is to be finite (i.e. for $\hat u(\hat{\bf x})$ to be in $H^s(\hat K)$), we need that
@f[
	|\hat U_{\bf k}| = {\cal O}\left(|{\bf k}|^{-\left(s+1/2+\frac{d-1}{2}+\epsilon\right)}\right).
@f]
Put differently: the higher regularity $s$ we want, the faster the
Fourier coefficients have to go to zero. (If you wonder where the
additional exponent $\frac{d-1}2$ comes from: we would like to make
use of the fact that $\sum_l a_l < \infty$ if the sequence $a_l =
{\cal O}(l^{-1-\epsilon})$ for any $\epsilon>0$. The problem is that we
here have a summation not only over a single variable, but over all
the integer multiples of $\pi$ that are located inside the
$d$-dimensional sphere, because we have vector components $k_x, k_y,
\ldots$. In the same way as we prove that the sequence $a_l$ above
converges by replacing the sum by an integral over the entire line, we
can replace our $d$-dimensional sum by an integral over
$d$-dimensional space. Now we have to note that between distance $|{\bf k}|$
and $|{\bf k}|+d|{\bf k}|$, there are, up to a constant, $|{\bf k}|^{d-1}$ modes, in
much the same way as we can transform the volume element $dx\;dy$ into
$2\pi r\; dr$. Consequently, it is no longer $|{\bf k}|^{2s}|\hat
U_{\bf k}|^2$ that has to decay as ${\cal O}(|{\bf k}|^{-1-\epsilon})$, but
it is in fact $|{\bf k}|^{2s}|\hat U_{\bf k}|^2 |{\bf k}|^{d-1}$. A
comparison of exponents yields the result.)

We can turn this around: Assume we are given a function $\hat u$ of unknown
smoothness. Let us compute its Fourier coefficients $\hat U_{\bf k}$
and see how fast they decay. If they decay as
@f[
	|\hat U_{\bf k}| = {\cal O}(|{\bf k}|^{-\mu-\epsilon}),
@f]
then consequently the function we had here was in $H^{\mu-d/2}$.


<h4>What we have to do</h4>

So what do we have to do to estimate the local smoothness of $u({\bf x})$ on
a cell $K$? Clearly, the first step is to compute the Fourier series
of our solution. Fourier series being infinite series, we simplify our
task by only computing the first few terms of the series, such that
$|{\bf k}|\le N$ with a cut-off $N$. (Let us parenthetically remark
that we want to choose $N$ large enough so that we capture at least
the variation of those shape functions that vary the most. On the
other hand, we should not choose $N$ too large: clearly, a finite
element function, being a polynomial, is in $C^\infty$ on any given
cell, so the coefficients will have to decay exponentially at one
point; since we want to estimate the smoothness of the function this
polynomial approximates, not of the polynomial itself, we need to
choose a reasonable cutoff for $N$.) Either way, computing this series
is not particularly hard: from the definition
@f[
	\hat U_{\bf k}
	= \frac 1{(2\pi)^{d/2}} \int_{\hat K} e^{i {\bf k}\cdot \hat{\bf x}} \hat u(\hat{\bf x}) d\hat{\bf x}
@f]
we see that we can compute the coefficient $\hat U_{\bf k}$ as
@f[
	\hat U_{\bf k}
	= \frac 1{(2\pi)^{d/2}}
          \sum_{i=0}^{\textrm{\tiny dofs per cell}}
          \left[\int_{\hat K} e^{i {\bf k}\cdot \hat{\bf x}} \hat \varphi_i(\hat{\bf x})
	  d\hat{\bf x} \right] u_i,
@f]
where $u_i$ is the value of the $i$th degree of freedom on this
cell. In other words, we can write it as a matrix-vector product
@f[
	\hat U_{\bf k}
	= {\cal F}_{{\bf k},j} u_j,
@f]
with the matrix
@f[
	{\cal F}_{{\bf k},j}
	= \frac 1{(2\pi)^{d/2}}
	\int_{\hat K} e^{i {\bf k}\cdot \hat{\bf x}} \hat \varphi_j(\hat{\bf x}) d\hat{\bf x}.
@f]
This matrix is easily computed for a given number of shape functions
$\varphi_j$ and Fourier modes $N$. Consequently, finding the
coefficients $\hat U_{\bf k}$ is a rather trivial job.

The next task is that we have to estimate how fast these coefficients
decay with $|{\bf k}|$. The problem is that, of course, we have only
finitely many of these coefficients in the first place. In other
words, the best we can do is to fit a function $\alpha |{\bf k}|^{-\mu}$
to our data points $\hat U_{\bf k}$, for example by
determining $\alpha,\mu$ via a least-squares procedure:
@f[
	\min_{\alpha,\mu}
	\frac 12 \sum_{{\bf k}, |{\bf k}|\le N}
	\left( |\hat U_{\bf k}| - \alpha |{\bf k}|^{-\mu}\right)^2
@f]
However, the problem with this is that it leads to a nonlinear
problem, a fact that we would like to avoid. On the other hand, we can
transform the problem into a simpler one if we try to fit the
logarithm of our coefficients to the logarithm of $\alpha |{\bf k}|^{-\mu}$,
like this:
@f[
	\min_{\alpha,\mu}
	Q(\alpha,\mu) =
	\frac 12 \sum_{{\bf k}, |{\bf k}|\le N}
	\left( \ln |\hat U_{\bf k}| - \ln (\alpha |{\bf k}|^{-\mu})\right)^2.
@f]
Using the usual facts about logarithms, we see that this yields the
problem
@f[
	\min_{\beta,\mu}
	Q(\beta,\mu) =
	\frac 12 \sum_{{\bf k}, |{\bf k}|\le N}
	\left( \ln |\hat U_{\bf k}| - \beta + \mu \ln |{\bf k}|\right)^2,
@f]
where $\beta=\ln \alpha$. This is now a problem for which the
optimality conditions $\frac{\partial Q}{\partial\beta}=0,
\frac{\partial Q}{\partial\mu}=0$, are linear in $\beta,\mu$. We can
write these conditions as follows:
@f[
	\left(\begin{array}{cc}
	\sum_{{\bf k}, |{\bf k}|\le N} 1 &
	\sum_{{\bf k}, |{\bf k}|\le N} \ln |{\bf k}|
	\\
	\sum_{{\bf k}, |{\bf k}|\le N} \ln |{\bf k}| &
	\sum_{{\bf k}, |{\bf k}|\le N} (\ln |{\bf k}|)^2
	\end{array}\right)
	\left(\begin{array}{c}
	\beta \\ -\mu
	\end{array}\right)
	=
	\left(\begin{array}{c}
	\sum_{{\bf k}, |{\bf k}|\le N} \ln |\hat U_{{\bf k}}|
	\\
	\sum_{{\bf k}, |{\bf k}|\le N} \ln |\hat U_{{\bf k}}| \ln |{\bf k}|
	\end{array}\right)
@f]
This linear system is readily inverted to yield
@f[
	\beta =
	\frac 1{\left(\sum_{{\bf k}, |{\bf k}|\le N} 1\right)
                \left(\sum_{{\bf k}, |{\bf k}|\le N} (\ln |{\bf k}|)^2\right)
		-\left(\sum_{{\bf k}, |{\bf k}|\le N} \ln |{\bf k}|\right)^2}
	\left[
	  \left(\sum_{{\bf k}, |{\bf k}|\le N} (\ln |{\bf k}|)^2\right)
	  \left(\sum_{{\bf k}, |{\bf k}|\le N} \ln |\hat U_{{\bf k}}|\right)
	  -
	  \left(\sum_{{\bf k}, |{\bf k}|\le N} \ln |{\bf k}|\right)
	  \left(\sum_{{\bf k}, |{\bf k}|\le N} \ln |\hat U_{{\bf k}}| \ln |{\bf k}| \right)
	\right]
@f]
and
@f[
	\mu =
	\frac 1{\left(\sum_{{\bf k}, |{\bf k}|\le N} 1\right)
                \left(\sum_{{\bf k}, |{\bf k}|\le N} (\ln |{\bf k}|)^2\right)
		-\left(\sum_{{\bf k}, |{\bf k}|\le N} \ln |{\bf k}|\right)^2}
	\left[
	  \left(\sum_{{\bf k}, |{\bf k}|\le N} \ln |{\bf k}|\right)
	  \left(\sum_{{\bf k}, |{\bf k}|\le N} \ln |\hat U_{{\bf k}}|\right)
	  -
	  \left(\sum_{{\bf k}, |{\bf k}|\le N} 1\right)
	  \left(\sum_{{\bf k}, |{\bf k}|\le N} \ln |\hat U_{{\bf k}}| \ln |{\bf k}| \right)
	\right].
@f]

While we are not particularly interested in the actual value of
$\beta$, the formula above gives us a mean to calculate the value of
the exponent $\mu$ that we can then use to determine that
$\hat u(\hat{\bf x})$ is in $H^s(\hat K)$ with $s=\mu-\frac d2$.


<h4>Compensating for anisotropy</h4>

In the formulas above, we have derived the Fourier coefficients $\hat U_{\vec
k}$. Because ${\bf k}$ is a vector, we will get a number of Fourier
coefficients $\hat U_{{\bf k}}$ for the same absolute value $|{\bf k}|$,
corresponding to the Fourier transform in different directions. If we now
consider a function like $|x|y^2$ then we will find lots of large Fourier
coefficients in $x$-direction because the function is non-smooth in this
direction, but fast-decaying Fourier coefficients in $y$-direction because the
function is smooth there. The question that arises is this: if we simply fit
our polynomial decay $\alpha |{\bf k}|^\mu$ to <i>all</i> Fourier coefficients,
we will fit it to a smoothness <i>averaged in all spatial directions</i>. Is
this what we want? Or would it be better to only consider the largest
coefficient $\hat U_{{\bf k}}$ for all ${\bf k}$ with the same magnitude,
essentially trying to determine the smoothness of the solution in that spatial
direction in which the solution appears to be roughest?

One can probably argue for either case. The issue would be of more interest if
deal.II had the ability to use anisotropic finite elements, i.e. ones that use
different polynomial degrees in different spatial directions, as they would be
able to exploit the directionally variable smoothness much better. Alas, this
capability does not exist at the time of writing this tutorial program.

Either way, because we only have isotopic finite element classes, we adopt the
viewpoint that we should tailor the polynomial degree to the lowest amount of
regularity, in order to keep numerical efforts low. Consequently, instead of
using the formula
@f[
	\mu =
	\frac 1{\left(\sum_{{\bf k}, |{\bf k}|\le N} 1\right)
                \left(\sum_{{\bf k}, |{\bf k}|\le N} (\ln |{\bf k}|)^2\right)
		-\left(\sum_{{\bf k}, |{\bf k}|\le N} \ln |{\bf k}|\right)^2}
	\left[
	  \left(\sum_{{\bf k}, |{\bf k}|\le N} \ln |{\bf k}|\right)
	  \left(\sum_{{\bf k}, |{\bf k}|\le N} \ln |\hat U_{{\bf k}}|\right)
	  -
	  \left(\sum_{{\bf k}, |{\bf k}|\le N} 1\right)
	  \left(\sum_{{\bf k}, |{\bf k}|\le N} \ln |\hat U_{{\bf k}}| \ln |{\bf k}| \right)
	\right].
@f]
to calculate $\mu$ as shown above, we have to slightly modify all sums:
instead of summing over all Fourier modes, we only sum over those for which
the Fourier coefficient is the largest one among all $\hat U_{{\bf k}}$ with
the same magnitude $|{\bf k}|$, i.e. all sums above have to replaced by the
following sums:
@f[
  \sum_{{\bf k}, |{\bf k}|\le N}
  \longrightarrow
  \sum_{\begin{matrix}{{\bf k}, |{\bf k}|\le N} \\ {|\hat U_{{\bf k}}| \ge |\hat U_{{\bf k}'}|
  \ \textrm{for all}\ {\bf k}'\ \textrm{with}\ |{\bf k}'|=|{\bf k}|}\end{matrix}}
@f]
This is the form we will implement in the program.


<h4>Questions about cell sizes</h4>

One may ask whether it is a problem that we only compute the Fourier transform
on the <i>reference cell</i> (rather than the real cell) of the
solution. After all, we stretch the solution by a factor $\frac 1h$ during the
transformation, thereby shifting the Fourier frequencies by a factor of
$h$. This is of particular concern since we may have neighboring cells with
mesh sizes $h$ that differ by a factor of 2 if one of them is more refined
than the other. The concern is also motivated by the fact that, as we will see
in the results section below, the estimated smoothness of the solution should
be a more or less continuous function, but exhibits jumps at locations where
the mesh size jumps. It therefore seems natural to ask whether we have to
compensate for the transformation.

The short answer is "no". In the process outlined above, we attempt to find
coefficients $\beta,\mu$ that minimize the sum of squares of the terms
@f[
	\ln |\hat U_{{\bf k}}| - \beta + \mu \ln |{\bf k}|.
@f]
To compensate for the transformation means not attempting to fit a decay
$|{\bf k}|^\mu$ with respect to the Fourier frequencies ${\bf k}$ <i>on the unit
cell</i>, but to fit the coefficients $\hat U_{{\bf k}}$ computed on the
reference cell <i>to the Fourier frequencies on the real cell $|\vec
k|h$</i>, where $h$ is the norm of the transformation operator (i.e. something
like the diameter of the cell). In other words, we would have to minimize the
sum of squares of the terms
@f[
	\ln |\hat U_{{\bf k}}| - \beta + \mu \ln (|{\bf k}|h).
@f]
instead. However, using fundamental properties of the logarithm, this is
simply equivalent to minimizing
@f[
	\ln |\hat U_{{\bf k}}| - (\beta - \mu \ln h) + \mu \ln (|{\bf k}|).
@f]
In other words, this and the original least squares problem will produce the
same best-fit exponent $\mu$, though the offset will in one case be $\beta$
and in the other $\beta-\mu \ln h$. However, since we are not interested in
the offset at all but only in the exponent, it doesn't matter whether we scale
Fourier frequencies in order to account for mesh size effects or not, the
estimated smoothness exponent will be the same in either case.



<h3>Complications with linear systems for hp discretizations</h3>

<h4>Creating the sparsity pattern</h4>

One of the problems with $hp$ methods is that the high polynomial degree of
shape functions together with the large number of constrained degrees of
freedom leads to matrices with large numbers of nonzero entries in some
rows. At the same time, because there are areas where we use low polynomial
degree and consequently matrix rows with relatively few nonzero
entries. Consequently, allocating the sparsity pattern for these matrices is a
challenge.

Most programs built on deal.II use the DoFTools::make_sparsity_pattern
function to allocate the sparsity pattern of a matrix, and later add a few
more entries necessary to handle constrained degrees of freedom using
ConstraintMatrix::condense. The sparsity pattern is then compressed using
SparsityPattern::compress. This method is explained in step-6 and used in
most tutorial programs. In order to work, it needs an initial upper estimate
for the maximal number of nonzero entries per row, something that can be had
from the DoFHandler::max_couplings_between_dofs function. This is necessary
due to the data structure used in the SparsityPattern class.

Unfortunately, DoFHandler::max_couplings_between_dofs is unable to produce an
efficient upper estimate in 3d and for higher order elements. If used in these
situations, it therefore leads the SparsityPattern class to allocate much too
much memory, almost all of which will be released again when we call
SparsityPattern::compress. This deficiency, caused by the fact that
DoFHandler::max_couplings_between_dofs must produce a single number for the
maximal number of elements per row even though most rows will be significantly
shorter, can be so severe that the initial memory allocation for the
SparsityPattern exceeds the actual need by a factor of 10 or larger, and can
lead to a program running out of memory when in fact there would be plenty of
memory for all computations.

A solution to the problem has already been discussed in step-11
and step-18. It used an intermediate object of type
CompressedSparsityPattern. This class uses a different memory storage scheme
that is optimized to <i>creating</i> a sparsity pattern when maximal numbers
of entries per row are not accurately available, but is unsuitable for use as
the sparsity pattern actually underlying a sparse matrix. After building the
intermediate object, it is therefore copied into a true SparsityPattern
object, something that can be done very efficient and without having to
over-allocate memory. Typical code doing this is shown in the documentation of
the CompressedSparsityPattern class. This solution is slower than directly
building a SparsityPattern object, but only uses as much memory as is really
necessary.

As it now turns out, the storage format used in the
CompressedSparsityPattern class is not very good for matrices with
truly large numbers of entries per row &mdash; where truly large
numbers mean in the hundreds. This isn't typically the case for lower
order elements even in 3d, but happens for high order elements in 3d;
for example, a vertex degree of freedom of a $Q_5$ element in 3d may
couple to as many as 1700 other degrees of freedom. In such a case
CompressedSparsityPattern will work, but by tuning the memory storage
format used internally in that class a bit will make it work several
times faster. This is what we did with the
CompressedSetSparsityPattern class &mdash; it has exactly the same
interface as the CompressedSparsityPattern class but internally stores
things somewhat differently. For most cases, there is not much of a
difference in performance in the classes (though the old class has a
slight advantage for lower order elements in 3d), but for high order
and $hp$ elements in 3d, the CompressedSetSparsityPattern has a
definite edge. We will therefore use it later when we build the
sparsity pattern in this tutorial program.


<h4>Eliminating constrained degrees of freedom</h4>

A second problem particular to $hp$ methods arises because we have so
many constrained degrees of freedom: typically up to about one third
of all degrees of freedom (in 3d) are constrained because they either
belong to cells with hanging nodes or because they are on cells
adjacent to cells with a higher or lower polynomial degree. This is,
in fact, not much more than the fraction of constrained degrees of
freedom in non-$hp$ mode, but the difference is that each constrained
hanging node is constrained not only against the two adjacent degrees
of freedom, but is constrained against many more degrees of freedom.

It turns out that the strategy presented first in step-6 to eliminate the
constraints while computing the element matrices and vectors with
ConstraintMatrix::distribute_local_to_global is the most efficient approach
also for this case. The alternative strategy to first build the matrix without
constraints and then "condensing" away constrained degrees of freedom is
considerably more expensive. It turns out that building the sparsity pattern
by this inefficient algorithm requires at least ${\cal O}(N \log N)$ in the
number of unknowns, whereas an ideal finite element program would of course
only have algorithms that are linear in the number of unknowns. Timing the
sparsity pattern creation as well as the matrix assembly shows that the
algorithm presented in step-6 (and used in the code below) is indeed faster.

In our program, we will also treat the boundary conditions as (possibly
inhomogeneous) constraints and eliminate the matrix rows and columns to
those as well. All we have to do for this is to call the function that
interpolates the Dirichlet boundary conditions already in the setup phase in
order to tell the ConstraintMatrix object about them, and then do the
transfer from local to global data on matrix and vector simultaneously. This
is exactly what we've shown in step-6.


<h3>The test case</h3>

The test case we will solve with this program is a re-take of the one we
already look at in step-14: we solve the Laplace equation
@f[
	-\Delta u = f
@f]
in 2d, with $f=(x+1)(y+1)$, and with zero Dirichlet boundary values for
$u$. We do so on the domain $[-1,1]^2\backslash[-\frac 12,\frac 12]^2$, i.e. a
square with a square hole in the middle.

The difference to step-14 is of course that we use $hp$ finite
elements for the solution. The testcase is of interest because it has
re-entrant corners in the corners of the hole, at which the solution has
singularities. We therefore expect that the solution will be smooth in the
interior of the domain, and rough in the vicinity of the singularities. The
hope is that our refinement and smoothness indicators will be able to see this
behavior and refine the mesh close to the singularities, while the polynomial
degree is increased away from it. As we will see in the results section, this
is indeed the case.
