% language=us runpath=texruns:manuals/cld

\startcomponent cld-introduction

\environment cld-environment

\startchapter[title=Introduction]

Sometimes you hear folks complain about the \TEX\ input language, i.e.\ the
backslashed commands that determine your output. Of course, when alternatives are
being discussed everyone has a favorite programming language. In practice
coding a document in each of them triggers similar sentiments with regards to
coding as \TEX\ itself does. And in then, as the problem doesn't change, one ends
up with the same annoyances.

So, just for fun, I added a couple of commands to \CONTEXT\ \MKIV\ that permit
coding a document in \LUA. In retrospect it has been surprisingly easy to
implement a feature like this using metatables. Of course it's a bit slower than
using \TEX\ as input language but sometimes the \LUA\ interface is more readable
given the problem at hand.

After a while I decided to use that interface in non|-|critical core \CONTEXT\
code and in styles (modules) and solutions for projects. Using the \LUA\ approach
is sometimes more convenient, especially if the code mostly manipulates data. For
instance, if you process \XML\ files of database output you can use the interface
that is available at the \TEX\ end, or you can use \LUA\ code to do the work, or
you can use a combination. So, from that moment on, in \CONTEXT\ you can code
your style and document source in (a mixture of) \TEX, \XML, \METAPOST\ and \LUA.

Because \CONTEXT\ is a \TEX\ macro package quite some action happens at the \TEX\
end. Although we do a lot in \LUA, there is no need to abandon the macro
language. If I wanted a typesetting system written in procedural language I'd
already come up with one but I have no need for it and \TEX\ is more fun anyway.
But the fact that we mix these conceptual different languages means that we need
ways to communicate between them. The possibilities to communicate and switch
between \TEX, \LUA, and of course \METAPOST\ have evolved over time. It started
with just \type {\directlua} but we now can use tokens scanners that permit nice
interfaces. We have access to scanners in \TEX\ and \METAPOST\ and can pipe back
data to both in efficient ways. Actually, we seldom use that direct command, at
most we use an equivalent \type {\ctxlua}, which permits a future overload. As a
consequence the interfaces in \CONTEXT\ also evolved, although mostly deep down
out of sight of users: \CONTEXT\ always tries to offer abstract interfaces that
as designed in a way that permits upward compatible upgrades.

Going from \LUA\ to \TEX\ is mostly done with the \type {context} command. The
most low level approach would be to use the \type {tex.print} functions but using
the \type {context} interface is often better. Directly going from \TEX\ to \LUA\
is done with \type {\ctxlua} and friends. There is no need to use the low level
\quote {print} interfaces and it can even be counter-productive when you want
tracing. We keep the low-level (primitive) interfaces unblocked but best first
try what \CONTEXT\ provides on top of them.

In these internet times it is no problem to find comments on code and coding
practices (of course not seldom by people who themselves produce code that
deserves a lot of comment but get away with it). The same is true for complaints
about bugs that not always are put in the perspective of the total amount of well
working code or experiments; on the average \TEX\ systems behave quite well and
have not that many bugs, but some low level features they can be confusing. We
don't impose a coding style in user files but we do so in the core, but that only
hampers ourselves.

Another issue is complaints about manuals or lack of documentation, again, often
by people who themselves never produced something that those who they complain
about can use, but, that said, lack of documentation has the benefit that one can
learn by playing around. There are plenty of examples to learn from, also in the
over ten thousand pages of \CONTEXT\ documentation and articles. And of course
there is the test suite which has examples. Some users even like to look into the
source code to learn some tricks. Now, the internet is not always the friendly
and tolerant environment that its marketing suggests and one can run into
patronizing comments on how to do thing, but: we don't (and cannot) enforce best
practices.

However, when you define \LUA\ functions you should {\em not} overload existing
functions without knowing what you deal with, simply because the whole \CONTEXT\
ecosystem could be affected. Don't expect support when you do so. The same is
true for \TEX\ macros and even more so for primitives. Although there is some
protection going on we think the system should be as open as possible, so little
is hidden for the user. \footnote {In the core code and modules that come with
\CONTEXT\ we are rather rigorous. There we have patterns that need to be
followed. Hacking around and patching at will is not an option there. In the
rather open system that a macro package provides it is no problem to quickly mess
up the lot with bad code. There is a strong relationship between the coding and
way syntax highlighting evolved, as offered in the \SCITE\ module that comes
with \CONTEXT.}

The reason for mentioning this is that we do see users come up with (often
surprising) solutions and we invite them to keep playing around, even with low
level code. When writing a document, playing around can be a nice distraction.
But, using a high level interface has some benefits too. It gives a bit of
protection against bad interactions with other code. It also often involves less
coding. Performance might be better although that is not always a real issue as
there are plenty of ways to make a \TEX\ system slow anyway.

In the following chapters I will introduce typesetting in \LUA, but as we rely on
\CONTEXT\ it is unavoidable that some regular \CONTEXT\ code shows up. The fact
that you can ignore backslashes does not mean that you can do without knowledge
of the underlying system. I expect that the user is somewhat familiar with this
macro package. Some chapters are follow ups on articles or earlier publications.

Some information (and mechanism) show up in more than one chapter. This is a side
effect of \LUA\ being integrated in many places, so an isolated discussion is a
bit hard.

In the meantime most of the code is rather stable and proven. However, this
manual will never be complete. You can find examples all over the code base, and
duplicating everything here makes no sense. If you find errors, please let me
know. If you think that something is missing, you can try to convince me to add
it. It's hard to keep up with what gets added so input is welcome.

I started writing this manual in the early days if \MKIV\ using \LUATEX\ but in
the meantime we've arrived at \MKXL\ using \LUAMETATEX. This means that there can
be features discussed here that are only available in the latest greatest. In that
perspective it is important to notice that we no longer expect \LUAJITTEX\ to be used,
and assume \LUA\ 5.5 or later, which comes with some additional functionality and
parsing rules.

\blank[2*big]

\startlines
Hans Hagen
Hasselt NL
2009 \emdash\ 2025\high{+}
\stoplines

\stopchapter

\stopcomponent
