% language=us runpath=texruns:manuals/luametatex

\environment luametatex-style

\startdocument[title=PDF]

\startsection[title={Introduction}]

There is no backend, not even a \DVI\ one. In \CONTEXT\ the main backend is a
\PDF\ backend and it is written in \LUA. The \PDF\ format makes it possible to
embed \JPEG\ and \PNG\ encoded images as well as \PDF\ images. All these have to
be dealt with in \LUA. Although we can parse \PDF\ files with \LUA, the engine
has a dedicated \PDF\ library on board written by Paweł Jackowski.

A \PDF\ file is basically a tree of objects and one descends into the tree via
dictionaries (key|/|value) and arrays (index|/|value). There are a few topmost
dictionaries that start at the document root and those are accessed more
directly.

Although everything in \PDF\ is basically an object we have to wrap a few in so
called userdata \LUA\ objects.

\starttabulate[|l|l|]
\FL
\BC PDF        \BC \LUA         \NC \NR
\TL
\NC null       \NC <t:nil>      \NC \NR
\NC boolean    \NC <t:boolean>  \NC \NR
\NC integer    \NC <t:integer>  \NC \NR
\NC float      \NC <t:number>   \NC \NR
\NC name       \NC <t:string>   \NC \NR
\NC string     \NC <t:string>   \NC \NR
\NC array      \NC <t:userdata> \NC \NR
\NC dictionary \NC <t:userdata> \NC \NR
\NC stream     \NC <t:userdata> \NC \NR
\NC reference  \NC <t:userdata> \NC \NR
\LL
\stoptabulate

The interface is rather limited to creating an instance and getting objects and
values. Aspects like compression and encryption are mostly dealt with
automatically. In \CONTEXT\ users use an interface layer around these, if they
use this kind of low level code at all as it assumes familiarity with how \PDF\
is constructed.

\stopsection

\startsection[title={\LUA\ interfaces}]

\startsubsection[title={Opening and closing}]

There are two ways to open a \PDF\ file:

\starttyping[option=LUA]
function pdfe.open ( <t:string> filename )
    return <t:pdf>  -- pdffile
end

function pdfe.openfile( <t:file> filehandle )
    return <t:pdf> -- pdffile
end
\stoptyping

Instead of from file, we can read from a string:

\starttyping[option=LUA]
function pdfe.new ( <t:string>  somestring, <t:integer> somelength )
    return <t:pdf> -- pdffile
end
\stoptyping

Closing the instance is done with:

\starttyping[option=LUA]
function pdfe.close ( <t:pdf> pdffile )
    -- no return values
end
\stoptyping

When we used \type {pdfe.open} the library manages the file and closes it when
done. You can check if a document opened as expected by calling:

\starttyping[option=LUA]
function pdfe.getstatus ( <t:pdf> pdffile )
    return <t:integer> -- status
end
\stoptyping

A table of possible return codes can be queried with:

\starttyping[option=LUA]
function pdfe.getstatusvalues ( )
    return <t:table> -- values
end
\stoptyping

Currently we have these:

\ctxlua{moduledata.pdfe.codes("getstatusvalues")}

An encrypted document can be decrypted by the next command where instead of
either password you can give \type {nil} and hope for the best:

\starttyping[option=LUA]
function pdfe.unencrypt (
    <t:pdf>    pdffile,
    <t:string> userpassword,
    <t:string> ownerpassword
)
    return <t:integer> -- status
end
\stoptyping

\stopsubsection

\startsubsection[title=Getting basic information]

A successfully opened document can provide some information:

\starttyping[option=LUA]
function pdfe.getsize( <t:pdf> pdffile )
    return <t:integer> -- nofbytes
end

function pdfe.getversion( <t:pdf> pdffile )
    return
        <t:integer>, -- major
        <t:integer>  -- minor
end

function pdfe.getnofobjects ( <t:pdf> pdffile )
    return <t:integer> -- nofobjects
end

function pdfe.getnofpages ( <t:pdf> pdffile )
    return <t:integer> -- nofpages
end

function pdfe.memoryusage ( <t:pdf> pdffile )
    return
        <t:integer>, -- bytes
        <t:integer>  -- waste
end
\stoptyping

\stopsubsection

\startsubsection[title={The main structure}]

For accessing the document structure you start with the so called catalog, a
dictionary:

\starttyping[option=LUA]
function pdfe.getcatalog( <t:pdf> pdffile )
    return <t:userdata> -- dictionary
end
\stoptyping

The other two root dictionaries are accessed with:

\starttyping[option=LUA]
function pdfe.gettrailer ( <t:pdf> pdffile )
    return <t:userdata> -- dictionary
end

function pdfe.getinfo ( <t:pdf> pdffile )
    return <t:userdata> -- dictionary
end
\stoptyping

\stopsubsection

\startsubsection[title={Getting content}]

A specific page can conveniently be reached with the next command, which returns
a dictionary.

\starttyping[option=LUA]
function pdfe.getpage ( <t:pdf> pdffile, <t:integer> pagenumber )
    return <t:userdata> -- dictionary
end
\stoptyping

Another convenience command gives you the (bounding) box of a (normally page)
which can be inherited from the document itself. An example of a valid box name
is \type {MediaBox}.

\starttyping[option=LUA]
function pdfe.getbox ( <t:pdf> pdffile, <t:string> boxname )
    return <t:table> -- boundingbox
end
\stoptyping

\stopsubsection

\startsubsection[title={Getters}]

Common values in dictionaries and arrays are strings, integers, floats, booleans
and names (which are also strings) and these are also normal \LUA\ objects. In
some cases a value is a userdata object and you can use this helper to get some
more information:

\starttyping[option=LUA]
function pdfe.type ( <t:whatever> value )
    return type -- string
end
\stoptyping

Stings are special because internally they are delimited by parenthesis (often
\typ {pdfdoc} encoding) or angle brackets (hexadecimal or 16 bit \UNICODE).

\starttyping[option=LUA]
function pdfe.getstring (
    <t:userdata> object,
    <t:string>   key | <t:integer> index
)
    return
        <t:string>  -- decoded value
end
\stoptyping

When you ask for more you get more:

\starttyping[option=LUA]
function pdfe.getstring (
    <t:userdata> object,
    <t:string>   key | <t:integer> index,
    <t:boolean>  more
)
    return
        <t:string>, -- original
        <t:boolean>  -- hexencoded
end
\stoptyping

Basic types are fetched with:

\starttyping[option=LUA]
function pdfe.getinteger ( <t:userdata>, <t:string> key | <t:integer> index )
    return <t:integer> -- value
end

function pdfe.getnumber ( <t:userdata>, <t:string> key | <t:integer> index )
    return <t:number> -- value
end

function pdfe.getboolean ( <t:userdata>, <t:string> key | <t:integer> index )
    return <t:boolean> -- value
end
\stoptyping

A name is (in the \PDF\ file) a string prefixed by a slash, like \typ
[option=PDF] {<< /Type /Foo >>}, for instance keys in a dictionary or keywords in
an array or constant values.

\starttyping[option=LUA]
function pdfe.getname ( <t:userdata>, <t:string> key | <t:integer> index )
    return <t:string> -- value
end
\stoptyping

Normally you will use an index in an array and key in a dictionary but dictionaries
also accept an index. The size of an array or dictionary is available with the
usual \type {#} operator.

\starttyping[option=LUA]
function pdfe.getdictionary ( <t:userdata>, <t:string> key | <t:integer> index )
    return <t:userdata> -- dictionary
end

function pdfe.getarray ( <t:userdata>, <t:string> key | <t:integer> index )
    return <t:userdata> -- array
end

function pdfe.getstream ( <t:userdata>, <t:string> key | <t:integer> index )
    return
        <t:userdata> -- stream
        <t:userdata> -- dictionary
end
\stoptyping

These commands return dictionaries, arrays and streams, which are dictionaries
with a blob of data attached.

Before we come to an alternative access mode, we mention that the objects provide
access in a different way too, for instance this is valid:

\starttyping[option=LUA]
print(pdfe.open("foo.pdf").Catalog.Type)
\stoptyping

At the topmost level there are \type {Catalog}, \type {Info}, \type {Trailer}
and \type {Pages}, so this is also okay:

\starttyping[option=LUA]
print(pdfe.open("foo.pdf").Pages[1])
\stoptyping

\stopsubsection

\startsubsection[title={Streams}]

Streams are sort of special. When your index or key hits a stream you get back a
stream object and dictionary object. The dictionary you can access in the usual
way and for the stream there are the following methods:

\starttyping[option=LUA]
function pdfe.openstream ( <t:userdata> stream, <t:boolean> decode)
    return <t:boolean> okay
end

function pdfe.closestream ( <t:userdata> stream )
    -- no return values
end

function pdfe.readfromstream ( <t:userdata> stream )
    return
        <t:string>  str,
        <t:integer> size
end

function pdfe.readwholestream ( <t:userdata> stream, <t:boolean> decode)
    return
        <t:string>  str,
        <t:integer> size
end
\stoptyping

You either read in chunks, or you ask for the whole. When reading in chunks, you
need to open and close the stream yourself. The \type {decode} parameter controls
if the stream data gets uncompressed.

As with dictionaries, you can access fields in a stream dictionary in the usual
\LUA\ way too. You get the content when you \quote {call} the stream. You can
pass a boolean that indicates if the stream has to be decompressed.

\stopsubsection

\startsubsection[title={Low level getters}]

In addition to the getters described before, there is also a bit lower level
interface available.

\starttyping[option=LUA]
function pdfe.getfromdictionary ( <t:userdata>, <t:integer> index )
    return
        <t:string>   key,
        <t:string>   type,
        <t:whatever> value,
        <t:whatever> detail
end

function pdfe.getfromarray ( <t:userdata>, <t:integer> index )
    return
        <t:integer>  type,
        <t:whatever> value,
        <t:integerr> detail
end
\stoptyping

The \type {type} is one of the following:

\startfourrows
\ctxlua{moduledata.pdfe.codes("getfieldtypes")}
\stopfourrows

This list was acquired with:

\starttyping[option=LUA]
function pdfe.getfieldtypes ( )
    return <t:table> -- types
end
\stoptyping

Here \type {detail} is a bitset with possible bits:

\startfourrows
\ctxlua{moduledata.pdfe.codes("getencodingvalues")}
\stopfourrows

This time we used:

\starttyping[option=LUA]
function pdfe.getencodingvalues ( )
    return <t:table> -- values
end
\stoptyping

\stopsubsection

\startsubsection[title={Getting tables}]

All entries in a dictionary or table can be fetched with the following commands
where the return values are a hashed or indexed table.

\starttyping[option=LUA]
function pdfe.dictionarytotable ( <t:userdata> )
    return <t:table> -- hash
end

function pdfe.arraytotable ( <t:userdata> )
    return <t:table> -- array
end
\stoptyping

You can get a list of pages with:

\starttyping[option=LUA]
function pdfe.pagestotable(<t:pdf> pdffile)
    return {
        {
            <t:userdata>, -- dictionary
            <t:integer>,  -- size
            <t:integer>,  -- objectnumber
        },
        ...
    }
end
\stoptyping

\stopsubsection

\startsubsection[title={References}]

In order to access a \PDF\ file efficiently there is lazy evaluation of
references so when you run into a reference as value or array entry you have to
resolve it explicitly. An unresolved references object can be resolved with:

\starttyping
function pdfe.getfromreference( <t:integer> reference ) -- NEEDS CHECKING
    return
        <t:integer>,  -- type
        <t:whatever>, -- value
        <t:whatever>  -- detail
\stoptyping

So, as second value you can get back a new \type {pdfe} userdata object that you
can query.

\stopsubsection

\stopsection

\stopdocument
