.. index:: explain first job, first job detail

.. _explain_first_job:

Explaining the first job in detail
##################################

`Download / view <examples/test-jobs/qemu-pipeline-first-job.yaml>`_ the
complete first job YAML.

As the first job, this is a simple emulated test which does not need any extra
hardware, it does not use protocols and runs as a single job. The elements of
the first job are described in detail below.

.. note:: This kind of test is generally termed ``singlenode``.

Top level elements of a test job
********************************

* **comments** - One of the reasons why V2 uses YAML for job
  submission is because YAML supports comments. **Please use comments
  liberally**. Comments are preserved in the job definition which can
  be viewed and downloaded from the instance. Comments are not passed
  to the dispatcher.

* **device_type** - a label which must match the type of :term:`device`
  declared on the instance to which you want to submit this job. Each
  :term:`device type` can have one or more devices available. In this case,
  ``qemu``.

* **job_name** - your own free text name for this job. This name is used in the
  server side tables but has no further relevance for the dispatcher. There is
  no need to make this into a single long word, any valid YAML string is
  allowed, so commas, capitalisation and whitespace are fine. Long job names,
  in particular, need to include whitespace. Avoid the temptation to use a
  build URL as the job name, use :ref:`job_metadata` instead.

* **timeouts** - a range of things can go wrong inside a LAVA test job, so the
  test writer needs to indicate how long parts of the job should take. This
  allows LAVA to fail jobs which have hung and return the device to scheduling
  for the next test job. Timeouts at the top level include the job, the default
  timeout per action and can include the default timeout for the device to
  respond using a connection.

  .. seealso:: :ref:`timeouts`

* **priority** - supports any integers in [0, 100]. Three shortcuts are also
  available: ``high`` (100), ``medium`` (50) and ``low`` (0). The
  :term:`scheduler` considers the job priority when ordering the queue to
  consider which job should run next. **Do not abuse priority**.

* **visibility** - supports values of ``public``, ``personal`` and ``group``
  and controls who is allowed to view the job and the results generated by the
  job. This includes whether the results are available to queries and to
  charts.

  ``group`` visibility setting should list the groups users must be in to
  be allowed to see the job. If more than one group is listed, users must
  be in all the listed groups to be able to view the job or the results::

    visibility:
      group:
        - developers
        - project

  In this example, users must be members of both ``developers`` group and
  ``project`` group.

* **context** - Not all testjobs will use a :term:`job context`, it is
  available to override some of the server-side device configuration templates.
  In this case, ``arch: amd64`` sets the template to use the
  ``qemu-system-x86_64`` executable when starting the emulation.

* **metadata** - Once the test job becomes part of an automated submission or
  a :abbr:`CI (Continous Integration)` :term:`loop <ci loop>`, metadata needs
  to be used to identify each submission, provide information on exactly what
  changed from the last test job and information on how other users can modify
  the test job artifacts to extend or debug the results.

  .. seealso:: :ref:`job_metadata`, :ref:`continuous_integration` and
    :ref:`ci_loop`

General details of the test job are specified in the top level elements, such
as in this snippet from the first job:

.. include:: examples/test-jobs/qemu-pipeline-first-job.yaml
   :code: yaml
   :start-after: x86_64 QEMU
   :end-before: # context

Actions within the test job
***************************

Each test job needs a list of actions, comprising of ``deploy``, ``boot`` and
``test``.

* **deploy** - download files required to boot the device and prepare an
  overlay of files to run in the test action.

* **boot** - specify the method to boot the device and the prompts which will
  dictate whether the device is considered to have been booted correctly.

* **test** - specify the repositories to clone which will provide the scripts
  which will run the test you want to execute for this job.

.. _first_deploy_action_qemu:

Deploy action for QEMU
======================

Deployment methods are identified by the ``to`` parameter, for QEMU the
supported deployment method is to ``tmpfs``. The image to deploy is specified
as an image dictionary which supports the formatting of the options to QEMU and
the compression algorithm to use to decompress the download prior to passing to
QEMU.

Arguments to QEMU need to include the filename, often embedded within a
specific option string. To achieve this, LAVA supports a ``label`` for the
image which will be substituted into the option.

.. include:: examples/test-jobs/qemu-pipeline-first-job.yaml
   :code: yaml
   :start-after: ACTION_BLOCK
   :end-before: # BOOT_BLOCK

The other role of a deploy action is to prepare the overlay which will contain
the test shell scripts and repositories. These will be added to the booted
image and then executed automatically, generating the results for the test job.

Certain aspects of executing tests on a booted device require knowledge about
which :abbr:`OS (operating system)` will be running after the device has
booted. This is particularly relevant when the test scripts may require
additional dependencies to be installed in the running system. The test scripts
need to know whether to use ``apt`` or ``yum`` or something else to do the
installation work. Some other OS deployments may change other elements within
the test, so the test job submission will **fail** if the ``os`` parameter is
not set or is set to an unrecognised string.

Supported operating systems include ``debian``, ``ubuntu``, ``oe`` (for
OpenEmbedded) and ``fedora``.

Example of deploy label substitution
------------------------------------

``https://images.validation.linaro.org/kvm/standard/stretch-2.img.gz`` is
downloaded and then decompressed using the ``gz`` algorithm to create a file
called ``stretch-2.img`` in a ``tmpfs`` location. This location is then
substituted into the ``image_arg``::

 -drive format=raw,file=/tmp/tmp.23FDsf/stretch-2.img

Boot action for QEMU
====================

One of the primary roles of the boot action parameters is to ensure that the
correct pipeline is constructed for this test job. The specified method is used
to match against the available boot methods. In this case, the boot method is
to call QEMU. The ``qemu`` boot method also needs the ``media`` parameter set
to ``tmpfs`` to distinguish this from other boot methods. The first job uses a
:term:`test shell` and needs to specify the list of :term:`prompts` which LAVA
can recognise to start the operation of the test shell.

.. include:: examples/test-jobs/qemu-pipeline-first-job.yaml
   :code: yaml
   :start-after: - boot:
   :end-before: - test:

.. note:: **prompts** - it is **essential** that the test writer records the
   information on what prompt will be presented when the system boots into that
   image successfully. If this information is not available from the location
   from which you downloaded the image, you will need to ask for that
   information or boot the image yourself and identify the prompt.

Test action for QEMU
====================

The test action block in the first job contains two sets of definition
parameters, each consisting of:

* **repository** - the URL to pass to git or bzr to clone the repository

* **from** - the method of obtaining the repository.

* **path** - the YAML file inside the repository which contains the Lava Test
  Shell Definition to be used for this test.

* **name** - the name to use when executing this test.

The test action block in the first job also includes a timeout as an example of
how to specify a timeout for a particular section of the job.

.. include:: examples/test-jobs/qemu-pipeline-first-job.yaml
   :code: yaml
   :start-after: username: root

.. seealso:: Back to your first job :ref:`job_submission`.
