.. _coding-practices:

================
Coding practices
================

Please refer to :ref:`Contributing <contributing>` for general contributing
guidelines. Once you are setup to contribute and know what you want to
work on, you should read this document to learn our expectations on the
code that you will submit as well as on the process leading to integrate
your contribution.

.. _tdd:

Introduction to Test Driven Development
---------------------------------------

We expect you to follow this methodology when developing new code so
if you are not familiar with it, have a look at `Test-Driven Web
Development with Python <https://www.obeythetestinggoat.com/>`_.

The suggested workflow looks like this:

  1. Add a functional test that covers the new feature from the point of
     view of the user. This test will fail since the feature doesn't exist
     yet.

  2. Think about what's the next step to let the functional test go
     further (i.e. fail later).

  3. Write a failing unit test for the new code that you want to write.

  4. Write the minimal code to make your unit test pass. You will
     typically run this very often:

     .. code-block:: console

        $ ./manage.py test path-to-the-testing-folder

  5. Refactor (if needed). You might have introduced some duplication in
     your code or in your tests. Clean that up now before it's too late.

  6. Commit (optional). Commit together the (working) unit tests and the
     new code.

  7. If you made progress from the functional tests point of view, go back
     to point 2, otherwise go back to point 3. If the functional test
     passes, continue.

  8. Commit. The functional tests are committed at this point to ensure
     that they are committed in a working state:

     .. code-block:: console

        $ git add .
        $ git commit

When you don't develop a new feature, your workflow is restricted to steps
3 to 6.

About merge requests and code reviews
-------------------------------------

Size of merge requests
~~~~~~~~~~~~~~~~~~~~~~

Submitted Merge Requests should consist of a single commit and should have
a reasonable size: that is no more than 1000 lines of changes (total of
added and removed lines, excluding test data and other auto-generated
files) for large changes. We expect the majority of MR to be smaller than
that. The single commit approach ensures that the CI runs on all commits
that we merge, and makes it easy to push fixup commits that are all
squashed together prior to merge.

If you are working on a large feature, we expect you to submit it
progressively in multiple independent steps. If the separate merge of
those individual steps would break debusine, then ask us to create a
staging branch for your new feature and target that separate branch in
your merge requests.

If you want to provide a merge request with multiple commits to make
it easier to review, it's possible but be aware that the commits will be
squashed together: thus please write the global commit message in your
first commit. If you don't want your commits to be squashed together,
please say so in the merge request description and explain your reasons.

Discussions in merge requests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Discussions within Merge Requests should be limited to code reviews:
pointing out mistakes and inconsistencies with the associated issue,
suggesting improvements, etc. If there are architecture or design issues
that need to be addressed, or if there are disagreements between the coder
and the reviewer, then those discussions should be moved to a separate
issue and be resolved there before getting back to the MR.

Use of 'Draft' status
~~~~~~~~~~~~~~~~~~~~~

Draft merge requests are not open for detailed review (unless the
submitter explicitly asks for early review). They are usually
created early to benefit from the full CI testing during development. Thus
they can be freely rebased, refactored and can be subject to large
changes. They also don't have any assignee or reviewer set.

Once the draft status is dropped, the merge request is open for review and
subsequent changes to respond to review comments will be restricted to
adding new commits.

Usage of 'Assignee' and 'Reviewer'
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

As a submitter of a merge request, you don't have to set those fields,
by default every contributor is encouraged to review open merge requests.
There are two cases where we might set the Reviewer field:

* when the submitter would like to ask someone specific to review the MR
  (due to former experience on the code being modified for example)
* when a reviewer wants to notify other team members that he is currently
  reviewing that merge request

Coding style
------------

  1. In regard to coding style, we observe `PEP8
     <https://peps.python.org/pep-0008/>`_ with a few exceptions.
     And we format the code with `black <https://github.com/psf/black>`_
     by running ``make black`` (we use a different line length and don't
     impose the single type of quote for strings).

  2. Functions are documented using docstrings with `Sphinx markup
     <https://www.sphinx-doc.org/en/master/>`_.

  3. Imports are sorted in multiple groups separated by one empty line:
     first a group for ``__future__`` imports, then a single group for all
     the Python standard modules, then one group for each third-party
     module (and groups are sorted between them as well), followed by
     groups for the project modules, and last, one group for
     relative imports.

     Within each group the ``import foo`` statements are grouped and
     sorted at the top, while the ``from foo import bar`` statements
     are grouped and sorted at the end.

     Example:

.. code-block:: python3

   from __future__ import print_function

   import datetime
   import os
   from datetime import timedelta
   from email.utils import getaddresses, parseaddr

   from django.conf import settings
   from django.db import connection, models
   from django.utils.safestring import mark_safe

   import requests
   from requests.structures import CaseInsensitiveDict

   from debusine.artifacts.models import Artifact

Git commit messages
-------------------

Please invest some time to write good commit messages. Just like your code,
you write it once but it will be read many times by different persons
looking to understand why you made the change. So make it pleasant to
read.

The first line is the “summary” (or title) and describes briefly what the
commit changes. It's followed by an empty line and a long description. The
long description can be as long as you want and should explain why you
implemented the change seen in the commit.

The long description can also be used to close bugs by putting some
pseudo-fields at the end of the description:

 * for a GitLab issue, use ``Fixes: #XX`` (this is a standard GitLab
   feature)
 * for a Debian bug, use ``Closes: #XXXXXX`` (this is implemented by a
   `webhook <https://salsa.debian.org/salsa/webhook>`_)

Good code, good design
----------------------

This section documents different decisions about implementation,
naming, etc. that happened during merge requests. It is not an
exclusive list of all the discussions and is subject to change.

Those rules are meant to help improve consistency and to obtain
a cleaner overall design.

Models
~~~~~~

Avoid changing fields from outside the model
********************************************

Avoid changing fields in the models from their users. Do not do:

.. code-block:: python3

   worker.connected_at = timezone.now()

Instead, create a method in Worker describing the action that you are doing:

.. code-block:: python3

   worker.mark_connected()

And change the relevant fields from ``mark_connected()``.

This allows the model's fields or logic to change without having to change
the code which accesses it.

Read more in `Push actions to the model layer <https://spookylukey.github.io/django-views-the-right-way/thin-views.html#example-push-actions-to-the-model-layer>`_.

.. _push-filtering-model-layer:

Push filtering to the model layer
*********************************

In order to encapsulate logic for ``filter`` and other queries, add a ModelManager
to the Model and do the filtering there. Do not do:

.. code-block:: python3

   worker.objects.filter(connected_at__isnull=False)

Instead create a ``connected`` method in the Worker's Manager and use it:

.. code-block:: python3

   worker.objects.connected()

This allows the code base to be consistent in the filtering.

Read more in `Push filtering to the model layer <https://spookylukey.github.io/django-views-the-right-way/thin-views.html#example-push-filtering-to-the-model-layer>`_.

Push Model.objects.create() to the model layer
**********************************************

Similar to
:ref:`Push filtering to the model layer <push-filtering-model-layer>`:
avoid using ``Model.objects.create()`` (or ``.get_or_create()``) and add a method
in the ModelManager describing the operation, such as:

.. code-block:: python3

   Worker.objects.create_with_fqdn(fqdn, token)


Naming fields
*************
Add the suffix _at for the fields of type `DateTime`:

.. code-block:: python3

   created_at = models.DateTimeField()

Tests
~~~~~

Private methods
***************
To facilitate Test-Driven Development and localised tests, it is ok to
call private methods from the tests.

Assert function: order of the parameters
****************************************
In the assert methods, put the "expected" value as second parameter, for example:

.. code-block:: python3

   self.assertEqual(actual, expected)

Reason: some test methods such as ``assertQuerysetEqual`` expect "actual"
to be the first parameter. Always using this order helps the tests to be
more easily read.

Assert functions: assertEqual or specific
*****************************************
When there is a TestCase method with specific semantics, use them:

 * ``self.assertQuerysetEqual()`` for testing querysets
 * ``self.assertTrue()`` or ``self.assertFalse()`` for testing boolean expressions
 * ``self.assertIn()`` or ``self.assertNotIn()``

Using the specific methods such as ``self.assertIn()`` helps to have a better test
output compared with constructions such as ``self.assertTrue('john' in people)``.

When possible (actual and expected are the same type), use ``self.assertEqual()``
instead of methods such as ``self.assertDictEqual()``. ``self.assertEqual()`` will
use the correct underlying method.

General guidelines
~~~~~~~~~~~~~~~~~~

Constants
*********

If one of our dependencies provides defined public constants, use them instead
of re-defining them or using magic numbers.

.. code-block:: python3

    # Use:
    from rest_framework import status
    code = status.HTTP_501_NOT_IMPLEMENTED

    # Instead of:
    code = 501

This helps readability for readers that might not know all the internal codes,
might avoid typos and if the "constants" depended on versions, environment, etc.
the library will take care of them.

Type hints
**********
If you want to indicate the type of a variable, type hints are preferable to
adding suffixes to the variable.

.. code-block:: python3

    # Use:
    def method(information: dict):
       ...

    # Instead of:
    def method(information_dict):
        ...

This helps (IDEs, mypy) to give hints to the programmer and it keeps
the variable names shorter, avoiding the type repetition.


Early exit
**********

To exit early:

.. code-block:: python3

  # Use:
  raise SystemExit(3)

  # Instead of:
  sys.exit(3)
  exit(3)

It says explicitly what it does and there is no need to import the `sys` module.

If any utility in Debusine must exit early:
 * Use **exit code 3**. Exit code 1 is used by the Python interpreter for
   unhandled exceptions and exit code 2 by the argparse module for invalid
   command syntax.
 * Make sure to **log or print** (depending on the circumstances) why an early
   exit has happened so the user or admin can fix the situation.
