GmManualReportGenerator < Gnumed < Foswiki
Description: GmManualReportGenerator

# The GNUmed Report Generator

GNUmed offers two fundamentally different ways to search the database:

1. across the EMR of the currently active patient
2. across the entire medical database regardless of the active patient

The second approach is sometimes called data mining. GNUmed has a plugin called Reports to enable you to create database-wide reports.

## Scope

The plugin is intended to generate simple reports. The powers of this tool do not go beyond what you can do from within PostgreSQL. However, one can enhance PostgreSQL with the "R" procedural language (or, in fact, any other one) in order to unleash considerable statistical powers right from within the SQL query.

For anything more sophisticated than that (say, post-processing the report results) one will have to turn to custom scripting, off-the-shelf report generators or data mining tools such as NetEpi.

## Usage

### Generating Reports

To generate a report from the database you need to run an SQL query. The query has to be typed into the Command (SQL) field of the Reports plugin. Then hit the button [Run]. The results will be shown in the list at the bottom. The columns of the list will correspond to the columns of the database table(s) you collect data from with the query. You may want to use the SQL AS column alias syntax to re-express ("map") database column names AS friendlier display labels.

Here are a few things to know:

- you don't need to worry about leading or trailing whitespace/linefeeds
- your query may span several lines
- you don't need to end your query with a semicolon (";") - but you can
- you can paste query text from elsewhere via the clipboard
- you can drag a file onto the Reports plugin and GNUmed will interpret the file content as the query to run
  - note that you need to drag the file onto an area of the plugin outside the actual query command field (this will be improved later)

- the plugin will artificially limit the results list to 1000 rows (one thousand) such as to somewhat safeguard against queries going berserk

- the queries are run in a read-only connection with the credentials of the user who logged on with this client
  - no writing to the database is possible, not even as a side-effect of a `select my_writing_func()` query

- you might want to make your query return a column named `pk_patient`
  - this will enable you to double-click on any row in the results list which will activate the patient identified by the database ID found in `pk_patient`
  - if there is no such column nothing will happen (besides an error message being shown)
  - note that you can, of course, make a query return an arbitrary number in a column named `pk_patient` in which case the corresponding patient will, indeed, be activated upon double-clicking a row but there may not, in fact, be any other meaningful correlation of the patient with that row

- you can limit your query to the currently-active patient by including, in the WHERE clause, the placeholder $<ID_active_patient>$ with something like: `<table_or_view_relatable_to.pk_identity> = $<ID_active_patient>$` or something like

          EXISTS (
             SELECT 1
             FROM
                table_or_view_containing_else_relatable_to_pk_patient
             WHERE
                column_containing_pk = $<ID_active_patient>$
             )
          ;

### Visualizing reports

Hitting the [Visualize] button will let you select a column from the report results list from each of the x- and y-axis. The data in those columns is extracted from the report and sent to gnuplot for display.

### Reusing report definitions

The Report field acts as a phrasewheel offering names of reports that were previously saved in the database. You can either type part of a name our part of a query (such as a table name) and select a report definition from the appearing dropdown match list. The corresponding query will be loaded from the database.

If you press [Save] the report definition will be saved in the database. If the report name is already known in the database the existing report definition will be overwritten. If not a new report definition will be created.

Hitting [Contribute] will email the report definition (name and query - nothing else) to the mailing list of the GNUmed community for all to share. This will happen anonymously. If you want to receive credit for it you'll have to actively claim it on the mailing list.

Note that report results are only preserved as long as the client instance they were generated in stays open. They will, however, survive changing the active patient.

The [Schema] button will take you to the GNUmed database schema documentation in our wiki for your reference.

### Sample queries

How many believed-to-be-alive patients remain in the praxis database?

       select count(*) from dem.identity
       where deleted is FALSE / TRUE
       where deceased is NULL / NOT NULL

List for me the (hopefully less than 1024) patients in this database:

       select lastnames, firstnames, title, pk_identity AS pk_patient
       from dem.v_basic_person
       where dem.v_basic_person.lastnames is NOT NULL
       order by lastnames, firstnames

List for me patients having a particular postal code

    select number, street, dem.v_basic_person.lastnames, dem.v_basic_person.preferred, dem.v_basic_person.firstnames, suburb, urb, postcode, pk_identity as pk_patient
    from
       dem.v_basic_person
          inner join
       dem.v_pat_addresses
          using (pk_identity)
    where
       LOWER(dem.v_pat_addresses.postcode) = 'inputDesiredPostalCodeHereInLowerCase'
    order by
       street, number

List for me the patients waitlisted (without a waiting_zone specied) for more than 14 days:

    select lastnames, firstnames, title, comment, waiting_time_formatted, pk_identity as pk_patient
       from
           clin.v_waiting_list
          where
             waiting_time < '14 days'
          and waiting_zone is NULL

A query that (new in gnumed_v9) can search for patients based on the diagnostic code

       select *
       from
          dem.v_basic_person
             inner join
          clin.v_coded_item_narrative
             using (pk_identity)
       where
          code = ...
          and coding_system = ...
          and soap_cat = ...
       ;

A query that would return the number of patients seen, based on encounters of particular type, during an interval of time. Notice that here, the selection is made based on when the encounters of interest started (note: to count each patient only once, one must use 'select distinct fk_patient' in place of 'select fk_patient'):

    select count(1)
    from dem.identity d_i
    where
       d_i.pk in (
          select fk_patient from clin.encounter
          where
             fk_type in (select pk from clin.encounter_type where description in ('list', 'of', 'interesting', 'types')
                and
             started between 'year-01-01' and 'year-12-31'
       )
    ;

A query that, upon removing or fixing ", ...", would return all patients whose encounter of types of interest ('seen in clinic' etc) which the current user has created or modified in the past 21 days:

    SELECT
       to_char(c_e.modified_when,'yyyy.mm.dd hh:mm') modified,
       d_n.lastnames || ', ' || d_n.firstnames person_encountered,
       c_et.description encounter_type,
       to_char(c_e.started,'yyyy.mm.dd hh:mm') started,
       c_e.assessment_of_encounter aoe,
       c_e.pk AS pk_encounter,
       c_e.fk_patient pk_patient
    FROM
       clin.encounter c_e INNER JOIN dem.names d_n ON c_e.fk_patient = d_n.id_identity INNER JOIN clin.encounter_type c_et ON c_e.fk_type = c_et.pk
    WHERE
       c_e.modified_by = "current_user"()
          AND
       c_e.modified_when > now() - interval '21 days'
          AND
       c_e.fk_type IN (select pk from clin.encounter_type where description IN ('seen in clinic', 'seen elsewhere', 'not seen but called', ...))
    ORDER BY
       c_e.started DESC
    ;

and a query that would return the list of encounter descriptions from which to be able to build the query above:

    select * from clin.encounter_type ORDER BY description ;

A query that would help by providing more fields (and sample values) that could be altered and used to find a patient when the standard patient search field did not permit a patient to be found, perhaps including the communication channels (phone numbers). Such queries could be

       select *
       from
          dem.v_basic_person
             inner join
          dem.v_person_comms / dem.v_person_jobs / dem.v_external_ids4identity
             using (pk_identity)
       where
          dem.v_person_comms.url = ... /
          dem.v_person_jobs.l10n_occupation = ... /
          dem.v_external_ids4identity.value = ...
       ;

A query that would fetch, from the inbox audit table, the messages deleted within the past 7 days, ordered by recency of last-modified

    SELECT *
    FROM audit.log_message_inbox
    WHERE
       fk_staff = <staff ID of provider>
          AND
       audit_action = 'DELETE'
          AND
       audit_when > (now() - '7 days'::interval)
    ORDER BY
       <audit_when / orig_when / modified_when> DESC
    ;

A query that could identify auto-created persons as might result from a data importer

    SELECT  * from dem.clin_ext_id_type, dem.identity where
    dem.clin_ext_id_type.name = "lab autoimport fake person" WHERE
    dem.clin_ext_id_type.fk_person = dem.identity.pk
    ;

There has been discussion offlist between Karsten and Jim "on theory of primary care" modeling levels of clinician diagnostic certainly. Once this would be captured in the encounters it would make for interesting queries to the effect of

    The patients I would most worry about would be those who
    - remain our responsibility (they did not abandon us)
       --> last seen in the most recent 6 (?) months
    - and have an active issue or episode of certainty of A or B or C that is
       --> persisting over multiple encounters
          >= 2 encounters if symptom(s) are "alarming" or "worsening"
          >= 3 encounters if B or C

    Many patients have a chronic single symptoms at level A, and maybe a
    chronic symptom complex at level B, but &#8211; as long as their episode is
    not worsening (or provided the patient's episodes are not becoming more
    frequent which would be a separate clinically informative query) &#8211; then
    it may be tolerable to optionally and by default omit such patients with
    chronicity of > 6 or 9 months from inclusion in the result of a query if
    the purpose is "who must I make sure I do not overlook a condition that I
    should perhaps be diagnosing?"

## Backend note

Report queries are stored in cfg.report_query.