Gnumed.pycommon.gmPG2

Package Gnumed :: Package pycommon :: Module gmPG2 |
[frames] | no frames]

# Module gmPG2

source code

    GNUmed PostgreSQL connection handling.

    TODO: iterator/generator batch fetching:
            - http://groups-beta.google.com/group/comp.lang.python/msg/7ff516d7d9387dad
            - search Google for "Geneator/Iterator Nesting Problem - Any Ideas? 2.4"

    winner:
    def resultset_functional_batchgenerator(cursor, size=100):
            for results in iter(lambda: cursor.fetchmany(size), []):
                    for rec in results:
                            yield rec

Author: K.Hilbert <Karsten.Hilbert@gmx.net>

License: GPL v2 or later (details at http://www.gnu.org)

Classes
| cConnectionPool GNUmed database connection pool.
| cAuthenticationError
| cEncodingError
| cAdapterPyDateTime

Functions
set_default_client_encoding(encoding=None) | source code
set_default_client_timezone(timezone=None) | source code
request_login_params() Request login parameters for database connection. | source code
make_psycopg2_dsn(database=None, host=None, port=5432, user=None, password=None) | source code
get_default_login() | source code
get_default_dsn() | source code
set_default_login(login=None) | source code
log_auth_environment() | source code
database_schema_compatible(link_obj=None, version=None, verbose=True) | source code
get_schema_version(link_obj=None) | source code
get_schema_structure(link_obj=None) | source code
get_schema_hash(link_obj=None) | source code
get_schema_revision_history(link_obj=None) | source code
get_current_user() | source code
get_foreign_keys2column(schema='public', table=None, column=None, link_obj=None) Get the foreign keys pointing to schema.table.column. | source code
get_index_name(indexed_table=None, indexed_column=None, link_obj=None) | source code
get_foreign_key_names(src_schema=None, src_table=None, src_column=None, target_schema=None, target_table=None, target_column=None, link_obj=None) | source code
get_child_tables(schema='public', table=None, link_obj=None) Return child tables of <table>. | source code
schema_exists(link_obj=None, schema='gm') | source code
table_exists(link_obj=None, schema=None, table=None) Returns false, true. | source code
function_exists(link_obj=None, schema=None, function=None) | source code
get_col_indices(cursor=None) | source code
get_col_defs(link_obj=None, schema='public', table=None) | source code
get_col_names(link_obj=None, schema='public', table=None) Return column attributes of table | source code
export_translations_from_database(filename=None) | source code
delete_translation_from_database(link_obj=None, language=None, original=None) | source code
update_translation_in_database(language=None, original=None, translation=None, link_obj=None) | source code
get_translation_languages() | source code
get_database_translations(language=None, order_by=None) | source code
get_current_user_language() | source code
set_user_language(user=None, language=None) Set the user language in the database. | source code
force_user_language(language=None) Set the user language in the database. | source code
send_maintenance_notification() | source code
send_maintenance_shutdown() | source code
is_pg_interval(candidate=None) | source code
lock_row(link_obj=None, table=None, pk=None, exclusive=False) Uses pg_advisory(_shared). | source code
unlock_row(link_obj=None, table=None, pk=None, exclusive=False) Uses pg_advisory_unlock(_shared). | source code
row_is_locked(table=None, pk=None) Looks at pk_locks | source code
bytea2file(data_query=None, filename=None, chunk_size=0, data_size=None, data_size_query=None, conn=None, link2cached=True) | source code
bytea2file_object(data_query=None, file_obj=None, chunk_size=0, data_size=None, data_size_query=None, conn=None) Store data from a bytea field into a file. | source code
file2bytea(query=None, filename=None, args=None, conn=None, file_md5=None) Store data from a file into a bytea field. | source code
file2lo(filename=None, conn=None, check_md5=False) | source code
file2bytea_lo(filename=None, conn=None, file_md5=None) | source code
file2bytea_copy_from(table=None, columns=None, filename=None, conn=None, md5_query=None, file_md5=None) | source code
file2bytea_overlay(query=None, args=None, filename=None, conn=None, md5_query=None, file_md5=None) Store data from a file into a bytea field. | source code
run_sql_script(sql_script, conn=None) | source code
sanitize_pg_regex(expression=None, escape_all=False) Escape input for use in a PostgreSQL regular expression. | source code
capture_conn_state(conn=None) | source code
capture_cursor_state(cursor=None) | source code
run_ro_queries(link_obj=None, queries=None, verbose=False, return_data=True, get_col_idx=False) Run read-only queries. | source code
run_rw_queries(link_obj=None, queries=None, end_tx=False, return_data=None, get_col_idx=False, verbose=False) Convenience function for running a transaction that is supposed to get committed. | source code
run_insert(link_obj=None, schema=None, table=None, values=None, returning=None, end_tx=False, get_col_idx=False, verbose=False) Generates SQL for an INSERT query. | source code
get_raw_connection(dsn=None, verbose=False, readonly=True, connection_name=None, autocommit=False) Get a raw, unadorned connection. | source code
get_connection(dsn=None, readonly=True, encoding=None, verbose=False, pooled=True, connection_name=None, autocommit=False) Get a new connection. | source code
discard_pooled_connection(conn_key=None) | source code
shutdown() | source code
log_database_access(action=None) | source code
sanity_check_time_skew(tolerance=60) Check server time and local time to be within the given tolerance of each other. | source code
sanity_check_database_settings() Checks database settings. | source code
make_pg_exception_fields_unicode(exc) | source code
extract_msg_from_pg_exception(exc=None) | source code

Variables
| FixedOffsetTimezone = `dbapi.tz.FixedOffsetTimezone`
| default_database = `'gnumed_v22'`
| postgresql_version_string = `None`
| postgresql_version = `None`
| auto_request_login_params = `True`
| known_schema_hashes = `{0: 'not released, testing only', 2: 'b0`...``
| map_schema_hash2version = `{'b09d50d7ed3f91ddf4c4ddb8ea507720':`...``
| map_client_branch2required_db_version = `{'GIT tree': 0, '0.3':`...``
| map_psyco_tx_status2str = `['TRANSACTION_STATUS_IDLE', 'TRANSAC`...``
| map_psyco_conn_status2str = `['0 - ?', 'STATUS_READY', 'STATUS_`...``
| map_psyco_iso_level2str = `{None: 'ISOLATION_LEVEL_DEFAULT (con`...``
| query_table_col_defs = `"""sel`...``
| query_table_attributes = `"""sel`...``
| SQL_foreign_key_name = `"""SEL`...``
| SQL_get_index_name = ``...``
| SQL_get_pk_col_def = ``...``

Imports: time, sys, os, stat, io, codecs, logging, pydt, regex, threading, hashlib, shutil, gmLoginInfo, gmExceptions, gmDateTime, gmBorg, gmI18N, gmLog2, gmTools, prompted_input, u_replacement_character, format_dict_like, dbapi, psycopg2, sql_error_codes

Function Details

### get_foreign_keys2column(schema='public', table=None, column=None, link_obj=None) | source code
Get the foreign keys pointing to schema.table.column. Does not properly work with multi-column FKs. GNUmed doesn't use any, however.

### set_user_language(user=None, language=None) | source code
Set the user language in the database. user = None: current db user language = None: unset

### force_user_language(language=None) | source code
    Set the user language in the database.

    - regardless of whether there is any translation available.
    - only for the current user

### lock_row(link_obj=None, table=None, pk=None, exclusive=False) | source code
    Uses pg_advisory(_shared).

    - locks stack upon each other and need one unlock per lock
    - same connection:
            - all locks succeed
    - different connections:
            - shared + shared succeed
            - shared + exclusive fail

### unlock_row(link_obj=None, table=None, pk=None, exclusive=False) | source code
    Uses pg_advisory_unlock(_shared).

    - each lock needs one unlock

### row_is_locked(table=None, pk=None) | source code
    Looks at pk_locks

    - does not take into account locks other than 'advisory', however

### bytea2file_object(data_query=None, file_obj=None, chunk_size=0, data_size=None, data_size_query=None, conn=None) | source code
    Store data from a bytea field into a file.

    <data_query>
    - dict {'cmd': ..., 'args': ...}
    - 'cmd' must be a string containing "... substring(data from %(start)s for %(size)s) ..."
    - 'args' must be a dict
    - must return one row with one field of type bytea
    <file>
    - must be a file like Python object
    <data_size>
    - integer of the total size of the expected data or None
    <data_size_query>
    - dict {'cmd': ..., 'args': ...}
    - must return one row with one field with the octet_length() of the data field
    - used only when <data_size> is None

### file2bytea(query=None, filename=None, args=None, conn=None, file_md5=None) | source code
    Store data from a file into a bytea field.

    The query must:
    - be in unicode
    - contain a format spec identifying the row (eg a primary key)
      matching <args> if it is an UPDATE
    - contain a format spec " <field> = %(data)s::bytea"

    The query CAN return the MD5 of the inserted data:
            RETURNING md5(<field>) AS md5
    in which case it will compare it to the md5
    of the file.

### file2bytea_overlay(query=None, args=None, filename=None, conn=None, md5_query=None, file_md5=None) | source code
    Store data from a file into a bytea field.

    The query must:
    - 'cmd' must be in unicode
    - 'cmd' must contain a format spec identifying the row (eg
      a primary key) matching <args> if it is an UPDATE
    - 'cmd' must contain "... SET ... <some_bytea_field> = OVERLAY(some_bytea_field PLACING %(data)s::bytea FROM %(start)s FOR %(size)s) ..."
    - 'args' must be a dict matching 'cmd'

    The query CAN return the MD5 of the inserted data:
            RETURNING md5(<field>) AS md5
    in which case it will compare it to the md5
    of the file.

    UPDATE
            the_table
    SET
            bytea_field = OVERLAY (
                    coalesce(bytea_field, '':bytea),
                    PLACING
                            %(data)s::bytea
                    FROM
                            %(start)s
                    FOR
                            %(size)s
            )
    WHERE
            primary_key = pk_value

    SELECT md5(bytea_field) FROM the_table WHERE primary_key = pk_value

### sanitize_pg_regex(expression=None, escape_all=False) | source code
    Escape input for use in a PostgreSQL regular expression.

    If a fragment comes from user input and is to be used
    as a regular expression we need to make sure it doesn't
    contain invalid regex patterns such as unbalanced ('s.

    <escape_all>
            True: try to escape *all* metacharacters
            False: only escape those which render the regex invalid

### run_ro_queries(link_obj=None, queries=None, verbose=False, return_data=True, get_col_idx=False) | source code
    Run read-only queries.

    <queries> must be a list of dicts:
            [
                    {'cmd': <string>, 'args': <dict> or <tuple>},
                    {...},
                    ...
            ]

### run_rw_queries(link_obj=None, queries=None, end_tx=False, return_data=None, get_col_idx=False, verbose=False) | source code
    Convenience function for running a transaction
       that is supposed to get committed.

    <link_obj>
            can be either:
            - a cursor
            - a connection

    <queries>
            is a list of dicts [{'cmd': <string>, 'args': <dict> or <tuple>)
            to be executed as a single transaction, the last
            query may usefully return rows (such as a
            "SELECT currval('some_sequence')" statement)

    <end_tx>
            - controls whether the transaction is finalized (eg.
              committed/rolled back) or not, this allows the
              call to run_rw_queries() to be part of a framing
              transaction
            - if link_obj is a connection then <end_tx> will
              default to False unless it is explicitly set to
              True which is taken to mean "yes, you do have full
              control over the transaction" in which case the
              transaction is properly finalized
            - if link_obj is a cursor we CANNOT finalize the
              transaction because we would need the connection for that
            - if link_obj is None <end_tx> will, of course, always be True

    <return_data>
            - if true, the returned data will include the rows
              the last query selected
            - if false, it returns None instead

    <get_col_idx>
            - if true, the returned data will include a dictionary
              mapping field names to column positions
            - if false, the returned data returns None instead

    method result:
            - returns a tuple (data, idx)
            - <data>:
                    * (None, None) if last query did not return rows
                    * ("fetchall() result", <index>) if last query returned any rows
                    * for <index> see <get_col_idx>

### run_insert(link_obj=None, schema=None, table=None, values=None, returning=None, end_tx=False, get_col_idx=False, verbose=False) | source code
Generates SQL for an INSERT query. values: dict of values keyed by field to insert them into

### get_raw_connection(dsn=None, verbose=False, readonly=True, connection_name=None, autocommit=False) | source code
    Get a raw, unadorned connection.

    - this will not set any parameters such as encoding, timezone, datestyle
    - the only requirement is a valid DSN
    - hence it can be used for "service" connections
      for verifying encodings etc

### get_connection(dsn=None, readonly=True, encoding=None, verbose=False, pooled=True, connection_name=None, autocommit=False) | source code
Get a new connection. This assumes the locale system has been initialized unless an encoding is specified.

### sanity_check_time_skew(tolerance=60) | source code
Check server time and local time to be within the given tolerance of each other. tolerance: seconds

### sanity_check_database_settings() | source code
    Checks database settings.

    returns (status, message)
    status:
            0: no problem
            1: non-fatal problem
            2: fatal problem

Variables Details

  ### known_schema_hashes Value:
      {0: 'not released, testing only', 2: 'b09d50d7ed3f91ddf4c4ddb8ea507720
      ', 3: 'e73718eaf230d8f1d2d01afa8462e176', 4: '4428ccf2e54c289136819e70
      1bb095ea', 5: '7e7b093af57aea48c288e76632a382e5', 6: '90e2026ac2efd236
      da9c8608b8685b2d', 7: '6c9f6d3981483f8e9433df99d1947b27', 8: '89b13a7a
      f83337c3aad153b717e52360', 9: '641a9b2be3c378ffc2bb2f0b1c9f051d', 10:
      '7ef42a8fb2bd929a2cdd0c63864b4e8a', 11: '03042ae24f3f92877d986fb0a6184
      d76', 12: '06183a6616db62257e22814007a8ed07', 13: 'fab7c1ae408a6530c47
      f9b5111a0841e', 14: 'e170d543f067d1ea60bfe9076b1560cf', 15: '70012ff96
      ...

  ### map_schema_hash2version Value:
      {'b09d50d7ed3f91ddf4c4ddb8ea507720': 2, 'e73718eaf230d8f1d2d01afa8462e
      176': 3, '4428ccf2e54c289136819e701bb095ea': 4, '7e7b093af57aea48c288e
      76632a382e5': 5, '90e2026ac2efd236da9c8608b8685b2d': 6, '6c9f6d3981483
      f8e9433df99d1947b27': 7, '89b13a7af83337c3aad153b717e52360': 8, '641a9
      b2be3c378ffc2bb2f0b1c9f051d': 9, '7ef42a8fb2bd929a2cdd0c63864b4e8a': 1
      0, '03042ae24f3f92877d986fb0a6184d76': 11, '06183a6616db62257e22814007
      a8ed07': 12, 'fab7c1ae408a6530c47f9b5111a0841e': 13, 'e170d543f067d1ea
      60bfe9076b1560cf': 14, '70012ff960b77ecdff4981c94b5b55b6': 15, '0bcf44
      ...

  ### map_client_branch2required_db_version Value:
      {'GIT tree': 0, '0.3': 9, '0.4': 10, '0.5': 11, '0.6': 12, '0.7': 13,
      '0.8': 14, '0.9': 15, '1.0': 16, '1.1': 16, '1.2': 17, '1.3': 18, '1.4
      ': 19, '1.5': 20, '1.6': 21, '1.7': 22, '1.8': 22}

  ### map_psyco_tx_status2str Value:
      ['TRANSACTION_STATUS_IDLE', 'TRANSACTION_STATUS_ACTIVE', 'TRANSACTION_
      STATUS_INTRANS', 'TRANSACTION_STATUS_INERROR', 'TRANSACTION_STATUS_UNK
      NOWN']

  ### map_psyco_conn_status2str Value:
      ['0 - ?', 'STATUS_READY', 'STATUS_BEGIN_ALIAS_IN_TRANSACTION', 'STATUS
      _PREPARED']

  ### map_psyco_iso_level2str Value:
      {None: 'ISOLATION_LEVEL_DEFAULT (configured on server)', 0: 'ISOLATION
      _LEVEL_AUTOCOMMIT', 1: 'ISOLATION_LEVEL_READ_UNCOMMITTED', 2: 'ISOLATI
      ON_LEVEL_REPEATABLE_READ', 3: 'ISOLATION_LEVEL_SERIALIZABLE', 4: 'ISOL
      ATION_LEVEL_READ_UNCOMMITTED'}

  ### query_table_col_defs Value:
      """select
      	cols.column_name,
      	cols.udt_name
      from
      	information_schema.columns cols
      where
      	cols.table_schema = %s
      		and
      ...

  ### query_table_attributes Value:
      """select
      	cols.column_name
      from
      	information_schema.columns cols
      where
      	cols.table_schema = %s
      		and
      	cols.table_name = %s
      ...

  ### SQL_foreign_key_name Value:
      """SELECT
      	fk_tbl.*,
      	(SELECT nspname FROM pg_catalog.pg_namespace WHERE oid = fk_tbl.conna
      mespace) AS constraint_schema,
      	fk_tbl.conname AS constraint_name,
      	(SELECT nspname FROM pg_catalog.pg_namespace WHERE oid = (SELECT reln
      amespace FROM pg_class where oid = fk_tbl.conrelid)) AS source_schema,
      	(SELECT relname FROM pg_catalog.pg_class where oid = fk_tbl.conrelid)
      ...

  ### SQL_get_index_name Value:
      """
      SELECT
      	(SELECT nspname FROM pg_namespace WHERE pg_namespace.oid = pg_class.r
      elnamespace)
      		AS index_schema,
      	pg_class.relname
      		AS index_name
      FROM
      ...

  ### SQL_get_pk_col_def Value:
      """
      SELECT
      	pg_attribute.attname
      		AS pk_col,
      	format_type(pg_attribute.atttypid, pg_attribute.atttypmod)
      		AS pk_type
      FROM pg_index, pg_class, pg_attribute, pg_namespace
      WHERE
      ...