patroni.config module

Facilities related to Patroni configuration.

class patroni.config.Config(configfile: str, validator: ~typing.Callable[[~typing.Dict[str, ~typing.Any]], ~typing.List[str]] | None = <function default_validator>)View on GitHub

Bases: object

Handle Patroni configuration.

This class is responsible for:

  1. Building and giving access to effective_configuration from:

    • Config.__DEFAULT_CONFIG – some sane default values;

    • dynamic_configuration – configuration stored in DCS;

    • local_configuration – configuration from config.yml or environment.

  2. Saving and loading dynamic_configuration into ‘patroni.dynamic.json’ file located in local_configuration[‘postgresql’][‘data_dir’] directory. This is necessary to be able to restore dynamic_configuration if DCS was accidentally wiped.

  3. Loading of configuration file in the old format and converting it into new format.

  4. Mimicking some dict interfaces to make it possible to work with it as with the old config object.

Variables:

  • PATRONI_CONFIG_VARIABLE – name of the environment variable that can be used to load Patroni configuration from.

  • __CACHE_FILENAME – name of the file used to cache dynamic configuration under Postgres data directory.

  • __DEFAULT_CONFIG – default configuration values for some Patroni settings.

PATRONI_CONFIG_VARIABLE = 'PATRONI_CONFIGURATION'
__CACHE_FILENAME = 'patroni.dynamic.json'
__DEFAULT_CONFIG: Dict[str, Any] = {'loop_wait': 10, 'postgresql': {'parameters': <CaseInsensitiveDict{'wal_level': 'hot_standby', 'hot_standby': 'on', 'max_connections': 100, 'max_wal_senders': 10, 'max_prepared_transactions': 0, 'max_locks_per_transaction': 64, 'track_commit_timestamp': 'off', 'max_replication_slots': 10, 'max_worker_processes': 8, 'wal_log_hints': 'on'} at 7f8bcc185710>, 'use_slots': True}, 'retry_timeout': 10, 'standby_cluster': {'archive_cleanup_command': '', 'create_replica_methods': '', 'host': '', 'port': '', 'primary_slot_name': '', 'recovery_min_apply_delay': '', 'restore_command': ''}, 'ttl': 30}
__get_and_maybe_adjust_int_value(config: Dict[str, Any], param: str, min_value: int) → int

Get, validate and maybe adjust a param integer value from the config dict.

Parameters:

  • config – dict object with new global configuration.

  • param – name of the configuration parameter we want to read/validate/adjust.

  • min_value – the minimum possible value that a given param could have.

Returns:

an integer value which corresponds to a provided param.

__init__(configfile: str, validator: ~typing.Callable[[~typing.Dict[str, ~typing.Any]], ~typing.List[str]] | None = <function default_validator>) → NoneView on GitHub

Create a new instance of Config and validate the loaded configuration using validator.

Note

Patroni will read configuration from these locations in this order:

  • file or directory path passed as command-line argument (configfile), if it exists and the file or files found in the directory can be parsed (see _load_config_path()), otherwise

  • YAML file passed via the environment variable (see PATRONI_CONFIG_VARIABLE), if the referenced file exists and can be parsed, otherwise

  • from configuration values defined as environment variables, see _build_environment_configuration().

Parameters:

  • configfile – path to Patroni configuration file.

  • validator – function used to validate Patroni configuration. It should receive a dictionary which represents Patroni configuration, and return a list of zero or more error messages based on validation.

Raises:

ConfigParseError: if any issue is reported by validator.

_build_effective_configuration(dynamic_configuration: Dict[str, Any], local_configuration: Dict[str, Dict[str, Any] | Any]) → Dict[str, Any]View on GitHub

Build effective configuration by merging dynamic_configuration and local_configuration.

Note

local_configuration takes precedence over dynamic_configuration if a setting is defined in both.

Parameters:

  • dynamic_configuration – Patroni dynamic configuration.

  • local_configuration – Patroni local configuration.

Returns:

_description_

static _build_environment_configuration() → Dict[str, Any]View on GitHub

Get local configuration settings that were specified through environment variables.

Returns:

dictionary containing the found environment variables and their values, respecting the expected structure of Patroni configuration.

_load_cache() → NoneView on GitHub

Load dynamic configuration from patroni.dynamic.json.

_load_config_file() → Dict[str, Any]View on GitHub

Load configuration file(s) from filesystem and apply values which were set via environment variables.

Returns:

final configuration after merging configuration file(s) and environment variables.

_load_config_path(path: str) → Dict[str, Any]View on GitHub

Load Patroni configuration file(s) from path.

If path is a file, load the yml file pointed to by path. If path is a directory, load all yml files in that directory in alphabetical order.

Parameters:

path – path to either an YAML configuration file, or to a folder containing YAML configuration files.

Returns:

configuration after reading the configuration file(s) from path.

Raises:

ConfigParseError: if path is invalid.

static _process_postgresql_parameters(parameters: Dict[str, Any], is_local: bool = False) → Dict[str, Any]View on GitHub

Process Postgres parameters.

Note

If is_local configuration discard any setting from parameters that is listed under CMDLINE_OPTIONS as those are supposed to be set only through dynamic configuration.

When setting parameters from CMDLINE_OPTIONS through dynamic configuration their value will be validated as per the validator defined in that very same attribute entry. If the given value cannot be validated, a warning will be logged and the default value of the GUC will be used instead.

Some parameters from CMDLINE_OPTIONS cannot be set even if not is_local configuration:

  • listen_addresses: inferred from postgresql.listen local configuration or from

    PATRONI_POSTGRESQL_LISTEN environment variable;

  • port: inferred from postgresql.listen local configuration or from

    PATRONI_POSTGRESQL_LISTEN environment variable;

  • cluster_name: set through scope local configuration or through PATRONI_SCOPE environment

    variable;

  • hot_standby: always enabled;

  • wal_log_hints: always enabled.

Parameters:

  • parameters – Postgres parameters to be processed. Should be the parsed YAML value of postgresql.parameters configuration, either from local or from dynamic configuration.

  • is_local – should be True if parameters refers to local configuration, or False if parameters refers to dynamic configuration.

Returns:

new value for postgresql.parameters after processing and validating parameters.

_safe_copy_dynamic_configuration(dynamic_configuration: Dict[str, Any]) → Dict[str, Any]View on GitHub

Create a copy of dynamic_configuration.

Merge dynamic_configuration with __DEFAULT_CONFIG (dynamic_configuration takes precedence), and process postgresql.parameters from dynamic_configuration through _process_postgresql_parameters(), if present.

Note

The following settings are not allowed in postgresql section as they are intended to be local configuration, and are removed if present:

  • connect_address;

  • proxy_address;

  • listen;

  • config_dir;

  • data_dir;

  • pgpass;

  • authentication;

Besides that any setting present in dynamic_configuration but absent from __DEFAULT_CONFIG is discarded.

Parameters:

dynamic_configuration – Patroni dynamic configuration.

Returns:

copy of dynamic_configuration, merged with default dynamic configuration and with some sanity checks performed over it.

_validate_and_adjust_timeouts(config: Dict[str, Any]) → NoneView on GitHub

Validate and adjust loop_wait, retry_timeout, and ttl values if necessary.

Minimum values:

  • loop_wait: 1 second;

  • retry_timeout: 3 seconds.

  • ttl: 20 seconds;

Maximum values: In case if values don’t fulfill the following rule, retry_timeout and loop_wait are reduced so that the rule is fulfilled:

loop_wait + 2 * retry_timeout <= ttl
Parameters:

config – dict object with new global configuration.

_validate_failover_tags() → NoneView on GitHub

Check nofailover/failover_priority config and warn user if it’s contradictory.

Note

To preserve sanity (and backwards compatibility) the nofailover tag will still exist. A contradictory configuration is one where nofailover is True but failover_priority > 0, or where nofailover is False, but failover_priority <= 0. Essentially, nofailover and failover_priority are communicating different things. This checks for this edge case (which is a misconfiguration on the part of the user) and warns them. The behaviour is as if failover_priority were not provided (i.e nofailover is the bedrock source of truth)

property config_file: str | None

Path to Patroni configuration file, if any, else None.

copy() → Dict[str, Any]View on GitHub

Get a deep copy of effective Patroni configuration.

Returns:

a deep copy of the Patroni configuration.

property dynamic_configuration: Dict[str, Any]

Deep copy of cached Patroni dynamic configuration.

get(key: str, default: Any | None = None) → AnyView on GitHub

Get effective value of key setting from Patroni configuration root.

Designed to work the same way as dict.get().

Parameters:

  • key – name of the setting.

  • default – default value if key is not present in the effective configuration.

Returns:

value of key, if present in the effective configuration, otherwise default.

classmethod get_default_config() → Dict[str, Any]View on GitHub

Deep copy default configuration.

Returns:

copy of __DEFAULT_CONFIG

property local_configuration: Dict[str, Any]

Deep copy of cached Patroni local configuration.

Returns:

copy of _local_configuration

reload_local_configuration() → bool | NoneView on GitHub

Reload configuration values from the configuration file(s).

Note

Designed to be used when user applies changes to configuration file(s), so Patroni can use the new values with a reload instead of a restart.

Returns:

True if changes have been detected between current local configuration

save_cache() → NoneView on GitHub

Save dynamic configuration to patroni.dynamic.json under Postgres data directory.

Note

patroni.dynamic.jsonXXXXXX is created as a temporary file and than renamed to patroni.dynamic.json, where XXXXXX is a random suffix.

set_dynamic_configuration(configuration: ClusterConfig | Dict[str, Any]) → boolView on GitHub

Set dynamic configuration values with given configuration.

Parameters:

configuration – new dynamic configuration values. Supports dict for backward compatibility.

Returns:

True if changes have been detected between current dynamic configuration and the new dynamic configuration, False otherwise.

patroni.config.default_validator(conf: Dict[str, Any]) → List[str]View on GitHub

Ensure conf is not empty.

Designed to be used as default validator for Config objects, if no specific validator is provided.

Parameters:

conf – configuration to be validated.

Returns:

an empty list – Config expects the validator to return a list of 0 or more issues found while validating the configuration.

Raises:

ConfigParseError: if conf is empty.