patroni.config module
Facilities related to Patroni configuration.
- class patroni.config.Config(configfile: str, validator: ~typing.Callable[[~typing.Dict[str, ~typing.Any]], ~typing.List[str]] | None = <function default_validator>)View on GitHub
Bases:
object
Handle Patroni configuration.
This class is responsible for:
Building and giving access to
effective_configuration
from:Config.__DEFAULT_CONFIG
– some sane default values;dynamic_configuration
– configuration stored in DCS;local_configuration
– configuration from config.yml or environment.
Saving and loading
dynamic_configuration
into ‘patroni.dynamic.json’ file located in local_configuration[‘postgresql’][‘data_dir’] directory. This is necessary to be able to restoredynamic_configuration
if DCS was accidentally wiped.Loading of configuration file in the old format and converting it into new format.
Mimicking some
dict
interfaces to make it possible to work with it as with the oldconfig
object.
- Variables:
PATRONI_CONFIG_VARIABLE – name of the environment variable that can be used to load Patroni configuration from.
__CACHE_FILENAME – name of the file used to cache dynamic configuration under Postgres data directory.
__DEFAULT_CONFIG – default configuration values for some Patroni settings.
- PATRONI_CONFIG_VARIABLE = 'PATRONI_CONFIGURATION'
- __CACHE_FILENAME = 'patroni.dynamic.json'
- __DEFAULT_CONFIG: Dict[str, Any] = {'loop_wait': 10, 'postgresql': {'parameters': <CaseInsensitiveDict{'wal_level': 'hot_standby', 'hot_standby': 'on', 'max_connections': 100, 'max_wal_senders': 10, 'max_prepared_transactions': 0, 'max_locks_per_transaction': 64, 'track_commit_timestamp': 'off', 'max_replication_slots': 10, 'max_worker_processes': 8, 'wal_log_hints': 'on'} at 7fa1cc96a5d0>, 'use_slots': True}, 'retry_timeout': 10, 'standby_cluster': {'archive_cleanup_command': '', 'create_replica_methods': '', 'host': '', 'port': '', 'primary_slot_name': '', 'recovery_min_apply_delay': '', 'restore_command': ''}, 'ttl': 30}
- __get_and_maybe_adjust_int_value(config: Dict[str, Any], param: str, min_value: int) int
Get, validate and maybe adjust a param integer value from the config
dict
.- Parameters:
config –
dict
object with new global configuration.param – name of the configuration parameter we want to read/validate/adjust.
min_value – the minimum possible value that a given param could have.
- Returns:
an integer value which corresponds to a provided param.
- __init__(configfile: str, validator: ~typing.Callable[[~typing.Dict[str, ~typing.Any]], ~typing.List[str]] | None = <function default_validator>) None View on GitHub
Create a new instance of
Config
and validate the loaded configuration using validator.Note
Patroni will read configuration from these locations in this order:
file or directory path passed as command-line argument (configfile), if it exists and the file or files found in the directory can be parsed (see
_load_config_path()
), otherwiseYAML file passed via the environment variable (see
PATRONI_CONFIG_VARIABLE
), if the referenced file exists and can be parsed, otherwisefrom configuration values defined as environment variables, see
_build_environment_configuration()
.
- Parameters:
configfile – path to Patroni configuration file.
validator – function used to validate Patroni configuration. It should receive a dictionary which represents Patroni configuration, and return a list of zero or more error messages based on validation.
- Raises:
ConfigParseError
: if any issue is reported by validator.
- _build_effective_configuration(dynamic_configuration: Dict[str, Any], local_configuration: Dict[str, Dict[str, Any] | Any]) Dict[str, Any] View on GitHub
Build effective configuration by merging dynamic_configuration and local_configuration.
Note
local_configuration takes precedence over dynamic_configuration if a setting is defined in both.
- Parameters:
dynamic_configuration – Patroni dynamic configuration.
local_configuration – Patroni local configuration.
- Returns:
_description_
- static _build_environment_configuration() Dict[str, Any] View on GitHub
Get local configuration settings that were specified through environment variables.
- Returns:
dictionary containing the found environment variables and their values, respecting the expected structure of Patroni configuration.
- _load_cache() None View on GitHub
Load dynamic configuration from
patroni.dynamic.json
.
- _load_config_file() Dict[str, Any] View on GitHub
Load configuration file(s) from filesystem and apply values which were set via environment variables.
- Returns:
final configuration after merging configuration file(s) and environment variables.
- _load_config_path(path: str) Dict[str, Any] View on GitHub
Load Patroni configuration file(s) from path.
If path is a file, load the yml file pointed to by path. If path is a directory, load all yml files in that directory in alphabetical order.
- Parameters:
path – path to either an YAML configuration file, or to a folder containing YAML configuration files.
- Returns:
configuration after reading the configuration file(s) from path.
- Raises:
ConfigParseError
: if path is invalid.
- static _process_postgresql_parameters(parameters: Dict[str, Any], is_local: bool = False) Dict[str, Any] View on GitHub
Process Postgres parameters.
Note
If is_local configuration discard any setting from parameters that is listed under
CMDLINE_OPTIONS
as those are supposed to be set only through dynamic configuration.When setting parameters from
CMDLINE_OPTIONS
through dynamic configuration their value will be validated as per the validator defined in that very same attribute entry. If the given value cannot be validated, a warning will be logged and the default value of the GUC will be used instead.Some parameters from
CMDLINE_OPTIONS
cannot be set even if not is_local configuration:listen_addresses
: inferred frompostgresql.listen
local configuration or fromPATRONI_POSTGRESQL_LISTEN
environment variable;
port
: inferred frompostgresql.listen
local configuration or fromPATRONI_POSTGRESQL_LISTEN
environment variable;
cluster_name
: set throughscope
local configuration or throughPATRONI_SCOPE
environmentvariable;
hot_standby
: always enabled;wal_log_hints
: always enabled.
- Parameters:
parameters – Postgres parameters to be processed. Should be the parsed YAML value of
postgresql.parameters
configuration, either from local or from dynamic configuration.is_local – should be
True
if parameters refers to local configuration, orFalse
if parameters refers to dynamic configuration.
- Returns:
new value for
postgresql.parameters
after processing and validating parameters.
- _safe_copy_dynamic_configuration(dynamic_configuration: Dict[str, Any]) Dict[str, Any] View on GitHub
Create a copy of dynamic_configuration.
Merge dynamic_configuration with
__DEFAULT_CONFIG
(dynamic_configuration takes precedence), and processpostgresql.parameters
from dynamic_configuration through_process_postgresql_parameters()
, if present.Note
The following settings are not allowed in
postgresql
section as they are intended to be local configuration, and are removed if present:connect_address
;proxy_address
;listen
;config_dir
;data_dir
;pgpass
;authentication
;
Besides that any setting present in dynamic_configuration but absent from
__DEFAULT_CONFIG
is discarded.- Parameters:
dynamic_configuration – Patroni dynamic configuration.
- Returns:
copy of dynamic_configuration, merged with default dynamic configuration and with some sanity checks performed over it.
- _validate_and_adjust_timeouts(config: Dict[str, Any]) None View on GitHub
Validate and adjust
loop_wait
,retry_timeout
, andttl
values if necessary.Minimum values:
loop_wait
: 1 second;retry_timeout
: 3 seconds.ttl
: 20 seconds;
Maximum values: In case if values don’t fulfill the following rule,
retry_timeout
andloop_wait
are reduced so that the rule is fulfilled:loop_wait + 2 * retry_timeout <= ttl
- Parameters:
config –
dict
object with new global configuration.
- _validate_failover_tags() None View on GitHub
Check
nofailover
/failover_priority
config and warn user if it’s contradictory.Note
To preserve sanity (and backwards compatibility) the
nofailover
tag will still exist. A contradictory configuration is one wherenofailover
isTrue
butfailover_priority > 0
, or wherenofailover
isFalse
, butfailover_priority <= 0
. Essentially,nofailover
andfailover_priority
are communicating different things. This checks for this edge case (which is a misconfiguration on the part of the user) and warns them. The behaviour is as iffailover_priority
were not provided (i.enofailover
is the bedrock source of truth)
- copy() Dict[str, Any] View on GitHub
Get a deep copy of effective Patroni configuration.
- Returns:
a deep copy of the Patroni configuration.
- get(key: str, default: Any | None = None) Any View on GitHub
Get effective value of
key
setting from Patroni configuration root.Designed to work the same way as
dict.get()
.- Parameters:
key – name of the setting.
default – default value if key is not present in the effective configuration.
- Returns:
value of key, if present in the effective configuration, otherwise default.
- classmethod get_default_config() Dict[str, Any] View on GitHub
Deep copy default configuration.
- Returns:
copy of
__DEFAULT_CONFIG
- property local_configuration: Dict[str, Any]
Deep copy of cached Patroni local configuration.
- Returns:
copy of
_local_configuration
- reload_local_configuration() bool | None View on GitHub
Reload configuration values from the configuration file(s).
Note
Designed to be used when user applies changes to configuration file(s), so Patroni can use the new values with a reload instead of a restart.
- Returns:
True
if changes have been detected between current local configuration
- save_cache() None View on GitHub
Save dynamic configuration to
patroni.dynamic.json
under Postgres data directory.Note
patroni.dynamic.jsonXXXXXX
is created as a temporary file and than renamed topatroni.dynamic.json
, whereXXXXXX
is a random suffix.
- set_dynamic_configuration(configuration: ClusterConfig | Dict[str, Any]) bool View on GitHub
Set dynamic configuration values with given configuration.
- Parameters:
configuration – new dynamic configuration values. Supports
dict
for backward compatibility.- Returns:
True
if changes have been detected between current dynamic configuration and the new dynamic configuration,False
otherwise.
- patroni.config.default_validator(conf: Dict[str, Any]) List[str] View on GitHub
Ensure conf is not empty.
Designed to be used as default validator for
Config
objects, if no specific validator is provided.- Parameters:
conf – configuration to be validated.
- Returns:
an empty list –
Config
expects the validator to return a list of 0 or more issues found while validating the configuration.- Raises:
ConfigParseError
: if conf is empty.