patroni.ctl module

Implement patronictl: a command-line application which utilises the REST API to perform cluster operations.

var CONFIG_DIR_PATH:: path to Patroni configuration directory as per click.get_app_dir() output.
var CONFIG_FILE_PATH:: default path to patronictl.yaml configuration file.
var DCS_DEFAULTS:: auxiliary dictionary to build the DCS section of the configuration file. Mainly used to help parsing --dcs-url command-line option of patronictl.

Note

Most of the patronictl commands (restart/reinit/pause/resume/show-config/edit-config and similar) require the group argument and work only for that specific Citus group. If not specified in the command line the group might be taken from the configuration file. If it is also missing in the configuration file we assume that this is just a normal Patroni cluster (not Citus).

exception patroni.ctl.PatroniCtlException(message: str)View on GitHub 

Bases: ClickException

Raised upon issues faced by patronictl utility.

class patroni.ctl.PatronictlPrettyTable(header: str, *args: Any, **kwargs: Any)View on GitHub 

Bases: PrettyTable

Utilitary class to print pretty tables.

Extend PrettyTable to make it print custom information in the header line. The idea is to print a header line like this:

` + Cluster: batman --------+--------+---------+----+-----------+ `

Instead of the default header line which would contain only dash and plus characters.

__build_header(line: str) → str

Build the custom header line for the table.

Note

Expected to be called only against the very first header line of the table.

Parameters:: line – the original header line.
Returns:: the modified header line.

__init__(header: str, *args: Any, **kwargs: Any) → NoneView on GitHub 

Create a PatronictlPrettyTable instance with the given header.

Parameters:

header – custom string to be put in the first header line of the table.
args – positional arguments to be passed to PrettyTable constructor.
kwargs – keyword arguments to be passed to PrettyTable constructor.

_get_hline() → strView on GitHub 

Get string representation of a header line.

Inject the custom header line, if processing the first header line.

Note

Original implementation for injecting a custom header line, and is used up to prettytable 2.2.0. From prettytable 2.2.0 onwards _stringify_hrule() is used instead.

Returns:: string representing a header line.

property _hrule: str

Get string representation of a header line.

Inject the custom header line, if processing the first header line.

Note

Original implementation for injecting a custom header line, and is used up to prettytable 2.2.0. From prettytable 2.2.0 onwards _stringify_hrule() is used instead.

Returns:: string representing a header line.

_is_first_hline() → boolView on GitHub 

Check if the current line being processed is the very first line of the header.

Returns:: True if processing the first header line, False otherwise.

_set_hline(value: str) → NoneView on GitHub 

Set header line string representation.

Parameters:: value – string representing a header line.

_stringify_hrule(*args: Any, **kwargs: Any) → strView on GitHub 

Get the string representation of a header line.

Inject the custom header line, if processing the first header line.

Note

New implementation for injecting a custom header line, which is used from prettytable 2.2.0 onwards.

Returns:: string representation of a header line.

patroni.ctl._do_failover_or_switchover(action: str, cluster_name: str, group: int | None, switchover_leader: str | None, candidate: str | None, force: bool, scheduled: str | None = None) → NoneView on GitHub 

Perform a failover or a switchover operation in the cluster.

Informational messages are printed in the console during the operation, as well as the list of members before and after the operation, so the user can follow the operation status.

Note

If not able to perform the operation through the REST API, write directly to the DCS as a fall back.

Parameters:

action – action to be taken – failover or switchover.
cluster_name – name of the Patroni cluster.
group – filter Citus group within we should perform a failover or switchover. If None, user will be prompted for filling it – unless force is True, in which case an exception is raised.
switchover_leader – name of the leader member passed as switchover option.
candidate – name of a standby member to be promoted. Nodes that are tagged with nofailover cannot be used.
force – perform the failover or switchover without asking for confirmations.
scheduled – timestamp when the switchover should be scheduled to occur. If now perform immediately.

Raises:

PatroniCtlException: if:

Patroni is running on a Citus cluster, but no group was specified; or
a switchover was requested by the cluster has no leader; or
switchover_leader does not match the current leader of the cluster; or
cluster has no candidates available for the operation; or
no candidate is given for a failover operation; or
current leader and candidate are the same; or
candidate is tagged as nofailover; or
candidate is not a member of the cluster; or
trying to schedule a switchover in a cluster that is in maintenance mode; or
user aborts the operation.

patroni.ctl._get_configuration() → Dict[str, Any]View on GitHub 

Get configuration object.

Returns:: configuration object from the current context.

patroni.ctl.apply_config_changes(before_editing: str, data: Dict[str, Any], kvpairs: List[str]) → Tuple[str, Dict[str, Any]]View on GitHub 

Apply config changes specified as a list of key-value pairs.

Keys are interpreted as dotted paths into the configuration data structure. Except for paths beginning with postgresql.parameters where rest of the path is used directly to allow for PostgreSQL GUCs containing dots. Values are interpreted as YAML values.

Parameters:

before_editing – human representation before editing.
data – configuration data structure.
kvpairs – list of strings containing key value pairs separated by =.

Returns:

tuple of human-readable, parsed data structure after changes.

Raises:

PatroniCtlException: if any entry in kvpairs is None or not in the expected format.

patroni.ctl.apply_yaml_file(data: Dict[str, Any], filename: str) → Tuple[str, Dict[str, Any]]View on GitHub 

Apply changes from a YAML file to configuration.

Parameters:

data – configuration data structure.
filename – name of the YAML file, - is taken to mean standard input.

Returns:

tuple of human-readable and parsed data structure after changes.

patroni.ctl.check_response(response: HTTPResponse, member_name: str, action_name: str, silent_success: bool = False) → boolView on GitHub 

Check an HTTP response and print a status message.

Parameters:

response – the response to be checked.
member_name – name of the member associated with the response.
action_name – action associated with the response.
silent_success – if a status message should be skipped upon a successful response.

Returns:

True if the response indicates a sucessful operation (HTTP status < 400), False otherwise.

patroni.ctl.confirm_members_action(members: List[Member], force: bool, action: str, scheduled_at: datetime | None = None) → NoneView on GitHub 

Ask for confirmation if action should be taken by members.

Parameters:

members – list of member which will take the action.
force – if True skip the confirmation prompt and allow the action to proceed.
action –
the action that is being processed, one among:
- reload: reload PostgreSQL configuration; or
- restart: restart PostgreSQL; or
- reinitialize: reinitialize PostgreSQL data directory; or
- flush: discard scheduled actions.
scheduled_at – timestamp at which the action should be scheduled to. If None action is taken immediately.

Raises:

PatroniCtlException: if the user aborted the action.

patroni.ctl.format_config_for_editing(data: Any, default_flow_style: bool = False) → strView on GitHub 

Format configuration as YAML for human consumption.

Parameters:

data – configuration as nested dictionaries.
default_flow_style – passed down as default_flow_style argument of yaml.safe_dump().

Returns:

unicode YAML of the configuration.

patroni.ctl.format_pg_version(version: int) → strView on GitHub 

Format Postgres version for human consumption.

Parameters:

version – Postgres version represented as an integer.

Returns:

Postgres version represented as a human-readable string.

Example:

>>> format_pg_version(90624)
'9.6.24'

>>> format_pg_version(100000)
'10.0'

>>> format_pg_version(140008)
'14.8'

patroni.ctl.generate_topology(level: int, member: Dict[str, Any], topology: Dict[str | None, List[Dict[str, Any]]]) → Iterator[Dict[str, Any]]View on GitHub 

Recursively yield members with their names adjusted according to their level in the cluster topology.

Note

The idea is to get a tree view of the members when printing their names. For example, suppose you have a cascading replication composed of 3 nodes, say postgresql0, postgresql1, and postgresql2. This function would adjust their names to be like this:

'postgresql0' -> 'postgresql0'
'postgresql1' -> '+ postgresql1'
'postgresql2' -> ' + postgresql2'

So, if you ever print their names line by line, you would see something like this:

postgresql0
+ postgresql1
  + postgresql2

Parameters:

level – the current level being inspected in the topology.
member –
information about the current member being inspected in level of topology. Should countain at least this key: * name: name of the node, according to name configuration;

But may contain others, which although ignored by this function, will be yielded as part of the resulting object. The value of key name is changed as explained in the note.
topology – each key is the name of a node which has at least one replica attached to it. The corresponding value is a list of the attached replicas, each of them with the same structure described for member.

Yields:

the current member with its name changed. Besides that reyield values from recursive calls.

patroni.ctl.get_all_members(cluster: Cluster, group: int | None, role: str = 'leader') → Iterator[Member]View on GitHub 

Get all cluster members that have the given role.

Parameters:

cluster – the Patroni cluster.
group – filter which Citus group we should get members from. If None get from all groups.
role –
role to filter members. Can be one among:
- primary or master: the primary PostgreSQL instance;
- replica or standby: a standby PostgreSQL instance;
- leader: the leader of a Patroni cluster. Can also be used to get the leader of a Patroni standby cluster;
- standby-leader: the leader of a Patroni standby cluster;
- any: matches any node independent of its role.

Yields:

members that have the given role.

patroni.ctl.get_all_members_leader_first(cluster: Cluster) → Iterator[Member]View on GitHub 

Get all cluster members, with the cluster leader being yielded first.

Note

Only yield members that have a restapi.connect_address configured.

Yields:: all cluster members, with the leader first.

patroni.ctl.get_any_member(cluster: Cluster, group: int | None, role: str | None = None, member: str | None = None) → Member | NoneView on GitHub 

Get the first found cluster member that has the given role.

Parameters:

cluster – the Patroni cluster.
group – filter which Citus group we should get members from. If None get from all groups.
role – role to filter members. See get_all_members() for available options.
member – if specified, then besides having the given role, the cluster member’s name should be member.

Returns:

the first found cluster member that has the given role.

Raises:

PatroniCtlException: if both role and member are provided.

patroni.ctl.get_cluster_service_info(cluster: Dict[str, Any]) → List[str]View on GitHub 

Get complementary information about the cluster.

Parameters:

cluster – a Patroni cluster represented as an object created through cluster_as_json().

Returns:

a list of 0 or more informational messages. They can be about:

Cluster in maintenance mode;
Scheduled switchovers.

Get a cursor object to execute queries against a member that has the given role or member_name.

Note

Besides what is passed through connect_parameters, this function also sets the following parameters:

fallback_application_name: as Patroni ctl;
connect_timeout: as 5.

Parameters:

cluster – the Patroni cluster.
group – filter which Citus group we should get members to create a cursor against. If None consider members from all groups.
connect_parameters – database connection parameters.
role – role to filter members. See get_all_members() for available options.
member_name – if specified, then besides having the given role, the cluster member’s name should be member_name.

Returns:

a cursor object to execute queries against the database. Can be either:

A psycopg.Cursor if using psycopg; or
A psycopg2.extensions.cursor if using psycopg2;
None if not able to get a cursor that attendees role and member_name.

patroni.ctl.get_dcs(scope: str, group: int | None) → AbstractDCSView on GitHub 

Get the DCS object.

Parameters:

scope – cluster name.
group – if group is defined, use it to select which alternative Citus group this DCS refers to. If group is None and a Citus configuration exists, assume this is the coordinator. Coordinator has the group 0. Refer to the module note for more details.

Returns:

a subclass of AbstractDCS, according to the DCS technology that is configured.

Raises:

PatroniCtlException: if not suitable DCS configuration could be found.

patroni.ctl.get_members(cluster: Cluster, cluster_name: str, member_names: List[str], role: str, force: bool, action: str, ask_confirmation: bool = True, group: int | None = None) → List[Member]View on GitHub 

Get the list of members based on the given filters.

Note

Contain some filtering and checks processing that are common to several actions that are exposed by patronictl, like:

Get members of cluster that respect the given member_names, role, and group;

Bypass confirmations;

Prompt user for information that has not been passed through the command-line options;

etc.

Designed to handle both attended and unattended patronictl commands execution that need to retrieve and validate the members before doing anything.

In the very end may call confirm_members_action() to ask if the user would like to proceed with action over the retrieved members. That won’t actually perform the action, but it works as the “last confirmation” before the action is processed by the caller method.

Additional checks can also be implemented in the caller method, in which case you might want to pass ask_confirmation=False, and later call confirm_members_action() manually in the caller method. That way the workflow won’t look broken to the user that is interacting with patronictl.

Parameters:

cluster – Patroni cluster.
cluster_name – name of the Patroni cluster.
member_names – used to filter which members should take the action based on their names. Each item is the name of a Patroni member, as per name configuration. If member_names is an empty tuple no filters are applied based on names.
role – used to filter which members should take the action based on their role. See get_all_members() for available options.
force – if True, then it won’t ask for confirmations at any point nor prompt the user to select values for options that were not specified through the command-line.
action –
the action that is being processed, one among:
- reload: reload PostgreSQL configuration; or
- restart: restart PostgreSQL; or
- reinitialize: reinitialize PostgreSQL data directory; or
- flush: discard scheduled actions.
ask_confirmation – if False, then it won’t ask for the final confirmation regarding the action before returning the list of members. Usually useful as False if you want to perform additional checks in the caller method besides the checks that are performed through this generic method.
group – filter which Citus group we should get members from. If None consider members from all groups.

Returns:

a list of members that respect the given filters.

Raises:

PatroniCtlException: if

Cluster does not have members that match the given role; or
Cluster does not have members that match the given member_names; or
No member with given role is found among the specified member_names.

patroni.ctl.invoke_editor(before_editing: str, cluster_name: str) → Tuple[str, Dict[str, Any]]View on GitHub 

Start editor command to edit configuration in human readable format.

Note

Requires an editor program, and uses first found among:

Program given by EDITOR environemnt variable; or
editor; or
vi.

Parameters:

before_editing – human representation before editing.
cluster_name – name of the Patroni cluster.

Returns:

tuple of human-readable, parsed data structure after changes.

Raises:

PatroniCtlException: if

No suitable editor can be found; or
Editor call exits with unexpected return code.

patroni.ctl.is_citus_cluster() → boolView on GitHub 

Check if we are working with Citus cluster.

Returns:: True if configuration has citus section, otherwise False.

patroni.ctl.load_config(path: str, dcs_url: str | None) → Dict[str, Any]View on GitHub 

Load configuration file from path and optionally override its DCS configuration with dcs_url.

Parameters:

path – path to the configuration file.
dcs_url – the DCS URL in the format DCS://HOST:PORT/NAMESPACE, e.g. etcd3://random.com:2399/service. If given, override whatever DCS and namespace that are set in the configuration file. See parse_dcs() for more information.

Returns:

a dictionary representing the configuration.

Raises:

PatroniCtlException: if path does not exist or is not readable.

patroni.ctl.output_members(cluster: Cluster, name: str, extended: bool = False, fmt: str = 'pretty', group: int | None = None) → NoneView on GitHub 

Print information about the Patroni cluster and its members.

Information is printed to console through print_output(), and contains:

Cluster: name of the Patroni cluster, as per scope configuration;

Member: name of the Patroni node, as per name configuration;

Host: hostname (or IP) and port, as per postgresql.listen configuration;

Role: Leader, Standby Leader, Sync Standby or Replica;

State: stopping, stopped, stop failed, crashed, running, starting, start failed, restarting, restart failed, initializing new cluster, initdb failed, running custom bootstrap script, custom bootstrap failed, creating replica, streaming, in archive recovery, and so on;

TL: current timeline in Postgres; Lag in MB: replication lag.

Besides that it may also have:

Group: Citus group ID – showed only if Citus is enabled.
Pending restart: if the node is pending a restart – showed only if extended;
Scheduled restart: timestamp for scheduled restart, if any – showed only if extended;
Tags: node tags, if any – showed only if extended.

The 3 extended columns are always included if extended, even if the member has no value for a given column. If not extended, these columns may still be shown if any of the members has any information for them.

Parameters:

cluster – Patroni cluster.
name – name of the Patroni cluster.
extended – if extended information (pending restarts, scheduled restarts, node tags) should be printed, if available.
fmt – the output table printing format. See print_output() for available options. If fmt is neither topology nor pretty, then complementary information gathered through get_cluster_service_info() is not printed.
group – filter which Citus group we should get members from. If None get from all groups.

patroni.ctl.parse_dcs(dcs: str | None) → Dict[str, Any] | NoneView on GitHub 

Parse a DCS URL.

Parameters:

dcs –

the DCS URL in the format DCS://HOST:PORT/NAMESPACE. DCS can be one among:

consul
etcd
etcd3
exhibitor
zookeeper

If DCS is not specified, assume etcd by default. If HOST is not specified, assume localhost by default. If PORT is not specified, assume the default port of the given DCS. If NAMESPACE is not specified, use whatever is in config.

Returns:

None if dcs is None, otherwise a dictionary. The dictionary represents dcs as if it were parsed from the Patroni configuration file. Additionally, if a namespace is specified in dcs, return a namespace key with the parsed value.

Raises:

PatroniCtlException: if the DCS name in dcs is not valid.

Example:

>>> parse_dcs('')
{'etcd': {'host': 'localhost:2379'}}

>>> parse_dcs('etcd://:2399')
{'etcd': {'host': 'localhost:2399'}}

>>> parse_dcs('etcd://test')
{'etcd': {'host': 'test:2379'}}

>>> parse_dcs('etcd3://random.com:2399')
{'etcd3': {'host': 'random.com:2399'}}

>>> parse_dcs('etcd3://random.com:2399/customnamespace')
{'etcd3': {'host': 'random.com:2399'}, 'namespace': '/customnamespace'}

patroni.ctl.parse_scheduled(scheduled: str | None) → datetime | NoneView on GitHub 

Parse a string scheduled timestamp as a datetime object.

Parameters:

scheduled – string representation of the timestamp. May also be now.

Returns:

the corresponding datetime object, if scheduled is not now, otherwise None.

Raises:

PatroniCtlException: if unable to parse scheduled from str to datetime.

Example:

>>> parse_scheduled(None) is None
True

>>> parse_scheduled('now') is None
True

>>> parse_scheduled('2023-05-29T04:32:31')
datetime.datetime(2023, 5, 29, 4, 32, 31, tzinfo=tzlocal())

>>> parse_scheduled('2023-05-29T04:32:31-3')
datetime.datetime(2023, 5, 29, 4, 32, 31, tzinfo=tzoffset(None, -10800))

patroni.ctl.print_output(columns: List[str] | None, rows: List[List[Any]], alignment: Dict[str, str] | None = None, fmt: str = 'pretty', header: str = '', delimiter: str = '\t') → NoneView on GitHub 

Print tabular information.

Parameters:

columns – list of column names.
rows – list of rows. Each item is a list of values for the columns.
alignment –
alignment to be applied to column values. Each key is the name of a column to be aligned, and the corresponding value can be one among:
- l: left-aligned
- c: center-aligned
- r: right-aligned
A key in the dictionary is only required for a column that needs a specific alignment. Only apply when fmt is either pretty or topology.
fmt –
the printing format. Can be one among:
- json: to print as a JSON string – array of objects;
- yaml or yml: to print as a YAML string;
- tsv: to print a table of separated values, by default by tab;
- pretty: to print a pretty table;
- topology: similar to pretty, but with a topology view when printing cluster members.
header – a string to be included in the first line of the table header, typically the cluster name. Only apply when fmt is either pretty or topology.
delimiter – the character to be used as delimiter when fmt is tsv.

Execute SQL command against a member.

Parameters:

cluster – the Patroni cluster.
group – filter which Citus group we should get members from to perform the query. Refer to the module note for more details.
cursor – cursor through which command is executed. If None a new cursor is instantiated through get_cursor().
member – filter which member to create a cursor against based on its name, if cursor is None.
role – filter which member to create a cursor against based on their role, if cursor is None. See get_all_members() for available options.
command – SQL command to be executed.
connect_parameters – connection parameters to be passed down to get_cursor(), if cursor is None.

Returns:

a tuple composed of two items:

List of rows returned by the executed command;
List of columns related to the rows returned by the executed command.

If an error occurs while executing command, then returns the following values in the tuple:

List with 2 items:
- Current timestamp;
- Error message.
None.

patroni.ctl.request_patroni(member: Member, method: str = 'GET', endpoint: str | None = None, data: Any | None = None) → HTTPResponseView on GitHub 

Perform a request to Patroni REST API.

Parameters:

member – DCS member, used to get the base URL of its REST API server.
method – HTTP method to be used, e.g. GET.
endpoint – URL path of the request, e.g. patroni.
data – anything to be used as the request body.

Returns:

the response for the request.

patroni.ctl.show_diff(before_editing: str, after_editing: str) → NoneView on GitHub 

Show a diff between two strings.

Inputs are expected to be unicode strings.

If the output is to a tty the diff will be colored.

Note

If tty it requires a pager program, and uses first found among:

Program given by PAGER environment variable; or
less; or
more.

Parameters:

before_editing – string to be compared with after_editing.
after_editing – string to be compared with before_editing.

Raises:

PatroniCtlException: if no suitable pager can be found when printing diff output to a tty.

patroni.ctl.temporary_file(contents: bytes, suffix: str = '', prefix: str = 'tmp') → Iterator[str]View on GitHub 

Create a temporary file with specified contents that persists for the context.

Parameters:

contents – binary string that will be written to the file.
prefix – will be prefixed to the filename.
suffix – will be appended to the filename.

Yields:

path of the created file.

patroni.ctl.timestamp(precision: int = 6) → strView on GitHub 

Get current timestamp with given precision as a string.

Parameters:: precision – Amount of digits to be present in the precision.
Returns:: the current timestamp with given precision.

patroni.ctl.toggle_pause(cluster_name: str, group: int | None, paused: bool, wait: bool) → NoneView on GitHub 

Toggle the pause state in the cluster members.

Parameters:

cluster_name – name of the Patroni cluster.
group – filter which Citus group we should toggle the pause state of. Refer to the module note for more details.
paused – the desired state for pause in all nodes.
wait – True if it should block until the operation is finished or false for returning immediately.

Raises:

PatroniCtlException: if

pause state is already paused; or
cluster contains no accessible members.

patroni.ctl.topology_sort(members: List[Dict[str, Any]]) → Iterator[Dict[str, Any]]View on GitHub 

Sort members according to their level in the replication topology tree.

Parameters:

members –

list of members in the cluster. Each item should countain at least these keys:

name: name of the node, according to name configuration;
role: leader, standby_leader or replica.

Cascading replicas are identified through tags -> replicatefrom value – if that is set, and they are in fact attached to another replica.

Besides name, role and tags keys, it may contain other keys, which although ignored by this function, will be yielded as part of the resulting object. The value of key name is changed through generate_topology().

Yields:

members sorted by level in the topology, and with a new name value according to their level in the topology.

patroni.ctl.wait_until_pause_is_applied(dcs: AbstractDCS, paused: bool, old_cluster: Cluster) → NoneView on GitHub 

Wait for all members in the cluster to have pause state set to paused.

Parameters:

dcs – DCS object from where to get fresh cluster information.
paused – the desired state for pause in all nodes.
old_cluster – original cluster information before pause or unpause has been requested. Used to report which nodes are still pending to have pause equal paused at a given point in time.

patroni.ctl.watching(w: bool, watch: int | None, max_count: int | None = None, clear: bool = True) → Iterator[int]View on GitHub 

Yield a value every watch seconds.

Used to run a command with a watch-based approach.

Parameters:

w – if True and watch is None, then watch assumes the value 2.
watch – amount of seconds to wait before yielding another value.
max_count – maximum number of yielded values. If None keep yielding values indefinitely.
clear – if the screen should be cleared out at each iteration.

Yields:

0 each time watch seconds have passed.

Example:

>>> len(list(watching(True, 1, 0)))
1
>>> len(list(watching(True, 1, 1)))
2
>>> len(list(watching(True, None, 0)))
1