patroni.postgresql.mpp package

Submodules

• patroni.postgresql.mpp.citus module
  • Citus
    • Citus._abc_impl
    • Citus.coordinator_group_id
    • Citus.group
    • Citus.group_re
    • Citus.validate_config()
  • CitusHandler
    • CitusHandler.__init__()
    • CitusHandler._abc_impl
    • CitusHandler._add_task()
    • CitusHandler._in_flight
    • CitusHandler._pg_dist_node
    • CitusHandler._tasks
    • CitusHandler.add_task()
    • CitusHandler.adjust_postgres_gucs()
    • CitusHandler.bootstrap()
    • CitusHandler.find_task_by_group()
    • CitusHandler.handle_event()
    • CitusHandler.ignore_replication_slot()
    • CitusHandler.load_pg_dist_node()
    • CitusHandler.on_demote()
    • CitusHandler.pick_task()
    • CitusHandler.process_task()
    • CitusHandler.process_tasks()
    • CitusHandler.query()
    • CitusHandler.run()
    • CitusHandler.schedule_cache_rebuild()
    • CitusHandler.sync_meta_data()
    • CitusHandler.update_node()
  • PgDistNode
    • PgDistNode.__init__()
    • PgDistNode.deadline
    • PgDistNode.wait()
    • PgDistNode.wakeup()

Module contents

Abstract classes for MPP handler.

MPP stands for Massively Parallel Processing, and Citus belongs to this architecture. Currently, Citus is the only supported MPP cluster. However, we may consider adapting other databases such as TimescaleDB, GPDB, etc. into Patroni.

class patroni.postgresql.mpp.AbstractMPP(config: Dict[str, str | int])View on GitHub

Bases: ABC

An abstract class which should be passed to AbstractDCS.

Note

We create AbstractMPP and AbstractMPPHandler to solve the chicken-egg initialization problem. When initializing DCS, we dynamically create an object implementing AbstractMPP, later this object is used to instantiate an object implementing AbstractMPPHandler.

__init__(config: Dict[str, str | int]) → NoneView on GitHub

Init method for AbstractMPP.

Parameters:

config – configuration of MPP section.

_abc_impl = <_abc._abc_data object>

_get_handler_cls() → Iterator[Type[AbstractMPPHandler]]View on GitHub

Find Handler classes inherited from a class type of this object.

Yields:

handler classes for this object.

abstract property coordinator_group_id: AnyView on GitHub

The group id of the coordinator PostgreSQL cluster.

get_handler_impl(postgresql: Postgresql) → AbstractMPPHandlerView on GitHub

Find and instantiate Handler implementation of this object.

Parameters:

postgresql – a reference to Postgresql object.

Raises:

PatroniException: if the Handler class haven’t been found.

Returns:

an instantiated class that implements Handler for this object.

abstract property group: AnyView on GitHub

The group for a given MPP implementation.

group_re: Any

is_coordinator() → boolView on GitHub

Check whether this node is running in the coordinator PostgreSQL cluster.

Returns:

True if MPP is enabled and the group id of this node matches with the coordinator_group_id, otherwise False.

is_enabled() → boolView on GitHub

Check if MPP is enabled for a given MPP.

Note

We just check that the _config object isn’t empty and expect it to be empty only in case of Null.

Returns:

True if MPP is enabled, otherwise False.

is_worker() → boolView on GitHub

Check whether this node is running as a MPP worker PostgreSQL cluster.

Returns:

True if MPP is enabled and this node is known to be not running as the coordinator PostgreSQL cluster, otherwise False.

property k8s_group_labelView on GitHub

Group label used for kubernetes DCS of the MPP cluster.

Returns:

A string representation of the k8s group label of a given MPP implementation.

property type: strView on GitHub

The type of the MPP cluster.

Returns:

A string representation of the type of a given MPP implementation.

abstractmethod static validate_config(config: Any) → boolView on GitHub

Check whether provided config is good for a given MPP.

Parameters:

config – configuration of MPP section.

Returns:

True is config passes validation, otherwise False.

class patroni.postgresql.mpp.AbstractMPPHandler(postgresql: Postgresql, config: Dict[str, str | int])View on GitHub

Bases: AbstractMPP

An abstract class which defines interfaces that should be implemented by real handlers.

__init__(postgresql: Postgresql, config: Dict[str, str | int]) → NoneView on GitHub

Init method for AbstractMPPHandler.

Parameters:

• postgresql – a reference to Postgresql object.

• config – configuration of MPP section.

_abc_impl = <_abc._abc_data object>

abstractmethod adjust_postgres_gucs(parameters: Dict[str, Any]) → NoneView on GitHub

Adjust GUCs in the current PostgreSQL configuration.

Parameters:

parameters – dictionary of GUCs, with key as GUC name and the corresponding value as current GUC value.

abstractmethod bootstrap() → NoneView on GitHub

Bootstrap handler.

Is called when the new cluster is initialized (through initdb or a custom bootstrap method).

abstractmethod handle_event(cluster: Cluster, event: Dict[str, Any]) → NoneView on GitHub

Handle an event sent from a worker node.

Parameters:

• cluster – the currently known cluster state from DCS.

• event – the event to be handled.

abstractmethod ignore_replication_slot(slot: Dict[str, str]) → boolView on GitHub

Check whether provided replication slot existing in the database should not be removed.

Note

MPP database may create replication slots for its own use, for example to migrate data between workers using logical replication, and we don’t want to suddenly drop them.

Parameters:

slot – dictionary containing the replication slot settings, like name, database, type, and plugin.

Returns:

True if the replication slots should not be removed, otherwise False.

abstractmethod on_demote() → NoneView on GitHub

On demote handler.

Is called when the primary was demoted.

abstractmethod schedule_cache_rebuild() → NoneView on GitHub

Cache rebuild handler.

Is called to notify handler that it has to refresh its metadata cache from the database.

abstractmethod sync_meta_data(cluster: Cluster) → NoneView on GitHub

Sync meta data on the coordinator.

Parameters:

cluster – the currently known cluster state from DCS.

class patroni.postgresql.mpp.NullView on GitHub

Bases: AbstractMPP

Dummy implementation of AbstractMPP.

__init__() → NoneView on GitHub

Init method for Null.

_abc_impl = <_abc._abc_data object>

property coordinator_group_id: NoneView on GitHub

The group id of the coordinator PostgreSQL cluster.

Returns:

always None.

property group: NoneView on GitHub

The group for Null.

Returns:

always None.

static validate_config(config: Any) → boolView on GitHub

Check whether provided config is good for Null.

Returns:

always True.

class patroni.postgresql.mpp.NullHandler(postgresql: Postgresql, config: Dict[str, str | int])View on GitHub

Bases: Null, AbstractMPPHandler

Dummy implementation of AbstractMPPHandler.

__init__(postgresql: Postgresql, config: Dict[str, str | int]) → NoneView on GitHub

Init method for NullHandler.

Parameters:

• postgresql – a reference to Postgresql object.

• config – configuration of MPP section.

_abc_impl = <_abc._abc_data object>

adjust_postgres_gucs(parameters: Dict[str, Any]) → NoneView on GitHub

Adjust GUCs in the current PostgreSQL configuration.

Parameters:

parameters – dictionary of GUCs, with key as GUC name and corresponding value as current GUC value.

bootstrap() → NoneView on GitHub

Bootstrap handler.

Is called when the new cluster is initialized (through initdb or a custom bootstrap method).

handle_event(cluster: Cluster, event: Dict[str, Any]) → NoneView on GitHub

Handle an event sent from a worker node.

Parameters:

• cluster – the currently known cluster state from DCS.

• event – the event to be handled.

ignore_replication_slot(slot: Dict[str, str]) → boolView on GitHub

Check whether provided replication slot existing in the database should not be removed.

Note

MPP database may create replication slots for its own use, for example to migrate data between workers using logical replication, and we don’t want to suddenly drop them.

Parameters:

slot – dictionary containing the replication slot settings, like name, database, type, and plugin.

Returns:

always False.

on_demote() → NoneView on GitHub

On demote handler.

Is called when the primary was demoted.

schedule_cache_rebuild() → NoneView on GitHub

Cache rebuild handler.

Is called to notify handler that it has to refresh its metadata cache from the database.

sync_meta_data(cluster: Cluster) → NoneView on GitHub

Sync meta data on the coordinator.

Parameters:

cluster – the currently known cluster state from DCS.

patroni.postgresql.mpp.get_mpp(config: Config | Dict[str, Any]) → AbstractMPPView on GitHub

Attempt to load and instantiate a MPP module from known available implementations.

Parameters:

config – object or dictionary with Patroni configuration.

Returns:

The successfully loaded MPP or fallback to Null.

patroni.postgresql.mpp.iter_mpp_classes(config: Config | Dict[str, Any] | None = None) → Iterator[Tuple[str, Type[AbstractMPP]]]View on GitHub

Attempt to import MPP modules that are present in the given configuration.

Parameters:

config – configuration information with possible MPP names as keys. If given, only attempt to import MPP modules defined in the configuration. Else, if None, attempt to import any supported MPP module.

Yields:

tuples, each containing the module name and the imported MPP class object.