patroni.log module

Patroni logging facilities.

Daemon processes will use a 2-step logging handler. Whenever a log message is issued it is initially enqueued in-memory and is later asynchronously flushed by a thread to the final destination.

class patroni.log.PatroniLoggerView on GitHub 

Bases: Thread

Logging thread for the Patroni daemon process.

It is a 2-step logging approach. Any time a log message is issued it is initially enqueued in-memory, and then asynchronously flushed to the final destination by the logging thread.

See also

QueueHandler: object used for enqueueing messages in-memory.

Variables:

DEFAULT_TYPE – default type of log format (plain).
DEFAULT_LEVEL – default logging level (INFO).
DEFAULT_TRACEBACK_LEVEL – default traceback logging level (ERROR).
DEFAULT_FORMAT – default format of log messages (%(asctime)s %(levelname)s: %(message)s).
NORMAL_LOG_QUEUE_SIZE – expected number of log messages per HA loop when operating under a normal situation.
DEFAULT_MAX_QUEUE_SIZE – default maximum queue size for holding a backlog of log messages that are pending to be flushed.
LOGGING_BROKEN_EXIT_CODE – exit code to be used if it detects(5).
log_handler – log handler that is currently being used by the thread.
log_handler_lock – lock used to modify log_handler.

DEFAULT_FORMAT = '%(asctime)s %(levelname)s: %(message)s'

DEFAULT_LEVEL = 'INFO'

DEFAULT_MAX_QUEUE_SIZE = 1000

DEFAULT_TRACEBACK_LEVEL = 'ERROR'

DEFAULT_TYPE = 'plain'

LOGGING_BROKEN_EXIT_CODE = 5

NORMAL_LOG_QUEUE_SIZE = 2

__init__() → NoneView on GitHub : Prepare logging queue and proxy handlers as they become ready during daemon startup.

Note

While Patroni is starting up it keeps DEBUG log level, and writes log messages through a proxy handler. Once the logger thread is finally started, it switches from that proxy handler to the queue based logger, and applies the configured log settings. The switching is used to avoid that the logger thread prevents Patroni from shutting down if any issue occurs in the meantime until the thread is properly started.

_close_old_handlers() → NoneView on GitHub : Close old log handlers.

Note

It is used to remove different handlers that were configured previous to a reload in the configuration, e.g. if we are switching from RotatingFileHandler to class:~logging.StreamHandler and vice-versa.

_get_formatter(config: Dict[str, Any]) → FormatterView on GitHub 

Returns a logging formatter based on the type of logger in the given configuration.

Parameters:: config – log section from Patroni configuration.
Returns:: A logging.Formatter object that can be used to format log records.

Returns a logging formatter that outputs JSON formatted messages.

Note

If pythonjsonlogger library is not installed, prints an error message and returns a plain log formatter instead.

Parameters:

logformat – Specifies the log fields and their key names in the JSON log message.
dateformat – The format of the timestamp in the log messages.
static_fields – A dictionary of static fields that are added to every log message.

Returns:

A logging formatter object that can be used to format log records as JSON strings.

Returns a logging formatter with the specified format and date format.

Note

If the log format isn’t a string, prints a warning message and uses the default log format instead.

Parameters:

logformat – The format of the log messages.
dateformat – The format of the timestamp in the log messages.

Returns:

A logging formatter object that can be used to format log records.

_is_config_changed(config: Dict[str, Any]) → boolView on GitHub 

Checks if the given config is different from the current one.

Parameters:: config – log section from Patroni configuration.
Returns:: True if the config is changed, False otherwise.

property queue_size: int: Number of log records in the queue.

property records_lost: int: Number of logging records that have been lost while the queue was full.

reload_config(config: Dict[str, Any]) → NoneView on GitHub 

Apply log related configuration.

Note

It is also able to deal with runtime configuration changes.

Parameters:: config – log section from Patroni configuration.

run() → NoneView on GitHub 

Run logger’s thread main loop.

Keep consuming log queue until requested to quit through None special log record.

shutdown() → NoneView on GitHub : Shut down the logger thread.

update_loggers(config: Dict[str, Any]) → NoneView on GitHub 

Configure custom loggers’ log levels.

Note

It creates logger objects that are not defined yet in the log manager.

Parameters:

config –

dict object with custom loggers configuration, is set either from:

log.loggers section of Patroni configuration; or
from the method that is trying to make sure that the node name isn’t duplicated (to silence annoying urllib3 WARNING’s).

Example:

update_loggers({'urllib3.connectionpool': 'WARNING'})

class patroni.log.ProxyHandler(patroni_logger: PatroniLogger)View on GitHub 

Bases: Handler

Handle log records in place of pending log handlers.

Note

This is used to handle log messages while the logger thread has not started yet, in which case the queue-based handler is not yet started.

Variables:: patroni_logger – the logger thread.

__init__(patroni_logger: PatroniLogger) → NoneView on GitHub 

Create a new ProxyHandler instance.

Parameters:: patroni_logger – the logger thread.

emit(record: LogRecord) → NoneView on GitHub 

Emit each log record that is handled.

Will push the log record down to handle() method of the currently configured log handler.

Parameters:: record – the record that was emitted.

class patroni.log.QueueHandlerView on GitHub 

Bases: Handler

Queue-based logging handler.

Variables:: queue – queue to hold log messages that are pending to be flushed to the final destination.

__init__() → NoneView on GitHub : Queue initialised and initial records_lost established.

_put_record(record: LogRecord) → NoneView on GitHub 

Asynchronously enqueue a log record.

Parameters:: record – the record to be logged.

_try_to_report_lost_records() → NoneView on GitHub : Report the number of log messages that have been lost and reset the counter.

Note

It will issue an WARNING message in the logs with the number of lost log messages.

emit(record: LogRecord) → NoneView on GitHub 

Handle each log record that is emitted.

Call _put_record() to enqueue the emitted log record.

Also check if we have previously lost any log record, and if so, log a WARNING message.

Parameters:: record – the record that was emitted.

property records_lost: int: Number of log messages that have been lost while the queue was full.

patroni.log.debug_exception(self: Logger, msg: object, *args: Any, **kwargs: Any) → NoneView on GitHub 

Add full stack trace info to debug log messages and partial to others.

Handle exception() calls for self.

Note

If self log level is set to DEBUG, then issue a DEBUG message with the complete stack trace;
If self log level is INFO or higher, then issue an ERROR message with only the last line of
the stack trace.

Parameters:

self – logger for which exception() will be processed.
msg – the message related to the exception to be logged.
args – positional arguments to be passed to debug() or error().
kwargs – keyword arguments to be passed to debug() or error().

patroni.log.error_exception(self: Logger, msg: object, *args: Any, **kwargs: Any) → NoneView on GitHub 

Add full stack trace info to error messages.

Handle exception() calls for self.

Note

By default issue an ERROR message with the complete stack trace. If you do not want to show the complete stack trace, call with exc_info=False.

Parameters:

self – logger for which exception() will be processed.
msg – the message related to the exception to be logged.
args – positional arguments to be passed to error().
kwargs – keyword arguments to be passed to error().