patroni.validator module¶
Patroni configuration validation helpers.
This module contains facilities for validating configuration of Patroni processes.
- var schema
configuration schema of the daemon launched by
patroni
command.
- class patroni.validator.AtMostOne(*args: str)¶
Bases:
object
Mark that at most one option from a
Case
can be suplied.Represents a list of possible configuration options in a given scope, where at most one can actually be provided.
Note
It should be used together with a
Case
object.- __init__(*args: str) None ¶
Create a :class`AtMostOne` object.
- Parameters
*args – any arguments that the caller wants to be stored in this
Or
object.- Example
AtMostOne("nofailover", "failover_priority"): Case({ "nofailover": bool, "failover_priority": IntValidator(min=0, raise_assert=True), })
The :class`AtMostOne` object is used to define that at most one of
nofailover
andfailover_priority
can be provided.
- class patroni.validator.BinDirectory(contains: Optional[List[str]] = None, contains_executable: Optional[List[str]] = None)¶
Bases:
patroni.validator.Directory
Check if a Postgres binary directory contains the expected files.
It is a subclass of
Directory
with an extended capability: translatingBINARIES
according to configuredpostgresql.bin_name
, if any.- Variables
BINARIES – list of executable files that should exist directly under a given Postgres binary directory.
- BINARIES = ['pg_ctl', 'initdb', 'pg_controldata', 'pg_basebackup', 'postgres', 'pg_isready']¶
- validate(name: str) Iterator[patroni.validator.Result] ¶
Check if the expected executables can be found under name binary directory.
- Parameters
name – path to the base directory against which executables will be validated. Check against PATH if name is not provided.
- Yields
objects with the error message related to the failure, if any check fails.
- class patroni.validator.Case(schema: Dict[str, Any])¶
Bases:
object
Map how a list of available configuration options should be validated.
Note
It should be used together with an
Or
object. TheOr
object will define the list of possible configuration options in a given context, and theCase
object will dictate how to validate each of them, if they are set.- __init__(schema: Dict[str, Any]) None ¶
Create a
Case
object.- Parameters
schema – the schema for validating a set of attributes that may be available in the configuration. Each key is the configuration that is available in a given scope and that should be validated, and the related value is the validation function or expected type.
- Example
Case({ "host": validate_host_port, "url": str, })
That will check that
host
configuration, if given, is valid based onvalidate_host_port()
, and will also check thaturl
configuration, if given, is astr
instance.
- class patroni.validator.Directory(contains: Optional[List[str]] = None, contains_executable: Optional[List[str]] = None)¶
Bases:
object
Check if a directory contains the expected files.
The attributes of objects of this class are used by their
validate()
method.- Parameters
contains – list of paths that should exist relative to a given directory.
contains_executable – list of executable files that should exist directly under a given directory.
- __init__(contains: Optional[List[str]] = None, contains_executable: Optional[List[str]] = None) None ¶
Create a
Directory
object.- Parameters
contains – list of paths that should exist relative to a given directory.
contains_executable – list of executable files that should exist directly under a given directory.
- _check_executables(path: Optional[str] = None) Iterator[patroni.validator.Result] ¶
Check that all executables from contains_executable list exist within the given directory or within
PATH
.- Parameters
path – optional path to the base directory against which executables will be validated. If not provided, check within
PATH
.- Yields
objects with the error message containing the name of the executable, if any check fails.
- validate(name: str) Iterator[patroni.validator.Result] ¶
Check if the expected paths and executables can be found under name directory.
- Parameters
name – path to the base directory against which paths and executables will be validated. Check against
PATH
if name is not provided.- Yields
objects with the error message related to the failure, if any check fails.
- class patroni.validator.EnumValidator(allowed_values: Tuple[str, ...], case_sensitive: bool = False, raise_assert: bool = False)¶
Bases:
object
Validate enum setting
- Variables
allowed_values – a
set
orCaseInsensitiveSet
object with allowed enum values.raise_assert – if an
assert
call should be performed regarding expected type and valid range.
- __init__(allowed_values: Tuple[str, ...], case_sensitive: bool = False, raise_assert: bool = False) None ¶
Create an
EnumValidator
object with given allowed values.- Parameters
allowed_values – a tuple with allowed enum values
case_sensitive – set to
True
to do case sensitive comparisonsraise_assert – if an
assert
call should be performed regarding expected values.
- class patroni.validator.IntValidator(min: Optional[int] = None, max: Optional[int] = None, base_unit: Optional[str] = None, expected_type: Optional[Any] = None, raise_assert: bool = False)¶
Bases:
object
Validate an integer setting.
- Variables
min – minimum allowed value for the setting, if any.
max – maximum allowed value for the setting, if any.
base_unit – the base unit to convert the value to before checking if it’s within min and max range.
expected_type – the expected Python type.
raise_assert – if an
assert
test should be performed regarding expected type and valid range.
- __init__(min: Optional[int] = None, max: Optional[int] = None, base_unit: Optional[str] = None, expected_type: Optional[Any] = None, raise_assert: bool = False) None ¶
Create an
IntValidator
object with the given rules.- Parameters
min – minimum allowed value for the setting, if any.
max – maximum allowed value for the setting, if any.
base_unit – the base unit to convert the value to before checking if it’s within min and max range.
expected_type – the expected Python type.
raise_assert – if an
assert
test should be performed regarding expected type and valid range.
- class patroni.validator.Optional(name: str, default: Optional[Any] = None)¶
Bases:
object
Mark a configuration option as optional.
- Variables
name – name of the configuration option.
default – value to set if the configuration option is not explicitly provided
- class patroni.validator.Or(*args: Any)¶
Bases:
object
Represent the list of options that are available.
It can represent either a list of configuration options that are available in a given scope, or a list of validation functions and/or expected types for a given configuration option.
- __init__(*args: Any) None ¶
Create an
Or
object.- Parameters
*args – any arguments that the caller wants to be stored in this
Or
object.- Example
Or("host", "hosts"): Case({ "host": validate_host_port, "hosts": Or(comma_separated_host_port, [validate_host_port]), })
The outer
Or
is used to define thathost
andhosts
are possible options in this scope. The inner :class`Or` in thehosts
key value is used to define thathosts
option is valid if either ofcomma_separated_host_port()
orvalidate_host_port()
succeed to validate it.
- class patroni.validator.Result(status: bool, error: Optional[str] = "didn't pass validation", level: int = 0, path: str = '', data: Any = '')¶
Bases:
object
Represent the result of a given validation that was performed.
- Variables
status – If the validation succeeded.
path – YAML tree path of the configuration option.
data – value of the configuration option.
level – error level, in case of error.
error – error message if the validation failed, otherwise
None
.
- __init__(status: bool, error: Optional[str] = "didn't pass validation", level: int = 0, path: str = '', data: Any = '') None ¶
Create a
Result
object based on the given arguments.Note
error
attribute is only set if status is failed.- Parameters
status – if the validation succeeded.
error – error message related to the validation that was performed, if the validation failed.
level – error level, in case of error.
path – YAML tree path of the configuration option.
data – value of the configuration option.
- class patroni.validator.Schema(validator: Union[Dict[Any, Any], List[Any], Any])¶
Bases:
object
Define a configuration schema.
It contains all the configuration options that are available in each scope, including the validation(s) that should be performed against each one of them. The validations will be performed whenever the
Schema
object is called, or itsvalidate()
method is called.- Variables
validator –
validator of the configuration schema. Can be any of these:
str
: defines that a string value is required; ortype
: any subclass oftype
, defines that a value of the given type is required; orcallable
: any callable object, defines that validation will follow the code defined in the callable object. If the callable object contains anexpected_type
attribute, then it will check if the configuration value is of the expected type before calling the code of the callable object; orlist
: list representing one or more values in the configuration; ordict
: dictionary representing the YAML configuration tree.
- __init__(validator: Union[Dict[Any, Any], List[Any], Any]) None ¶
Create a
Schema
object.Note
This class is expected to be initially instantiated with a
dict
based validator argument. The idea is that dict represents the full YAML tree of configuration options. Thevalidate()
method will then walk recursively through the configuration tree, creating new instances ofSchema
with the new “base path”, to validate the structure and the leaf values of the tree. The recursion stops on leaf nodes, when it performs checks of the actual setting values.- Parameters
validator –
validator of the configuration schema. Can be any of these:
str
: defines that a string value is required; ortype
: any subclass oftype
, defines that a value of the given type is required; orcallable
: Any callable object, defines that validation will follow the code defined in the callable object. If the callable object contains anexpected_type
attribute, then it will check if the configuration value is of the expected type before calling the code of the callable object; orlist
: list representing it expects to contain one or more values in the configuration; ordict
: dictionary representing the YAML configuration tree.
The first 3 items in the above list are here referenced as “base validators”, which cause the recursion to stop.
If validator is a
dict
, then you should follow these rules:For the keys it can be either:
A
str
instance. It will be the name of the configuration option; orAn
Optional
instance. Thename
attribute of that object will be the name of the configuration option, and that class makes this configuration option as optional to the user, allowing it to not be specified in the YAML; or- An
Or
instance. Theargs
attribute of that object will contain a tuple of configuration option names. At least one of them should be specified by the user in the YAML;
- An
For the values it can be either:
A new
dict
instance. It will represent a new level in the YAML configuration tree; orA
Case
instance. This is required if the key of this value is anOr
instance, and theCase
instance is used to map each of theargs
inOr
to their corresponding base validator inCase
; orAn
Or
instance with one or more base validators; orA
list
instance with a single item which is the base validator; orA base validator.
- Example
Schema({ "application_name": str, "bind": { "host": validate_host, "port": int, }, "aliases": [str], Optional("data_directory"): "/var/lib/myapp", Or("log_to_file", "log_to_db"): Case({ "log_to_file": bool, "log_to_db": bool, }), "version": Or(int, float), })
This sample schema defines that your YAML configuration follows these rules:
It must contain an
application_name
entry which value should be astr
instance;It must contain a
bind.host
entry which value should be valid as per functionvalidate_host
;It must contain a
bind.port
entry which value should be anint
instance;It must contain a
aliases
entry which value should be alist
ofstr
instances;It may optionally contain a
data_directory
entry, with a value which should be a string;It must contain at least one of
log_to_file
orlog_to_db
, with a value which should be abool
instance;It must contain a
version
entry which value should be either anint
or afloat
instance.
- _data_key(key: Union[str, patroni.validator.Optional, patroni.validator.Or, patroni.validator.AtMostOne]) Iterator[str] ¶
Map a key from the
validator
dictionary to the corresponding key(s) in thedata
dictionary.- Parameters
key – key from the
validator
attribute.- Yields
keys that should be used to access corresponding value in the
data
attribute.
- iter() Iterator[patroni.validator.Result] ¶
Iterate over
validator
, if it is an iterable object, to validate the corresponding entries indata
.Only
dict
,list
,Directory
andOr
objects are considered iterable objects.- Yields
objects with the error message related to the failure, if any check fails.
- iter_dict() Iterator[patroni.validator.Result] ¶
Iterate over a
dict
basedvalidator
to validate the corresponding entries indata
.- Yields
objects with the error message related to the failure, if any check fails.
- iter_or() Iterator[patroni.validator.Result] ¶
Perform all validations defined in an
Or
object for a given configuration option.This method can be only called against leaf nodes in the configuration tree.
Or
objects defined in thevalidator
keys will be handled byiter_dict()
method.- Yields
objects with the error message related to the failure, if any check fails.
- validate(data: Union[Dict[Any, Any], Any]) Iterator[patroni.validator.Result] ¶
Perform all validations from the schema against the given configuration.
It first checks that data argument type is compliant with the type of
validator
attribute.- Additionally:
- If
validator
attribute is a callable object, calls it to validate data argument. Before doing so, if validator contains an
expected_type
attribute, check if data argument is compliant with that expected type.
- If
- Parameters
data – configuration to be validated against
validator
.- Yields
objects with the error message related to the failure, if any check fails.
- patroni.validator._get_type_name(python_type: Any) str ¶
Get a user-friendly name for a given Python type.
- Parameters
python_type – Python type which user friendly name should be taken.
- Returns
User friendly name of the given Python type.
- patroni.validator.assert_(condition: bool, message: str = 'Wrong value') None ¶
Assert that a given condition is
True
.If the assertion fails, then throw a message.
- Parameters
condition – result of a condition to be asserted.
message – message to be thrown if the condition is
False
.
- patroni.validator.comma_separated_host_port(string: str) bool ¶
Validate a list of host and port items.
Call
validate_host_port_list()
with a list represented by the CSV string.- Parameters
string – comma-separated list of host and port items.
- Returns
True
if all items in the CSV string are valid.
- patroni.validator.data_directory_empty(data_dir: str) bool ¶
Check if PostgreSQL data directory is empty.
- Parameters
data_dir – path to the PostgreSQL data directory to be checked.
- Returns
True
if the data directory is empty.
- patroni.validator.get_bin_name(bin_name: str) str ¶
Get the value of
postgresql.bin_name[*bin_name*]
configuration option.- Parameters
bin_name – a key to be retrieved from
postgresql.bin_name
configuration.- Returns
value of
postgresql.bin_name[*bin_name*]
, if present, otherwise bin_name.
- patroni.validator.is_ipv4_address(ip: str) bool ¶
Check if ip is a valid IPv4 address.
- Parameters
ip – the IP to be checked.
- Returns
True
if the IP is an IPv4 address.- Raises
ConfigParseError
: if ip is not a valid IPv4 address.
- patroni.validator.is_ipv6_address(ip: str) bool ¶
Check if ip is a valid IPv6 address.
- Parameters
ip – the IP to be checked.
- Returns
True
if the IP is an IPv6 address.- Raises
ConfigParseError
: if ip is not a valid IPv6 address.
- patroni.validator.validate_binary_name(bin_name: str) bool ¶
Validate the value of
postgresql.binary_name[*bin_name*]
configuration option.If
postgresql.bin_dir
is set and the value of the bin_name meets these conditions:The path join of
postgresql.bin_dir
plus the bin_name value exists; andThe path join as above is executable
If
postgresql.bin_dir
is not set, then validate that the value of bin_name meets this condition:Is found in the system PATH using
which
- Parameters
bin_name – the value of the
postgresql.bin_name[*bin_name*]
- Returns
True
if the conditions are true- Raises
ConfigParseError
if:bin_name is not set; or
the path join of the
postgresql.bin_dir
plus bin_name does not exist; orthe path join as above is not executable; or
the bin_name cannot be found in the system PATH
- patroni.validator.validate_connect_address(address: str) bool ¶
Check if options related to connection address were properly configured.
- Parameters
address – address to be validated in the format
host:ip
.- Returns
True
if the address is valid.- Raises
ConfigParseError
:If the address is not in the expected format; or
If the host is set to not allowed values (
127.0.0.1
,0.0.0.0
,*
,::1
, orlocalhost
).
- patroni.validator.validate_data_dir(data_dir: str) bool ¶
Validate the value of
postgresql.data_dir
configuration option.It requires that
postgresql.data_dir
is set and match one of following conditions:Point to a path that does not exist yet; or
Point to an empty directory; or
Point to a non-empty directory that seems to contain a valid PostgreSQL data directory.
- Parameters
data_dir – the value of
postgresql.data_dir
configuration option.- Returns
True
if the PostgreSQL data directory is valid.- Raises
ConfigParseError
:If no data_dir was given; or
If data_dir is a file and not a directory; or
- If data_dir is a non-empty directory and:
PG_VERSION
file is not available in the directorypg_wal
/pg_xlog
is not available in the directoryPG_VERSION
content does not match the major version reported bypostgres --version
- patroni.validator.validate_host_port(host_port: str, listen: bool = False, multiple_hosts: bool = False) bool ¶
Check if host(s) and port are valid and available for usage.
- Parameters
host_port –
the host(s) and port to be validated. It can be in either of these formats:
host:ip
, if multiple_hosts isFalse
; orhost_1,host_2,...,host_n:port
, if multiple_hosts isTrue
.
listen – if the address is expected to be available for binding.
False
means it expects to connect to that address, andTrue
that it expects to bind to that address.multiple_hosts – if host_port can contain multiple hosts.
- Returns
True
if the host(s) and port are valid.- Raises
ConfigParseError
:If the host_port is not in the expected format; or
If
*
was specified along with more hosts in host_port; orIf we are expecting to bind to an address that is already in use; or
If we are not able to connect to an address that we are expecting to do so; or
If
gaierror
is thrown by socket module when attempting to connect to the given address(es).
- patroni.validator.validate_host_port_list(value: List[str]) bool ¶
Validate a list of host(s) and port items.
Call
validate_host_port()
with each item in value.- Parameters
value – list of host(s) and port items to be validated.
- Returns
True
if all items are valid.
- patroni.validator.validate_host_port_listen(host_port: str) bool ¶
Check if host and port are valid and available for binding.
Call
validate_host_port()
with listen set toTrue
.- Parameters
host_port – the host and port to be validated. Must be in the format
host:ip
.- Returns
True
if the host and port are valid and available for binding.
- patroni.validator.validate_host_port_listen_multiple_hosts(host_port: str) bool ¶
Check if host(s) and port are valid and available for binding.
Call
validate_host_port()
with both listen and multiple_hosts set toTrue
.- Parameters
host_port –
the host(s) and port to be validated. It can be in either of these formats
host:ip
; orhost_1,host_2,...,host_n:port
- Returns
True
if the host(s) and port are valid and available for binding.
- patroni.validator.validate_log_field(field: Union[str, Dict[str, Any], Any]) bool ¶
Checks if log field is valid.
- Parameters
field – A log field to be validated.
- Returns
True
if the field is either a string or a dictionary with exactly one key that has string value,False
otherwise.
- patroni.validator.validate_log_format(logformat: Union[List[Union[str, Dict[str, Any], Any]], str, Any]) bool ¶
Checks if log format is valid.
- Parameters
logformat – A log format to be validated.
- Returns
True
if the log format is either a string or a list of valid log fields.- Raises
ConfigParseError
:If the logformat is not a string or a list; or
If the logformat is an empty list; or
If the log format is a list and it with values that don’t pass validation using
validate_log_field()
.