twindb_table_compare package

Submodules

twindb_table_compare.cli module

Command line routines

twindb_table_compare.compare module

Functions to find and print differences

twindb_table_compare.compare.build_chunk_query(db, tbl, chunk, conn, ch_db='percona', ch_tbl='checksums')

For a given database, table and chunk number construct a SELECT query that would return records in this chunk.

twindb_table_compare.compare.diff(master_lines, slave_lines, color=True)

Find differences between two set of lines.

Parameters:

• master_lines (list) – First set of lines
• slave_lines (list) – Second set of lines
• color (bool) – If True return colored diff

Returns:

Difference between two set of lines

Return type:

str

twindb_table_compare.compare.get_boundary(*args, **kwargs)

Get lower and upper boundary values of a chunk.

Parameters:

• args (tuple(Connection, str, str, int)) – Positional arguments.
• kwargs (dict) – Keyword arguments.

Returns:

tuple with values lower_boundary and upper_boundary of percona.checksums

Positional arguments

• connection: MySQL connection.
• database: database of the chunk.
• tbl: table of the chunk.
• chunk: chunk id.

Keyword arguments.

• ch_db: Database where checksums are stored. Default percona.
• ch_tbl: Table where checksums are stored. Default checksums.

twindb_table_compare.compare.get_boundary_clause(oper='>', index_fields=None, boundaries=None)

Generate a clause for the WHERE statement based on field names and values.

Parameters:

• oper – Can be either < for the lower boundary or > for the upper boundary.
• index_fields (list) – list of fields in the index.
• boundaries (list) – list of values for a boundary.

Returns:

a clause that defines a boundary (upper or lower) of a chunk.

twindb_table_compare.compare.get_chunk_index(*args, **kwargs)

Get index that was used to cut the chunk.

Parameters:

• args (tuple(Connection, str, str, int)) – Positional arguments.
• kwargs (dict) – Keyword arguments.

Returns:

index name or None if no index was used

Positional arguments

• connection: MySQL connection.
• database: database of the chunk.
• tbl: table of the chunk.
• chunk: chunk id.

Keyword arguments.

• ch_db: Database where checksums are stored. Default percona.
• ch_tbl: Table where checksums are stored. Default checksums.

twindb_table_compare.compare.get_fields(conn, db, tbl)

Construct fields list string for a SELECT. If a field is a binary type (BLOB, VARBINARY) then HEX() it.

Parameters:

• conn (Connection) – MySQL connection.
• db (str) – Database name.
• tbl (str) – Table name.

Returns:

A comma separated list of fields.

Return type:

str

twindb_table_compare.compare.get_inconsistencies(db, tbl, slave, user, passwd, ch_db='percona', ch_tbl='checksums', vertical=False, color=True)

Print differences between slave and its master.

Parameters:

• db – Database name of the inconsistent table.
• tbl – Table name of the inconsistent table.
• slave – Hostname of the slave.
• user – User to connect to MySQL.
• passwd – Password to connect to MySQL.
• ch_db – Database where checksums are stored.
• ch_tbl – Table name where checksums are stored.
• vertical – If True - print result vertically (G in MySQL)
• color – If True - print colorful output

twindb_table_compare.compare.get_inconsistent_tables(host, user, password, ch_db='percona', ch_tbl='checksums')

On a given MySQL server find tables that are inconsistent with the master.

Parameters:

• host – Hostname with potentially inconsistent tables.
• user – MySQL user.
• password – MySQL password.
• ch_db – Database where checksums are stored.
• ch_tbl – Table name where checksums are stored.

Returns:

List of tuples with inconsistent tables. Each tuple is database name, table name.

Return type:

list

twindb_table_compare.compare.get_index_fields(connection, db, tbl, index)

Get fields of the given index

Parameters:

• connection – MySQLDb connection
• db – database
• tbl – table
• index – index name

Returns:

list of field names

twindb_table_compare.compare.get_master(connection)

Get master host

Parameters:connection – MySQL connection Returns:Master hostname

twindb_table_compare.compare.get_where(lower_boundary, upper_boundary, index_fields)

Generate WHERE clause based on strings lower_boundary, upper_boundary from percona.checksums table and fields in the index.

Parameters:

• lower_boundary (str) – values of lower boundary.
• upper_boundary (str) – values of upper boundary.
• index_fields – list of fields in the index that was used. to access a chunk.

Returns:

a WHERE clause to read a chunk.

Return type:

str

twindb_table_compare.compare.is_printable(str_value)

Checks if str_value is printable string.

Parameters:str_value – Returns:True if str_value is printable. False otherwise

twindb_table_compare.compare.primary_exists(conn, db, tbl)

Check if PRIMARY index exists in table db.tbl

Parameters:

• conn (Connection) – MySQLdb connection.
• db (str) – Database name.
• tbl (str) – Table name.

Returns:

True if index PRIMARY exists in table db.tbl

Return type:

bool

twindb_table_compare.compare.print_horizontal(cur_master, cur_slave, query, color=True)

Find and return differences in horizontal format i.e. one line - one record

Parameters:

• cur_master (Cursor) – MySQLdb cursor on master
• cur_slave (Cursor) – MySQLdb cursor on slave
• query (str) – Query to find records in a chunk we compare
• color – If True - produce colorful output

Returns:

Differences in a chunk between master and slave

Return type:

str

twindb_table_compare.compare.print_vertical(master, slave, user, passwd, query, color=True)

Find and return differences in vertical format. The vertical format is when you end MySQL query with ‘G’

Parameters:

• master (str) – Hostname of the master.
• slave (str) – Hostname of the slave.
• query (str) – Query to find records in a chunk we compare
• color – If True - produce colorful output

Returns:

Differences in a chunk between master and slave

Return type:

str

Module contents

Module to read pr-table-checksum’s result table (percona.checksums) and show user which records are actually different.

twindb_table_compare.setup_logging(logger, debug=False, color=True)

Configure logging.

Parameters:

• logger (Logger) – Logger to configure.
• debug – If True - print debug messages
• color – If True - print colored messages