Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
fd8c0ca8ce04...
fd8c0ca8ce04d23d57149b49810bda5321efbd32
Brandon Nesterenko
MDEV-34705: Binlog-in-engine: Update rpl_row_basic_* MTR tests to use GTIDs
f177b768c2b5...
f177b768c2b5e58dca18ad72c51879410fdf2906
Kristian Nielsen
Binlog-in-engine: Fix SAVEPOINT name re-use

It is legal to re-use the same name for a SAVEPOINT in a single transaction.
In this case, the new savepoint overrides the old one, effectively deleting
the old one. The code was not correctly handling this, and could end up with
an invalid list of pending savepoints, causing assertion and malfunction.

Fix by searching the list for any existing occurrence of the savepoint being
set, and deleting any such occurrence found before inserting it at the end
of the list.

Signed-off-by: Kristian Nielsen <[email protected]>
e8160b55e8b7...
e8160b55e8b77817d59cf4672a135ba9271a2d66
Kristian Nielsen
Binlog-in-engine: Fix two recovery bugs

1. The binlog recovery layer gets duplicate records in case of multi-batch
recovery, seemingly any partial mini-transaction is re-applied from the
start at a batch boundary. Thus, we need to be able to accept multiple
duplicate records belonging to the same mtr.

2. All records in a mini-transaction share the same LSN. Thus, when an mtr
spans two binlog files, the LSN of the mtr may compare larger than the
start_lsn of file_no+1, while part of that mtr (but not all of it) applies
to file_no. In this case, the record could incorrectly be interpreted as
applying to file_no+2 instead of file_no.

Both bugs found as test failures of binlog_in_engine.recover_concurrent_dml.

Signed-off-by: Kristian Nielsen <[email protected]>
e4b0e591a85f...
e4b0e591a85f818efdc0e0d32af5da70982197e1
Kristian Nielsen
Binlog-in-engine: MDEV-38462: Large binlog write can overwrite head of InnoDB redo log

The code for binlogging out-of-band data was missing an appropriate call to
log_free_check(). This call is needed to throttle write activity and wait
for an InnoDB checkpoint, when the redo log is too small (or otherwise has
insufficient space available) to accomodate the write activity.

Signed-off-by: Kristian Nielsen <[email protected]>
e3331af9c843...
e3331af9c8431c7d66afd558effa8ae958385904
Kristian Nielsen
Binlog-in-engine: Fixes for mariabackup --binlog-directory

Don't use (and crash on) any --binlog-directory option specified for
--backup, always use the value fetched from the running server.

Ensure a slash in-between path components when using a relative path for options
such as --innodb-undo-directory and --binlog-directory.

Clarify the description of the --binlog-directory option.

Signed-off-by: Kristian Nielsen <[email protected]>
ce9fcca91125...
ce9fcca9112550665a86e3a967dcb423e18b9701
Brandon Nesterenko
MDEV-34705: Binlog-in-engine: Update rpl.rpl_ddl to use GTIDs to sync
cda2addeb692...
cda2addeb6928dc660552547dc6bcb5a4a3b8c2c
Brandon Nesterenko
MDEV-34705: Binlog-in-engine: Add MTR tests for rpl.rpl_ddl

Master uses binlog-in-engine, slave uses file binlog
cd94eed4b2e9...
cd94eed4b2e9b9c69b0194b921c3b13e041ffee7
Kristian Nielsen
Binlog-in-engine: Remove useless info messages in error log

Signed-off-by: Kristian Nielsen <[email protected]>
ccd0afc5942e...
ccd0afc5942e30d433f8aa11f9b55476b59e254f
Kristian Nielsen
Binlog-in-engine: Fix rpl.rpl_circular_for_4_hosts after changes to rpl_sync.inc

Signed-off-by: Kristian Nielsen <[email protected]>
c7a550e9c2a6...
c7a550e9c2a6e83796cb1d34fcb4b0c8f4ce5ec3
Kristian Nielsen
Binlog-in-engine: Fix race between FLUSH BINARY LOGS and GTID allocation

There was a race where a new GTID could be allocated (but not written to the
binlog)during the FLUSH, so that the GTID state written at the start of the
new binlog file was incorrect. This in turn could lead to duplicate GTID
being sent to the slave if it happens to reconnect at that exact point.

Signed-off-by: Kristian Nielsen <[email protected]>
c69ea9b286ac...
c69ea9b286aceee1e72ffe3713db543633115cbe
Vladislav Vaintroub
Skip main.mysql_options_err, if ha_example plugin is missing

This fixes Appveyor build.
c14e3adeb883...
c14e3adeb883d48ea332fa52c8f82521f533d7c0
Kristian Nielsen
Binlog-in-engine: Documentation

Add user documentation for the new binlog implementation. And add error messages for the remaining configuration options that are not available with the new binlog.

Signed-off-by: Kristian Nielsen <[email protected]>
b90b299769a6...
b90b299769a64d57f8797660df8ace43a607221e
Brandon Nesterenko
MDEV-34705: Binlog-in-engine: Add tests for rpl_row_basic

Converts test rpl_row_basic_3innodb to binlog in engine with two
variants:
1) Master uses binlog-in-innodb and slave uses file binlog
2) Master use file binlog and slave uses binlog-in-innodb
b0464a72c68e...
b0464a72c68eb2fb8e9514920a8f9795cb183401
Kristian Nielsen
MDEV-38429: Atomic DDL can binlog DDL during ddl log crash recovery without "ddl" mark on the GTID

When ddl log recovery needs to binlog during the crash recovery, the
GTID was binlogged without the required "ddl" marker. This caused wrong
behaviour on the slave when using parallel replication.

Fixed by explicitly marking the "current statement" as DDL when binlogging
in ddl log crash recovery.

Signed-off-by: Kristian Nielsen <[email protected]>
aef54e9dc6a0...
aef54e9dc6a0ac97a8a45d48697655cf0534c748
Kristian Nielsen
MDEV-38465: Savepoint in trigger causes transactional inconsistency

SAVEPOINT inside a trigger doesn't work correctly. Setting a savepoint
inside a trigger somehow loses the implicit savepoint set at transaction
start, so that the partial changes are left if the statement later fails.
Referencing an existing savepoint claims the savepoint does not exist (and
it is in any case very unclear what exactly it should mean to rollback to a
savepoint from the middle of a statement, or set in the middle of a prior
statement).

These problems are independent of binlog-in-engine, but in the new binlog
implementation we are trying to make things work more correctly and
robustly, so let's disallow use of savepoints inside triggers. The new
binlog is off by default, so backwards compatibility is less of a concern,
though arguably disallowing savepoints in triggers would be better done
unconditionally.

Signed-off-by: Kristian Nielsen <[email protected]>
ae8ac6fd472c...
ae8ac6fd472cf4b98bf2d4ba8b31848552698ec5
Kristian Nielsen
MDEV-38429: Atomic DDL can binlog DDL during ddl log crash recovery without "ddl" mark on the GTID

This patch fixes that ALTER TABLE can call wakeup_subsequent_commits() too
early and allow following event groups to commit out-of-order in parallel
replication. Fixed by calling suspend_subsequent_commits() at the start
of the ALTER.

Could be seen as an assertion:
  !tmp_gco->next_gco || tmp_gco->last_sub_id > sub_id

(Normally this is prevented because an ALTER TABLE will run in its own GCO,
and thus no following event groups can even start; however the missing DDL
mark caused by MDEV-38429 made this visible. And calling
wakeup_subsequent_commits() too early is wrong in any case).

Signed-off-by: Kristian Nielsen <[email protected]>
94a302d3b2bb...
94a302d3b2bb7171ff302b9cd17bf77dda8d1fb2
Kristian Nielsen
Binlog-in-engine: Fix sporadic test after rebase to latest

Signed-off-by: Kristian Nielsen <[email protected]>
8f972e80ecf0...
8f972e80ecf039e148c8679a829987670be9385b
Kristian Nielsen
Binlog-in-engine: Fix incorrect printf arguments

Signed-off-by: Kristian Nielsen <[email protected]>
86fa0a902443...
86fa0a902443ae800b1e53b4e508bdd11b4c3684
ParadoxV5
`decimal_out` `sometimes-uninitialized`

Be defensive: Do not assign out if any conversion step fails.
80a87a2cd663...
80a87a2cd663ad9dd0c82706fd300073d41d6add
Kristian Nielsen
Binlog-in-engine: Update docs with upgrading info

Signed-off-by: Kristian Nielsen <[email protected]>
73110a1f4ce2...
73110a1f4ce2bbb688a77b61a81cffbacc8f92e8
Kristian Nielsen
Binlog-in-innodb: Small docs clarifications

Signed-off-by: Kristian Nielsen <[email protected]>
6df4e0fd3923...
6df4e0fd3923cf2fa27ea1559509a9a7b9276cbd
Kristian Nielsen
Binlog-in-engine: Fix sporadic failure of binlog-in-engine.mysqlbinlog

FLUSH BINARY LOGS before dumping, to make sure the file is on disk and not
get different mysqlbinlog output depending on timing.

Treat completely empty (all zeros) file the same as file with the header
page written but no events yet.

Signed-off-by: Kristian Nielsen <[email protected]>
6575d117d20b...
6575d117d20ba07a8da4e401e9f428c483e87833
ParadoxV5
`decimal_out` `sometimes-uninitialized`

Be defensive: Do not assign out if any conversion step fails.
63a8a3d62d71...
63a8a3d62d71c1fab525953830b6a578c3152d42
Kristian Nielsen
Binlog-in-engine: Fix error handling for failure to write GTID event

The error handling path forgot to unlock the LOCK_log mutex, hanging the
server or causing assertion mysql_mutex_assert_not_owner.

Signed-off-by: Kristian Nielsen <[email protected]>
5df6dc0ba5fc...
5df6dc0ba5fcea9f671d3bda5e3b18ab403ca249
Kristian Nielsen
Binlog-in-engine: More robust binlog recovery

If binlog files are deleted or otherwise unreadable during server restart,
don't make the server unstartable. Instead, start up, recovering what is
available, but complaining in the error log.

Signed-off-by: Kristian Nielsen <[email protected]>
56605c87c35a...
56605c87c35a1dff3f6e18997cb061fe93f58624
Kristian Nielsen
Binlog-in-engine: Fix race causing hang during shutdown

When the tablespace is closed during shutdown, it waits for the last record
written in the tablespace to be durable in the InnoDB redo log.

There was an obvious mistake/race in the code in function
ibb_wait_durable_offset(), which did not check for the waited-for condition
before doing the wait.

Thus if the last record in the binlog file became durable between the check
in fsp_binlog_tablespace_close() and the wait in ibb_wait_durable_offset(),
it would wait for new data to be written; this could cause a hang.

Signed-off-by: Kristian Nielsen <[email protected]>
5388c37a37af...
5388c37a37afd0b1cf8b621920e2d500ea5673ef
Kristian Nielsen
Fix that InnoDB rolls back entire trx in failed CREATE TEMPORARY

For CREATE TEMPORARY TABLE ... SELECT, InnoDB had code to not start a new
transaction for the CREATE TEMPORARY (correct). But the code that handled
failure for the SELECT part (ha_innobase::extra(HA_EXTRA_ABORT_ALTER_COPY))
was missing a check for CREATE TEMPORARY, so it would roll back the entire
transaction, which is wrong, and could lead to inconsistency with binlog or
other engines in the same transaction.

Signed-off-by: Kristian Nielsen <[email protected]>
52a75d45325b...
52a75d45325b6c8539956a1c998b57ad71a606ad
Kristian Nielsen
Binlog-in-engine: Fix incorrect GTID position at slave connect

The GTID position at slave connect is found from the GTID state records
written at the start and at every --innodb-binlog-state-interval bytes of
the binlog files. There was a bug that for a binlog group commit, the binlog
state written was the one corresponding to the last GTID in the group,
regardless of where during the binlogging of the group it was written. Thus,
it could mistakenly write a GTID state record of 0-1-10, say, followed by a
lower GTID 0-1-9. This could cause a slave connecting at 0-1-10 to receive
an extra GTID 0-1-9, and replication would diverge.

Fix by maintaining a full GTID binlog state inside the engine binlog, same
as is done for the differential state.

Signed-off-by: Kristian Nielsen <[email protected]>
510d5cf2ee6b...
510d5cf2ee6b44bddaebfa1daec6060ec81e266e
Kristian Nielsen
Binlog-in-engine: Small cleanup/formatting

Signed-off-by: Kristian Nielsen <[email protected]>
3fbb3591bc7c...
3fbb3591bc7c045354fd16d6c200e2d5f9225380
Kristian Nielsen
Binlog-in-engine: Implement internal 2pc and user XA recovery.

When using the InnoDB-implemented binlog with another transactional storage
engine, or with explicit user XA transactions, recover such transactions
consistently from the binlog at server startup.

When a transaction is prepared with an XID, the binlog records a "prepare"
record containing the XID and link to the out-of-band replication event
data.

When a previously prepared transaction is committed, the commit record links
to the oob data referenced from the prepare record, and the record is
preceeded by an "XA complete" record containing the XID.

If instead a prepared transaction is rolled back, just an "XA complete"
record is binlogged with the XID and a "rollback" flag.

While any prepared XA transactions are active, maintain in-memory reference
counts in each binlog file, and in each binlog file record the file_no of
the earliest binlog file containing any XID records of still active
transactions.

When the server restarts (possibly after crash), look up the file_no of the
earliest binlog file that may contain active XID records, if any. Scan the
binlogs from that point and record any XID prepare or complete records.

For any XID prepare record, record oob data and reference count, recovering
the in-memory state present before the server restart. Return a hash to the
server layer containing each active XID in the binlog and its state
(prepared, committed, rolled back).

On the server layer, ask each engine for a list of pending XID in prepared
state. If the binlog state of an XID is committed, commit in the engine. If
the binlog state is rolled back or is missing, roll back in the engine. If
the binlog state is prepared, _and_ all participating engines have the
transaction prepared also, then leave the transaction prepared. If a binlog
prepared transaction is missing from an engine, then roll it back in any
other engines and in the binlog (this is to handle a crash in the middle of
an XA PREPARE).

The result is that multi-engine (or non-InnoDB) transactions, as well as
user XA transactions, will be recovered after a crash consisent with the
binlog content.

Signed-off-by: Kristian Nielsen <[email protected]>
3c5bfd164fe9...
3c5bfd164fe99be52c564876c8f91c275eb468e7
Kristian Nielsen
Binlog-in-engine: SAVEPOINT fixes.

1. Handle RELEASE SAVEPOINT, removing any released savepoint from the list
of savepoints pending in the cache. Also fix a bug in the server layer;
RELEASE SAVEPOINT removes the specified savepoint _and_ any later
savepoints; engines were not informed of the removal of the later ones, if
any.

2. Fix a bug when spilling non-transactional statement data inside of a
transaction using savepoints. The spill of the statement cache must not
spill any savepoints, those apply only to the trx cache.

Signed-off-by: Kristian Nielsen <[email protected]>
3553889664f1...
3553889664f14ee28efec35e1fe7d2ae1bf2c47e
Kristian Nielsen
Binlog-in-engine: Assertion 'sp == nullptr' with SAVEPOINT

Adjust an assertion that checks that no unspilled savepoints are left after
spilling the trx cache as OOB data to the engine binlog.

If the savepoint is at the very end of the trx cache when we spill, there is
no need to spill that particular savepoint to the engine (and we do not do
so); the savepoint can still be rolled back to by truncating the in-memory
part of the cache.

Signed-off-by: Kristian Nielsen <[email protected]>
32b853e6fe47...
32b853e6fe47a6dea54786f039ff248307d47d9e
Kristian Nielsen
Binlog-in-engine: Clean up few remaining todos, error handling

Signed-off-by: Kristian Nielsen <[email protected]>
2f551a081282...
2f551a081282caf1cde250641669328e33484453
Kristian Nielsen
Binlog-in-engine: Fix autopurge of still needed binlog file

This happened when the first OOB record of an event group spans two binlog
files, say N and N+1. The reference counting would wrongly attribute the OOB
to N+1, allowing N to be purged while it was still needed.

This for example could cause server restart to fail when it tries to recover
the GTID state from N+1, unable to follow OOB references to N because it was
purged before the server restart.

Fix by:
- Increment OOB refcount _before_ binlogging the first OOB record.
- Decrement refcount only _after binlogging complete.
- Protecting from purge any files referenced from the active file.

Signed-off-by: Kristian Nielsen <[email protected]>
2d2aa48c7a89...
2d2aa48c7a89bac50d66351e9318edd052bc0e89
Kristian Nielsen
Binlog-in-engine: Add mariadb-backup option to omit binlog from the backup

Signed-off-by: Kristian Nielsen <[email protected]>
2b658de9ea4a...
2b658de9ea4aac163e01c636a8b15e3a7cbb13b6
Kristian Nielsen
Binlog-in-engine: Add recovery testcase

This recovery testcase more aggressively exercises the recovery. It
runs a parallel DML load on the master and crashes it at arbitrary
point; then checks the self-consistency of the transactions to test
for partially/incorrectly recovered individual transaction, and
replicates to slave and tests consistency between master and slave.

Signed-off-by: Kristian Nielsen <[email protected]>
1c767fd934bd...
1c767fd934bdcce4d90d36034caee6875079f636
Kristian Nielsen
Binlog-in-engine: Tweak mysqlbinlog error handling

If the user copies manually an engine-implemented binlog file and runs
mysqlbinlog on it, any following or oob-referenced files may not be
available to read from. Treat this as end-of-file rather than an error
(so we can output at least any part of the file that is available).
But still output a message about the failure to open the file, to give
some indication why the dump stopped at that point.

Signed-off-by: Kristian Nielsen <[email protected]>
1320a67ba0d5...
1320a67ba0d5ee8996718ac11dc1354bb6ae50e6
Brandon Nesterenko
MDEV-34705: Binlog-in-engine: rpl_mysqlbinlog_slave_consistency tests

This patch creates binlog-in-engine MTR test equivalents for
rpl_mysqlbinlog_slave_consistency. One new test is a basic variant,
which tests the exact same behavior as the rpl test version, and the
other uses out-of-band binlogging to ensure the slave and mysqlbinlog
event replay is consistent. The rpl.rpl_mysqlbinlog_slave_consistency
test validates that the domain and server id filtering logic of
mariadb-binlog matches that of a replica server. The purpose of this
patch is then to ensure that the expected filtering behavior is still
consistent when using binlog-in-innodb.

Changes to the replication-side of tests are as follows:
1. So the binlog_in_engine tests can find the correct include files,
    the paths to the files from the `source` commands are made absolute
2. To support the out-of-band binlogging variant, the
    tables/transactions in sql_multisource.inc and
    sql_out_of_order_gtid.inc are extended with a configurable size
    longblob, so the OOB test can use large transactions.
3. To add some complexity for the OOB testing, the transactions in
    sql_multisource.inc and sql_out_of_order_gtid.inc are staggered
    using different connections and run concurrently so the OOB data
    is overlapping. This is done for cases using different domain ids
    as well as the same domain id (though commit order is still
    enforced).
11732fb574ab...
11732fb574ab0395e0c8847f59f068ee838c8370
Kristian Nielsen
Binlog-in-engine: Fix couple XA bugs

When an empty user XA transaction is committed (or rolled back), we do
not need to binlog any real transaction, but we still need to binlog a
rollback record to clear the XID for reuse and free the prepare record from
purge and from recovery.

Also fix a bug that in case of error adding to the innodb binlog internal
xid hash (eg. duplicate), we must still ensure that the written XA prepare
record is entered into the pending LSN fifo, so we can track when it becomes
durable (there was a shutdown hang possible if a prepare record was last in
the binlog and XID insert failed so the record was never marked durable).

Bugs found in RQG runs.

Signed-off-by: Kristian Nielsen <[email protected]>
BuildBot (0.8.9) working for the MariaDB project.
Page built: Sat 17 Jan 2026 01:02:47 (UTC)