Console View
|
Categories: connectors experimental galera main |
|
| connectors | experimental | galera | main | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
ARaveala
arandomsanti@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39762: Slave Overflow on Malformed Query_compressed_log_event A replica can crash when a compressed event from its master carries a corrupted uncompressed length. A compressed Query event whose length is near 4GB overwrites the IO thread's stack with the event's content. A compressed rows event of the same shape makes the replica ask for an allocation of 4GB. A debug build that gets that memory then fails an assertion in binlog_buf_uncompress(). On a build where a ulong is 32 bits wide, both events overwrite the IO thread's stack. The IO thread uncompresses a compressed event before writing the event to the relay log, and offers the uncompress function 4096 bytes of its own stack to hold the result. The function computes how large the event will be once uncompressed, and that size decides where the uncompressed event goes. An event that fits in the stack buffer is uncompressed there. A larger event is uncompressed into an allocation the function makes at that size. With either buffer, the function tells zlib that the room available is the uncompressed length the event declared. query_event_uncompress() never bounded the length read out of the event. The function added the header length to that value, rounded the sum up for alignment, and cast the result to uint32. The cast dropped the high bits of a sum above 4GB, so a length near 4GB produced a size of a few dozen bytes. That size fit the stack buffer, so content of any size went onto the IO thread's stack. row_log_event_uncompress() computes that sum in a ulong, which wraps where a ulong is 32 bits wide. The Query_compressed_log_event constructor and Rows_log_event::uncompress_buf() size their allocations from the same unbounded length. Reject an uncompressed length above MAX_MAX_ALLOWED_PACKET at every point the length is read, before any size is computed from it. No master writes a larger length, because max_allowed_packet is capped at 1GB. query_event_uncompress() also keeps that size in a size_t, so the allocation is made from the whole computed value. Both functions return the error code for the IO thread to report, so a length over the limit stops the IO thread with ER_TOO_BIG_FOR_UNCOMPRESS rather than the generic uncompress error. A compressed event with a corrupted uncompressed length now stops the replica's IO thread with an error instead of crashing the replica. Alexandra Raveala wrote the original patch and its first regression test, contributed through PR 5413. Brandon Nesterenko saw the work through for the 10.6 release, replacing the test with one that makes the master write the corrupt length and reworking how the IO thread reports the error. Co-authored-by: Brandon Nesterenko <[email protected]> Reviewed-by: Kristian Nielsen <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40648: Replication Undefined Behavior on Malformed Rotate Log Event A slave stops with a relay log read failure when its master sends an event whose header declares a length other than the number of bytes the event arrived in. The file and position the slave reports for that failure are not where the problem is. A master that logs no checksum does not stop the slave at all. Such a master can make the slave run one statement twice, leaving the slave's data holding a row the master's binary log never carried. A "malicious" master can already send whatever events it likes, so what this defeats is comparing a slave's applied stream against the master's binary log. The slave IO thread reads each event from the master as one network packet, and writes that packet into the relay log unchanged, using the packet's own length. Every later reader of that relay log frames the events by a different length: the one each event's header declares at EVENT_LEN_OFFSET. A master writing an event sets the two to the same value. Log_event::read_log_event(), which parses the events that queue_event() does not construct itself, checks only that the packet reaches EVENT_LEN_OFFSET, and never compares the declared length with the length of the packet. queue_event() never compared the two lengths either. An event declaring fewer bytes than the packet held reached the relay log with the extra bytes behind the event. The SQL thread framed its next read from inside the previous event. Where the master had placed a complete event in those extra bytes, and no checksum covered the packet, the SQL thread applied that second event. This patch adds validation to ensure the lengths are equal. An event whose lengths disagree will stop the IO thread with ER_SLAVE_FATAL_ERROR, and the relay log will never receive the event. Reviewed-by: TODO Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Daniel Black
daniel@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40572 SIGSEGV in JSON_OVERLAPS() on a truncated nested JSON object with different character sets/collations. The value->s.c_str and js->s.c_str became invalid once the function returned which caused the crash. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge branch 'bb-10.6-release' into bb-10.11-release | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
forkfun
alice.sherepa@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39146 Fix use-after-free/UAF races on THD::security_ctx SQL SECURITY DEFINER switches (events, stored routines, views, virtual columns) repoint thd->security_ctx without synchronizing with PROCESSLIST/KILL, which read it from other threads under LOCK_thd_data. For events specifically, the DEFINER context is stack-allocated in Event_job_data::execute(), so a concurrent PROCESSLIST read can dereference it after the frame is gone. - Add THD::set_security_context() as the single locked chokepoint for repointing security_ctx; convert all writers to use it. - Lock COM_CHANGE_USER's failed-auth free()+restore of ->user. - Lock kill_threads_callback()'s identity match test. - thd_visible_in_processlist()/processlist_callback()/list_callback(): trylock (never block, avoids stalling the whole server), falling back to priv_user/priv_host (fixed arrays, never freed) on contention instead of the freed/switched heap pointers. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40643: Corrupt Heartbeat Log Event can Crash Slave A slave can crash when its master sends an event shorter than the event's own header. A truncated heartbeat makes debug builds fail an assertion in Binary_string::realloc_raw(). On 64-bit release builds it stops the slave IO thread with an error message that omits the log file name. If the master also declares an oversized common header in its format description event, the slave allocates and fills nearly 4GB instead. On 32-bit release builds it writes that error message past the end of a stack buffer. An event under four bytes crashes every build, because the slave checksums roughly 16 EiB and reads far past the end of the packet. Neither queue_event() nor the Heartbeat_log_event constructor bounded the length the master sent. queue_event() handed that length to event_checksum_test(), which subtracted the checksum length from it, and the result wrapped on an event under four bytes. The constructor computed the log file name length as event_len minus the header lengths, and on a truncated heartbeat that subtraction wrapped to a value near 4GB. queue_event() rejected the heartbeat as invalid, so its error path appended the log file name to the error message using the wrapped length. Add the missing bound in both places. queue_event() now rejects an event shorter than the common header before anything reads that header, and the slave IO thread stops with an error. The Heartbeat_log_event constructor now compares event_len against the combined header lengths before the subtraction. A heartbeat that clears the first check but still stops inside its headers keeps ident_len at 0 and log_ident at NULL, so the error path appends nothing. That constructor check also precedes the read of the extended log position, which previously ran on a short event before any validation. A short event from the master now stops the slave IO thread with an error instead of crashing the server. Reviewed-by: Kristian Nielsen <[email protected]> Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge branch 'bb-11.8-release' into bb-12.3-release | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-40646 (Regression): Undersized Table_map Metadata can Crash Slave | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40553: unprintable gis ranges in trace and context When ranges were specified in a query for GIS types, the recorded trace and context couldn't print the range information. Instead, it only showed unprintable_geometry_value. This PR extends the geometric field type Field_geom to print appropriate key value, along with the comparison operator when printed in the range. Key value is decoded from binary if applicable, and is recorded in WKT format. For spatial indexes, the operators like MBRWITHIN, MBRCONTAINS, etc... are stored appropriately, and for normal indexes, operators like <, <=, >, >=, etc... are recorded appropriately. Implementation Details: - 1. overwrite print_key_value() for Field_geom to print the decoded binary value in WKT format. If binary value can't be decoded appropriately to WKT format, then the binary value itself is recorded. 2. Add an argument imagetype to Field::print_key_part_value(), to determine if an index key part value is to be printed in WKT or binary format. For Geometric type, imagetype is set to itMBR. When printing ranges, the MBR operators are printed as well, as implemented in print_mbr_range_operator(). |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40648 (Regression): Replication Undefined Behavior on Malformed Rotate Log Event test |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexey Yurchenko
alexey.yurchenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40179 Found N prepared transactions after mariabackup SST With log_bin=ON a transaction is committed via two-phase commit (the binary log is the second participant), so it passes through the InnoDB XA-prepare state. While a donor is held in BLOCK_COMMIT for a mariabackup backup, its parallel appliers (wsrep_slave_threads > 1) leave one or more such writesets prepared-but-not-yet-committed, and the snapshot captures them. On a freshly SST'd joiner nothing resolves these prepared transactions: binlog crash recovery does not run (the joiner has no in-use binlog to recover from), and the wsrep continuity-based commit is inactive because wsrep_emulate_bin_log is FALSE when log_bin is ON. The leftover prepared transactions then abort startup with "Found <N> prepared transactions!". Note this does not depend on the prepared set being non-contiguous - even a contiguous run aborts, because nothing commits or rolls it back. Rollback these transactions in xarecover_handlerton(). If rollback fails flag error to cause unireg_abort(). |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge branch '10.11' into bb-10.11-release | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-40644 (Regression): Slave SQL Thread Overflow on Malformed Table_map_log_event | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Razvan-Liviu Varzaru
razvan@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38754 s3.clone run in high concurrency environments s3.clone needs to source create_database.inc so that it can run in a high-concurrency environment where multiple MTR invocations are happening at the same time, while only a single S3 bucket (the S3 DB) is shared among all instances. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge branch 'bb-10.11-release' into bb-11.4-release | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-40678 mysql_json plugin OOB reads | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40644: Slave SQL Thread Overflow on Malformed Table_map_log_event A slave can crash when its master sends a Table_map event whose table name is longer than an identifier can be. The SQL thread applying the event copies the name into a fixed buffer and overruns it, and the SQL thread crashes. A Table_map event carries its database and table names each behind a one-byte length. The constructor that parses the event sizes a buffer for each name from its declared length, and do_apply_event() copies both names out of the event into NAME_LEN+1 byte buffers with strmov(). strmov() copies up to the terminating null. A real identifier is at most NAME_LEN bytes, so the name fits the buffer and carries that null. The constructor never bounded the two lengths. Each length is read as a byte, so each can reach 255. The constructor copied each name with strncpy() over the declared length, which writes no terminating null when the source holds none. do_apply_event() then copied the name into its NAME_LEN+1 byte buffer with strmov(), which ran past the buffer until it reached a null elsewhere in the heap. pack_info() read the same names with %s for SHOW RELAYLOG EVENTS, past the buffer in the same way. Bound the lengths in the constructor. It now rejects a Table_map event whose declared database name, or whose declared table name, exceeds NAME_LEN, before it sizes the name buffers, so is_valid() returns false. It also terminates each name it copies, so a source that holds no null does not leave the name unterminated. A master naming a table with an oversized name now stops the slave's SQL thread with ER_SLAVE_RELAY_LOG_READ_FAILURE instead of crashing it. Reviewed-by: TODO Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40648: Replication Undefined Behavior on Malformed Rotate Log Event A slave stops with a relay log read failure when its master sends an event whose header declares a length other than the number of bytes the event arrived in. The file and position the slave reports for that failure are not where the problem is. A master that logs no checksum does not stop the slave at all. Such a master can make the slave run one statement twice, leaving the slave's data holding a row the master's binary log never carried. A "malicious" master can already send whatever events it likes, so what this defeats is comparing a slave's applied stream against the master's binary log. The slave IO thread reads each event from the master as one network packet, and writes that packet into the relay log unchanged, using the packet's own length. Every later reader of that relay log frames the events by a different length: the one each event's header declares at EVENT_LEN_OFFSET. A master writing an event sets the two to the same value. Log_event::read_log_event(), which parses the events that queue_event() does not construct itself, checks only that the packet reaches EVENT_LEN_OFFSET, and never compares the declared length with the length of the packet. queue_event() never compared the two lengths either. An event declaring fewer bytes than the packet held reached the relay log with the extra bytes behind the event. The SQL thread framed its next read from inside the previous event. Where the master had placed a complete event in those extra bytes, and no checksum covered the packet, the SQL thread applied that second event. This patch adds validation to ensure the lengths are equal. An event whose lengths disagree will stop the IO thread with ER_SLAVE_FATAL_ERROR, and the relay log will never receive the event. Reviewed-by: Kristian Nielsen <[email protected]> Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-40492 (Regression): Oversized Fake Rotate Event can Crash Slave | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-24813 Signal full scan to storage engines. When starting to do a full table/index scan without a WHERE or JOIN condition, tell the storage engine so and the corresponding ulong-truncated LIMIT. Include an innodb implementation: added an innodb switch table_lock_on_full_scan, so that when the switch is on, on receiving the full scan signal from the sql layer, if the truncated LIMIT is ULONG_MAX (likely no LIMIT), attempt to acquire a table lock. Updated tests that have different results with the switch on. The three deadlock_*_race tests cannot reach their DEBUG_SYNC race under a table lock (it degenerates to a timeout), and the three I_S tests only restate a lock-mode change already covered by innodb_full_scan.test. (Comment and code edited by Sergei Petrunia <[email protected]> and Thirunarayanan Balathandayuthapani <[email protected]>) |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexey Yurchenko
alexey.yurchenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38147 error 1950 after mariabackup SST with gtid_strict_mode=ON After a mariabackup SST the joiner could fail with ER_GTID_STRICT_OUT_OF_ORDER (error 1950) while re-binlogging transactions received over IST. The cause is that the binary log copied from the donor carries a Gtid_list whose position can be ahead of the storage-engine snapshot because getting snapshot and binlog are no mutually atomic, so transactions can be present in the copied binlog that are not committed in the copied engine snapshot. After the SST the joiner reports the (committed) engine position to the cluster, IST resends those transactions, and re-binlogging them under gtid_strict_mode=ON collides with the ahead Gtid_list -> error 1950. (MDEV-34483 made the engine snapshot stop short of the binlog, which is what exposed this.) The copied binary log carries no transactions the joiner needs - only a Gtid_list - so instead of shipping and then having to truncate/reconcile it, the joiner now starts a fresh binary log and seeds its GTID position from the storage-engine checkpoint during recovery. That checkpoint is the committed cluster position, i.e. exactly where IST resumes, so the joiner's binary log stays in lockstep with the rest of the cluster and no out-of-order GTID can occur. This works for both wsrep_gtid_mode settings; only the binlog domain of the cluster stream differs: - wsrep_gtid_mode=ON : wsrep_gtid_domain_id (cluster writes are re-tagged to it), which is the domain stored in the checkpoint; - wsrep_gtid_mode=OFF: gtid_domain_id (cluster writes keep the node's configured domain). Async-replica positions (mysql.gtid_slave_pos) are part of the engine snapshot and survive the SST unchanged, so a Galera node can still serve as an async master or replica across the SST. This commit: - sql/log.cc: adds wsrep_seed_binlog_gtid_state(), called from do_binlog_recovery() when the joiner has no binary log, seeding the binlog GTID state for the cluster domain to the SE checkpoint position. - scripts/wsrep_sst_mariabackup.sh: no longer moves the donor's binary log into place on the joiner. - extra/mariabackup: backward compatibility: keep shipping binlog file in SST but - on donor fix the race between rotation and shipping so that the file shipped is the one that had been rotated; - on joiner discard shipped binlog in favour of one generated by wsrep_seed_binlog_gtid_state(). - sql/wsrep_sst.cc: logs the position actually adopted from storage (the authoritative post-SST position) rather than the script-reported one. - sql/handler.cc: downgrades the "Discovered discontinuity in recovered wsrep transaction XIDs" message in wsrep_order_and_check_continuity() from warning to debug level. With parallel appliers a snapshot routinely captures prepared XIDs that are not contiguous with the engine checkpoint, so this is normal during SST recovery and of no value in regular operation; the transactions past the checkpoint are re-delivered by the cluster (IST/SST) regardless. - Adds an MDEV-38147 MTR test reproducing the issue. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40643: Corrupt Heartbeat Log Event can Crash Slave A slave can crash when its master sends an event shorter than the event's own header. A truncated heartbeat makes debug builds fail an assertion in Binary_string::realloc_raw(). On 64-bit release builds it stops the slave IO thread with an error message that omits the log file name. If the master also declares an oversized common header in its format description event, the slave allocates and fills nearly 4GB instead. On 32-bit release builds it writes that error message past the end of a stack buffer. An event under four bytes crashes every build, because the slave checksums roughly 16 EiB and reads far past the end of the packet. Neither queue_event() nor the Heartbeat_log_event constructor bounded the length the master sent. queue_event() handed that length to event_checksum_test(), which subtracted the checksum length from it, and the result wrapped on an event under four bytes. The constructor computed the log file name length as event_len minus the header lengths, and on a truncated heartbeat that subtraction wrapped to a value near 4GB. queue_event() rejected the heartbeat as invalid, so its error path appended the log file name to the error message using the wrapped length. Add the missing bound in both places. queue_event() now rejects an event shorter than the common header before anything reads that header, and the slave IO thread stops with an error. The Heartbeat_log_event constructor now compares event_len against the combined header lengths before the subtraction. A heartbeat that clears the first check but still stops inside its headers keeps ident_len at 0 and log_ident at NULL, so the error path appends nothing. That constructor check also precedes the read of the extended log position, which previously ran on a short event before any validation. A short event from the master now stops the slave IO thread with an error instead of crashing the server. Reviewed-by: TODO Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40492: Oversized Fake Rotate Event can Crash Slave A slave can crash when the master's binlog_checksum setting differs from the checksum the slave's relay log carries. On the first Rotate event of such a connection, the slave IO thread copies the event onto a stack buffer sized for the longest binary log file name a master can send. A master that names a longer file makes the copy run off the end of that buffer, and what it writes past the end is the file name itself. A name that overruns the buffer by a few hundred bytes reaches only the other buffers of the same stack frame, which the slave does not read on this path, so it keeps running. A name that overruns the whole frame takes the return address with it, and the slave crashes. Builds with AddressSanitizer stop at the copy and report a stack-buffer-overflow. The fake Rotate event that opens a connection is rewritten when the master's checksum policy and the relay log's disagree. One case adds a checksum to the event, and the other strips one. Both copy the event into rot_buf using the length the master sent, and neither compared that length against the size of rot_buf. The Rotate_log_event constructed just above bounds its own copy of the file name at FN_REFLEN-1, which leaves the length the master sent untouched. rot_buf was itself one checksum shorter than the event the first case produces from the longest file name. Bound the event before either case copies it. queue_event() now compares the event's length against the longest Rotate a master can send: the two headers, a file name of at most FN_REFLEN bytes, and the checksum the master's own policy puts on the event. rot_buf now reserves that checksum length as well. A Rotate event naming a longer file stops the slave IO thread with an error reporting the length the master sent, instead of running off the end of the buffer. Reviewed-by: Kristian Nielsen <[email protected]> Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge branch 'bb-11.4-release' into bb-11.8-release | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40492: Oversized Fake Rotate Event can Crash Slave A slave can crash when the master's binlog_checksum setting differs from the checksum the slave's relay log carries. On the first Rotate event of such a connection, the slave IO thread copies the event onto a stack buffer sized for the longest binary log file name a master can send. A master that names a longer file makes the copy run off the end of that buffer, and what it writes past the end is the file name itself. A name that overruns the buffer by a few hundred bytes reaches only the other buffers of the same stack frame, which the slave does not read on this path, so it keeps running. A name that overruns the whole frame takes the return address with it, and the slave crashes. Builds with AddressSanitizer stop at the copy and report a stack-buffer-overflow. The fake Rotate event that opens a connection is rewritten when the master's checksum policy and the relay log's disagree. One case adds a checksum to the event, and the other strips one. Both copy the event into rot_buf using the length the master sent, and neither compared that length against the size of rot_buf. The Rotate_log_event constructed just above bounds its own copy of the file name at FN_REFLEN-1, which leaves the length the master sent untouched. rot_buf was itself one checksum shorter than the event the first case produces from the longest file name. Bound the event before either case copies it. queue_event() now compares the event's length against the longest Rotate a master can send: the two headers, a file name of at most FN_REFLEN bytes, and the checksum the master's own policy puts on the event. rot_buf now reserves that checksum length as well. A Rotate event naming a longer file stops the slave IO thread with an error reporting the length the master sent, instead of running off the end of the buffer. Reviewed-by: TODO Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Daniel Black
daniel@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40630: stunnel as Deb/RPM recommends for rsync sst stunnel has been a component used by the rsync SST mechanism of galera but has never been listed as a package dependency. Since MDEV-28233 corrected a unencrypted fallback in the the case of stunnel being absent, we add stunnel as a recommendation for Debian and RPM packages. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40646: Undersized Table_map Metadata can Crash Slave A slave can crash or leak heap content when its master sends a Table_map event that declares columns whose types need more field metadata than the event carries. Building its description of the table, the slave reads past the end of the field metadata the event holds. The bytes beyond the buffer are decoded as column metadata. A Table_map event carries a column count, one type byte per column, and a block of field metadata. A column's type fixes how many metadata bytes it consumes: two for the string, enum, set, bit, varchar, and decimal types, one for the blob, float, and high precision temporal types, none for the rest. The master writes exactly that many bytes per column, so the metadata block a well formed event carries is the sum over its columns. The slave sizes the metadata block from the event and, while building its table_def, walks the columns in order and reads each type's metadata bytes from that block. The slave checked only that the metadata block was no larger than two bytes per column. It never checked the block against the column types themselves. table_def() then walked every declared column and indexed the metadata block for each, so a column whose type needed metadata the block did not hold read past the end of it. A column count large enough carried that read far past the allocation. The Table_map reader now sums the metadata the declared column types consume and rejects the event when the block is smaller than that sum, so is_valid() reports the event as invalid and the SQL thread stops with a relay log read failure before any column is decoded. A block larger than the sum is still accepted, because every consumer reads only the bytes the types call for. A column type this slave does not recognize counts as zero in the sum, so Table_map events from a newer master using such a type still parse. A Table_map whose metadata cannot cover its column types now stops the slave with an error instead of reading past the buffer. Note that the regression test does not crash a regular debug build; an ASAN build is required to show the invalid memory access. The 24 column table map it injects overruns the metadata allocation by a bounded amount that lands in mapped heap, which a debug build reads without faulting and then stops the SQL thread with ER_SLAVE_CONVERSION_FAILED on the column type mismatch. Reviewed-by: TODO Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40644: Slave SQL Thread Overflow on Malformed Table_map_log_event A slave can crash when its master sends a Table_map event whose table name is longer than an identifier can be. The SQL thread applying the event copies the name into a fixed buffer and overruns it, and the SQL thread crashes. A Table_map event carries its database and table names each behind a one-byte length. The constructor that parses the event sizes a buffer for each name from its declared length, and do_apply_event() copies both names out of the event into NAME_LEN+1 byte buffers with strmov(). strmov() copies up to the terminating null. A real identifier is at most NAME_LEN bytes, so the name fits the buffer and carries that null. The constructor never bounded the two lengths. Each length is read as a byte, so each can reach 255. The constructor copied each name with strncpy() over the declared length, which writes no terminating null when the source holds none. do_apply_event() then copied the name into its NAME_LEN+1 byte buffer with strmov(), which ran past the buffer until it reached a null elsewhere in the heap. pack_info() read the same names with %s for SHOW RELAYLOG EVENTS, past the buffer in the same way. Bound the lengths in the constructor. It now rejects a Table_map event whose declared database name, or whose declared table name, exceeds NAME_LEN, before it sizes the name buffers, so is_valid() returns false. It also terminates each name it copies, so a source that holds no null does not leave the name unterminated. A master naming a table with an oversized name now stops the slave's SQL thread with ER_SLAVE_RELAY_LOG_READ_FAILURE instead of crashing it. Reviewed-by: TODO Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-40644 (Regression): Slave SQL Thread Overflow on Malformed Table_map_log_event | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Daniel Black
daniel@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40165 JSON_EQUAL/JSON_NORMALIZE/JSON_CONTAINS error handling Adds test cases that validate the incorrect null handling in the 2894e90b6545d3c7fa13ff00fb50c9e8631003e7 commit and other JSON_EQUALS,CONTAINS/OVERLAPS behaviour. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-40643 (Regression): Corrupt Heartbeat Log Event can Crash Slave | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||