Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
ARaveala
MDEV-39762: Slave Overflow on Malformed Query_compressed_log_event

A replica can crash when a compressed event from its master carries a
corrupted uncompressed length. A compressed Query event whose length
is near 4GB overwrites the IO thread's stack with the event's content.
A compressed rows event of the same shape makes the replica ask for an
allocation of 4GB. A debug build that gets that memory then fails an
assertion in binlog_buf_uncompress(). On a build where a ulong is 32
bits wide, both events overwrite the IO thread's stack.

The IO thread uncompresses a compressed event before writing the event
to the relay log, and offers the uncompress function 4096 bytes of its
own stack to hold the result. The function computes how large the event
will be once uncompressed, and that size decides where the uncompressed
event goes. An event that fits in the stack buffer is uncompressed
there. A larger event is uncompressed into an allocation the function
makes at that size. With either buffer, the function tells zlib that
the room available is the uncompressed length the event declared.

query_event_uncompress() never bounded the length read out of the
event. The function added the header length to that value, rounded the
sum up for alignment, and cast the result to uint32. The cast dropped
the high bits of a sum above 4GB, so a length near 4GB produced a size
of a few dozen bytes. That size fit the stack buffer, so content of any
size went onto the IO thread's stack. row_log_event_uncompress()
computes that sum in a ulong, which wraps where a ulong is 32 bits
wide. The Query_compressed_log_event constructor and
Rows_log_event::uncompress_buf() size their allocations from the same
unbounded length.

Reject an uncompressed length above MAX_MAX_ALLOWED_PACKET at every
point the length is read, before any size is computed from it. No
master writes a larger length, because max_allowed_packet is capped at
1GB. query_event_uncompress() also keeps that size in a size_t, so the
allocation is made from the whole computed value. Both functions return
the error code for the IO thread to report, so a length over the limit
stops the IO thread with ER_TOO_BIG_FOR_UNCOMPRESS rather than the
generic uncompress error. A compressed event with a corrupted
uncompressed length now stops the replica's IO thread with an error
instead of crashing the replica.

Alexandra Raveala wrote the original patch and its first regression
test, contributed through PR 5413. Brandon Nesterenko saw the work
through for the 10.6 release, replacing the test with one that makes
the master write the corrupt length and reworking how the IO thread
reports the error.

Co-authored-by: Brandon Nesterenko <[email protected]>
Reviewed-by: Kristian Nielsen <[email protected]>
Brandon Nesterenko
MDEV-40648 Fix rpl.rpl_corruption

The test would induce a corrupt binlog event on the master and send it
to the slave, and with the new event validation logic, this could
trigger the slave to output the new error message added in MDEV_40648.

Add this error message as an allowed error during the executin of this
test

Signed-off-by: Brandon Nesterenko <[email protected]>
Brandon Nesterenko
MDEV-40643: Corrupt Heartbeat Log Event can Crash Slave

A slave can crash when its master sends an event shorter than the
event's own header. A truncated heartbeat makes debug builds fail an
assertion in Binary_string::realloc_raw(). On 64-bit release builds it
stops the slave IO thread with an error message that omits the log
file name. If the master also declares an oversized common header in
its format description event, the slave allocates and fills nearly 4GB
instead. On 32-bit release builds it writes that error message past
the end of a stack buffer. An event under four bytes crashes every
build, because the slave checksums roughly 16 EiB and reads far past
the end of the packet.

Neither queue_event() nor the Heartbeat_log_event constructor bounded
the length the master sent. queue_event() handed that length to
event_checksum_test(), which subtracted the checksum length from it,
and the result wrapped on an event under four bytes. The constructor
computed the log file name length as event_len minus the header
lengths, and on a truncated heartbeat that subtraction wrapped to a
value near 4GB. queue_event() rejected the heartbeat as invalid, so
its error path appended the log file name to the error message using
the wrapped length.

Add the missing bound in both places. queue_event() now rejects an event
shorter than the common header before anything reads that header, and
the slave IO thread stops with an error. The Heartbeat_log_event
constructor now compares event_len against the combined header lengths
before the subtraction. A heartbeat that clears the first check but
still stops inside its headers keeps ident_len at 0 and log_ident at
NULL, so the error path appends nothing. That constructor check also
precedes the read of the extended log position, which previously ran on
a short event before any validation. A short event from the master now
stops the slave IO thread with an error instead of crashing the server.

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>
ParadoxV5
MDEV-40365 OOB read on malformed `Format_description_log_event`

The binary-parsing Format Description Event constructor did not
validate content length beyond the superclass `is_valid()` call.
When parsing a malformed FDE, to load the content fields added in
FDE v4 that are missing in this not-FDE, the parser constructor
would read from erroneous memory locations beyond the buffer.
If this did not outright crash the program, this would corrupt the FDE.
With the Format Description playing a critical role in determining how
to parse the events to follow, a corrupted FDE would also corrupt (or
trigger a crash in) the parsing of subsequent non-FDE events as well.

This commit fills in the validation with a FDE-specific guard.
It adds the FDE constant `ST_POST_HEADER_LEN_OFFSET` to assist
with comparing to the correct minimum size in the future.
(Both of these points are designed to merge with the
superclass’s guard as part of the MDEV-30128 merger).

Reviewed-by: Kristian Nielsen <[email protected]>
ParadoxV5
MDEV-40674 Include the checksum for non-corrupted Unknown events

This commit reörders code so the checksum is populated after
`mariadb-binlog --force` generates `Unknown_log_event` substitutes.
Previously, `mariadb-binlog --force` inconsistently omitted those
checksums from the output even if the checksum is presumably  usable.

After merging to 11.4 (MDEV-31273), this commit will also fix «
MDEV-40542 MSAN use-of-uninitialized-value on Unknown_log_event::read_checksum_alg
», which was exposed by MDEV-31273’s removal
of the base `Log_event::checksum_alg` field.

Reviewed-by: Kristian Nielsen <[email protected]>
Oleksandr Byelkin
Merge branch '10.6' into 10.11
Oleksandr Byelkin
Merge branch '12.3' into 13.0
ParadoxV5
MDEV-39485 Heap-buffer-overflow upon read in `Rows_log_event` constructor

MariaDB recognizes Version 2 Rows Events from MySQL, including the
format of the “extra data” field added in this version. (MDEV-5115)

When parsing this extra data according to the format, whether this data
has sufficient length was only checked by assertions in the
`Rows_log_event` constructor and the `mariadb-binlog --verbose` printer.
When parsing an event with malformed extra data, these assertions
* would straight up terminate the program in debug builds.
* were stripped in non-debug (release) builds.
  This would render the parser defenseless to reading from erroneous
  memory locations outside of the containing event, which will either
  crash the program or, for `mariadb-binlog --verbose`, snapshot the
  running memory to be exposed when outputting the event.

This commit replaces those assertions with an actual validity check.

Since MariaDB does not generate v2 Rows Events,
the included test uses a handcrafted binlog file.

Reviewed-by: Kristian Nielsen <[email protected]>
Daniel Black
sql_test: mallinfo2 msan exclusion no longer needed

MSAN interceptor was added in clang-18.1.
ParadoxV5
MDEV-40366 OOB read on malformed `Format_description_log_event`

Neither the binary-parsing Format Description Event constructor
nor the constructor-bypassing `get_checksum_alg()` function
validated the content length of the passed event buffer.
If they receive an FDE with undersized contents,
they would obtain corrupt results from an erronous memory location,
if not outright crash the program with that memory error.

This commit fixes both sites by adding content length checks.
Because the goal is not to solve the existence of two binary parsers,
`get_checksum_alg()` receives a check duplicated from the Format
Description constructor and is no longer a function that never errors.
Note, it is implementation detail that `get_checksum_alg()`’s
fix catches the invalidity before any code reaches the
parser-contructor’s fix, though the latter would come
to effect if we refactor `get_checksum_alg()` away.

Reviewed-by: Kristian Nielsen <[email protected]>
Daniel Black
MDEV-17846 Wrong result with grouping select (fix)

Prevent unused variable 'ref_type'  warnings on non-debug builds.
forkfun
MDEV-38472 Assertion in Diagnostics_area::set_error_status on UPDATE...LIMIT DEFAULT

Binding DEFAULT to a LIMIT ? placeholder via EXECUTE ... USING is never
rejected at bind time. unit->set_limit() later evaluates it and sets
ER_INVALID_DEFAULT_PARAM in the Diagnostics_area, but SQLCOM_UPDATE (and
SQLCOM_UPDATE_MULTI, SQLCOM_INSERT_SELECT/REPLACE_SELECT) proceeded into
open_tables() regardless. A concurrent metadata change (e.g. ALTER TABLE
FORCE) then triggers Reprepare_observer::report_error(), which tries to
set ER_NEED_REPREPARE on an already-set DA and hits the assert.

Add the same thd->is_error() check after unit->set_limit() (as in
SQLCOM_DELETE), aborting before open_tables() is reached.
Brandon Nesterenko
MDEV-40648: Replication Undefined Behavior on Malformed Rotate Log Event

A slave stops with a relay log read failure when its master sends an
event whose header declares a length other than the number of bytes the
event arrived in. The file and position the slave reports for that
failure are not where the problem is. A master that logs no checksum
does not stop the slave at all. Such a master can make the slave run one
statement twice, leaving the slave's data holding a row the master's
binary log never carried. A "malicious" master can already send whatever
events it likes, so what this defeats is comparing a slave's applied
stream against the master's binary log.

The slave IO thread reads each event from the master as one network
packet, and writes that packet into the relay log unchanged, using the
packet's own length. Every later reader of that relay log frames the
events by a different length: the one each event's header declares at
EVENT_LEN_OFFSET. A master writing an event sets the two to the same
value. Log_event::read_log_event(), which parses the events that
queue_event() does not construct itself, checks only that the packet
reaches EVENT_LEN_OFFSET, and never compares the declared length with
the length of the packet.

queue_event() never compared the two lengths either. An event declaring
fewer bytes than the packet held reached the relay log with the extra
bytes behind the event. The SQL thread framed its next read from inside
the previous event. Where the master had placed a complete event in
those extra bytes, and no checksum covered the packet, the SQL thread
applied that second event.

This patch adds validation to ensure the lengths are equal. An event
whose lengths disagree will stop the IO thread with
ER_SLAVE_FATAL_ERROR, and the relay log will never receive the event.

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>
Sergei Petrunia
Move all thread creation/cleanup into sql_parallel_thread.*

The cleanup API is not yet very pretty.
Alexander Barkov
MDEV-39518 Allow prepared statements in stored functions in assignment right hand

Allowing prepared statements in stored functions when
a stored function is used in an assignment right hand.

Both DEFAULT clause of a variable initialization and
the right side of the SET statement are supported:

  CREATE PROCEDURE p1()
  BEGIN
    -- case 1: DEFAULT clause
    DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK

    -- case 2: SP variable assignment statement
    DECLARE spvar2 INT;
    SET spvar= f1_with_ps(); -- OK
  END;

- Only assignments to SP variables works for now:
  * SET spvar= func_with_ps(); -- OK
  * SET @uvar= func_with_ps(); -- Error

- Only bare function calls are supported for now. Using a function in
  an expression does not make it PS-safe yet yet:
    SET v= f1()+0;

- The parser now does not reject PS statements in stored functions.
  PS applicability in stored functions is now detected at run time.
  Note, PS statements in triggers are still prohibited by the parser.

- Functions with PS do not acquire MDL locks on tables.
  They work like procedures in terms of table opening.

- Functions with PS are not replicated as a single `SELECT f1()` call.
  They are replicated per-statement, like procedures.
Sergei Petrunia
Move all thread creation/cleanup into sql_parallel_thread.*

The cleanup API is not yet very pretty.
drrtuy
chore: remove jemalloc extension from DuckDB CMake b/c since 1.5.4 is is a part of DuckDB core.
Daniel Black
MDEV-40801 ppc64le ro_after_init isn't pagesize aligned

Align ro_after_init using MAXPAGESIZE instead of COMMONPAGESIZE.

COMMONPAGESIZE may be smaller than the actual page size supported by
the target ABI. This can leave ro_after_init sharing an OS page with
adjacent sections, causing mprotect() to change permissions on data
outside ro_after_init.

Use MAXPAGESIZE so the section boundaries are aligned to the maximum
page size required by the target linker/ABI.

This is particularly important on architectures such as ppc64le and
aarch64, where the runtime page size can differ from COMMONPAGESIZE.

Before:
  .data          0x...1b80000
  ro_after_init  0x...1c70000
  .bss          0x...1c72000

After:
  ro_after_init starts and ends on MAXPAGESIZE boundaries, ensuring
  mprotect() only affects pages belonging to ro_after_init.

Co-authored-by: ChatGPT GPT-5.6 Luna <[email protected]>
Daniel Black
MDEV-40243 Fix MEMORY_LEAK_C leaks in mariadb-dump  (rockdb tests)

With memory leaks fixed in the rocksdb.mysqldump/mysqldump2 no
longer need to run with leak detection disabled.

There tests are still disabled as there's no --rockdb arg to
mariadb-dump, but removing so there's no precidence to ignoring
leaks.

ref: 2217477fb8fc82f4921d9d13afcd24b1d86b34d6
Brandon Nesterenko
MDEV-40644: Slave SQL Thread Overflow on Malformed Table_map_log_event

A slave can crash when its master sends a Table_map event whose table
name is longer than an identifier can be. The SQL thread applying the
event copies the name into a fixed buffer and overruns it, and the SQL
thread crashes.

A Table_map event carries its database and table names each behind a
one-byte length. The constructor that parses the event sizes a buffer
for each name from its declared length, and do_apply_event() copies
both names out of the event into NAME_LEN+1 byte buffers with strmov().
strmov() copies up to the terminating null. A real identifier is at
most NAME_LEN bytes, so the name fits the buffer and carries that null.

The constructor never bounded the two lengths. Each length is read as a
byte, so each can reach 255. The constructor copied each name with
strncpy() over the declared length, which writes no terminating null
when the source holds none. do_apply_event() then copied the name into
its NAME_LEN+1 byte buffer with strmov(), which ran past the buffer
until it reached a null elsewhere in the heap. pack_info() read the
same names with %s for SHOW RELAYLOG EVENTS, past the buffer in the
same way.

Bound the lengths in the constructor. It now rejects a Table_map event
whose declared database name, or whose declared table name, exceeds
NAME_LEN, before it sizes the name buffers, so is_valid() returns
false. It also terminates each name it copies, so a source that holds
no null does not leave the name unterminated. A master naming a table
with an oversized name now stops the slave's SQL thread with
ER_SLAVE_RELAY_LOG_READ_FAILURE instead of crashing it.

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>
drrtuy
chore: add DuckDB version info function.
Daniel Black
RocksDB: compile fix std::replace requires algorithm header

Otherwise it compile fails.

Found in clang-24.
drrtuy
feat: update DuckDB submodule to 1.5.5.
Daniel Black
MDEV-40749 period.create test leaks/faults in asan

Selecting from the information_schema.plugins causes the loading
of all plugins. Because rockdb leaks, and duckdb triggers an
address sanitizer warning on shutdown avoid this table.

Use the information_schema.ENGINES to validate that InnoDB is
disabled per the original request in the review of MDEV-32205.
Kristian Nielsen
Remove Format_description_log_event::event_type_permutation

dead) code remains that accesses the member. This patch removes the member
and the related dead code properly.

Signed-off-by: Kristian Nielsen <[email protected]>
Sergei Petrunia
Make sql_parallel_thread self-contained

- Move 'reaped' and 'kill_signal' OUT to sql_parallel_workers
- Move 'workers' and 'nworkers' OUT to sql_parallel_workers
- Move server_threads update logic IN.
Alexander Barkov
MDEV-28498 Incorrect information in file: './test/t0.frm' on CREATE TABLE

Applying HEX encoding write writting an ENUM/SET TYPELIB to FRM
if the TYPELIB has 0x00 bytes in the value.

This HEX encoding was earlier used only to write UCS2/UTF16/UTF32 TYPELIBs.

A new flag FIELDFLAG_FRM_HEX_ENCODED_TYPELIB was added to indicate
that the TYPELIB is hex encoded. It's used only inside FRM.
Note, it's mangled with FIELDFLAG_TREAT_BIT_AS_CHAR.
This should not be harmful:
- BIT and ENUM/SET columns are handled by two separate code branches
  when opening an FRM
- The flag is unset immediately after decoding TYPELIB, so the rest
  of the code does not se an unexpected flag combination.
ParadoxV5
MDEV-40647 OOB read in IO Thread if the FDEv does not support Rotate Events

If the replication IO Thread receives a Rotate event following a Format
Description event (FDE) with no post-header length for Rotate events,
the Rotate event’s parser constructor indexes
the FDE’s post-header lengths array out of bounds.
This commit defends against this situation by checking before the
constructor that the FDE describes Rotate events as recognized at all.

In practice, because the Binlog Dump thread generates a Fake
`ROTATE_EVENT` **before** sending the FDE, it has pinned
Rotate events’ post-header length to 8 regardless of FDEs.
This fix solution considers that the FDE’s description should still be
respected, matching the constructor.

Reviewed-by: Kristian Nielsen <[email protected]>
Marko Mäkelä
fixup! b625be27ca0a0b24730dd19d37d8bbf85917156c
Oleksandr Byelkin
Merge branch '11.4' into 11.8
Sergei Petrunia
Make sql_parallel_thread module isolated.

This required adding
virtual pwt_worker_base::on_fatal_error()

which pwt_worker overrides.

pwt_manager_base doesn't track if fatal_error has occurred
Should it? fatal_error is in sql_parallel_workers...
Georgi (Joro) Kodinov
A Days work: kept working on the services group
ParadoxV5
MDEV-40544 Assertion fail / Memory leak in `mariadb-binlog --force-read`

A checksum error in
`mariadb-binlog --verify-binlog-checksum --force-read` previously
resulted in both an error message and an Unknown event substitute,
the latter of which failed an assertion in debug builds or
became forgotten (memory leak) in non-debug (release) builds.

Since `mariadb-binlog --force-read` outputs “Unknown event”s
rather than errors in other invalid event cases,
this commit removes the error status from this situation to match.

Reviewed-by: Kristian Nielsen <[email protected]>
Daniel Black
MDEV-38405 Assertion `tbl->trn == 0' failed in _ma_set_trn_for_table

Aria bulk insert operations disables share->now_transaction meaning
a concurrent open of the stable table will have
trn == &dummy_transaction_object for its MARIA_HA object during opening.

As the _ma_set_trn_for_table is setting the trn, its harmless if the
current trn is the dummy_transaction_object.

Relax the assert to allow for this state.
Marko Mäkelä
squash! b625be27ca0a0b24730dd19d37d8bbf85917156c

aria_backup_end(): Removed.

aria_backup_start(): A crude single-threaded implementation
of copying files that do not belong to ACID storage engines.
Most files are copied in BACKUP_PHASE_NO_DDL after
flush_tables(thd, FLUSH_NON_TRANS_TABLES) has been invoked.
All ENGINE=Aria files (including TRANSACTIONAL=0) are copied
in BACKUP_PHASE_NO_COMMIT. Thanks to Andrzej Jarząbek
for implementing test cases and suggesting this logic.
Sergei Golubchik
rocksdb: don't abort early in submodule update

Fix for 1fb075512a7aeab8646a163cbb6f265c49f4c075 to allow
the ADD_SUBMODULE to perform updates.
Oleksandr Byelkin
Merge branch '10.11' into 11.4
Daniel Black
MDEV-34482 main.events_processlist test fix

As the test result is dependent of SHOW PROCESSLIST output,
adjust the wait condition to ensure the state is in sleeping
rather than "init" or another state.
Brandon Nesterenko
MDEV-40492: Oversized Fake Rotate Event can Crash Slave

A slave can crash when the master's binlog_checksum setting differs
from the checksum the slave's relay log carries. On the first Rotate
event of such a connection, the slave IO thread copies the event onto
a stack buffer sized for the longest binary log file name a master can
send. A master that names a longer file makes the copy run off the end
of that buffer, and what it writes past the end is the file name
itself. A name that overruns the buffer by a few hundred bytes reaches
only the other buffers of the same stack frame, which the slave does
not read on this path, so it keeps running. A name that overruns the
whole frame takes the return address with it, and the slave crashes.
Builds with AddressSanitizer stop at the copy and report a
stack-buffer-overflow.

The fake Rotate event that opens a connection is rewritten when the
master's checksum policy and the relay log's disagree. One case adds a
checksum to the event, and the other strips one. Both copy the event
into rot_buf using the length the master sent, and neither compared
that length against the size of rot_buf. The Rotate_log_event
constructed just above bounds its own copy of the file name at
FN_REFLEN-1, which leaves the length the master sent untouched.
rot_buf was itself one checksum shorter than the event the first case
produces from the longest file name.

Bound the event before either case copies it. queue_event() now
compares the event's length against the longest Rotate a master can
send: the two headers, a file name of at most FN_REFLEN bytes, and the
checksum the master's own policy puts on the event. rot_buf now
reserves that checksum length as well. A Rotate event naming a longer
file stops the slave IO thread with an error reporting the length the
master sent, instead of running off the end of the buffer.

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>
Brandon Nesterenko
MDEV-40646: Undersized Table_map Metadata can Crash Slave

A slave can crash or leak heap content when its master sends a Table_map
event that declares columns whose types need more field metadata than
the event carries. Building its description of the table, the slave
reads past the end of the field metadata the event holds. The bytes
beyond the buffer are decoded as column metadata.

A Table_map event carries a column count, one type byte per column,
and a block of field metadata. A column's type fixes how many metadata
bytes it consumes: two for the string, enum, set, bit, varchar, and
decimal types, one for the blob, float, and high precision temporal
types, none for the rest. The master writes exactly that many bytes
per column, so the metadata block a well formed event carries is the
sum over its columns. The slave sizes the metadata block from the
event and, while building its table_def, walks the columns in order
and reads each type's metadata bytes from that block.

The slave checked only that the metadata block was no larger than two
bytes per column. It never checked the block against the column types
themselves. table_def() then walked every declared column and indexed
the metadata block for each, so a column whose type needed metadata
the block did not hold read past the end of it. A column count large
enough carried that read far past the allocation.

The Table_map reader now sums the metadata the declared column types
consume and rejects the event when the block is smaller than that
sum, so is_valid() reports the event as invalid and the SQL thread
stops with a relay log read failure before any column is decoded. A
block larger than the sum is still accepted, because every consumer
reads only the bytes the types call for. A column type this slave
does not recognize counts as zero in the sum, so Table_map events
from a newer master using such a type still parse. A Table_map whose
metadata cannot cover its column types now stops the slave with an
error instead of reading past the buffer.

Note that the regression test does not crash a regular debug build;
an ASAN build is required to show the invalid memory access.  The 24
column table map it injects overruns the metadata allocation by a
bounded amount that lands in mapped heap, which a debug build reads
without faulting and then stops the SQL thread with
ER_SLAVE_CONVERSION_FAILED on the column type mismatch.

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>