Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Fariha Shaikh
MDEV-39153 Fix sporadic main.change_master_default mismatch

The test used "restart_abort:" in the expect file, which MTR never
recognized. It fell through to the else branch, deleted restart_opts,
and started the server with defaults (heartbeat_period=60 instead of 0).

Replace restart_abort with direct --exec $MYSQLD calls.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
bsrikanth-mariadb
MDEV-39226: Add multi-table update, delete feature
Brandon Nesterenko
MDEV-40648 Fix rpl.rpl_corruption

The test would induce a corrupt binlog event on the master and send it
to the slave, and with the new event validation logic, this could
trigger the slave to output the new error message added in MDEV_40648.

Add this error message as an allowed error during the executin of this
test

Signed-off-by: Brandon Nesterenko <[email protected]>
ParadoxV5
MDEV-40365 OOB read on malformed `Format_description_log_event`

The binary-parsing Format Description Event constructor did not
validate content length beyond the superclass `is_valid()` call.
When parsing a malformed FDE, to load the content fields added in
FDE v4 that are missing in this not-FDE, the parser constructor
would read from erroneous memory locations beyond the buffer.
If this did not outright crash the program, this would corrupt the FDE.
With the Format Description playing a critical role in determining how
to parse the events to follow, a corrupted FDE would also corrupt (or
trigger a crash in) the parsing of subsequent non-FDE events as well.

This commit fills in the validation with a FDE-specific guard.
It adds the FDE constant `ST_POST_HEADER_LEN_OFFSET` to assist
with comparing to the correct minimum size in the future.
(Both of these points are designed to merge with the
superclass’s guard as part of the MDEV-30128 merger).

Reviewed-by: Kristian Nielsen <[email protected]>
ParadoxV5
MDEV-40674 Include the checksum for non-corrupted Unknown events

This commit reörders code so the checksum is populated after
`mariadb-binlog --force` generates `Unknown_log_event` substitutes.
Previously, `mariadb-binlog --force` inconsistently omitted those
checksums from the output even if the checksum is presumably  usable.

After merging to 11.4 (MDEV-31273), this commit will also fix «
MDEV-40542 MSAN use-of-uninitialized-value on Unknown_log_event::read_checksum_alg
», which was exposed by MDEV-31273’s removal
of the base `Log_event::checksum_alg` field.

Reviewed-by: Kristian Nielsen <[email protected]>
Oleksandr Byelkin
Merge fix
Ahmad
MDEV-39858: Reloading COSINE metric index from disk degrades search recall due to abs2 quantization noise

When a vector is created in-memory using FVector::create() during normal inserts, its squared magnitude (abs2) under the COSINE metric is hardcoded to 0.5f.

However, when the index is reloaded from disk (after a server restart, FLUSH TABLES, or ALTER TABLE), the index uses FVectorNode::load_from_record(). This method reads the stored scale and quantized int16 coordinates from the database record, and runs postprocess(). Inside postprocess(), abs2 is dynamically recomputed using floating-point math:
abs2 = subabs2 + scale * scale * dot_product(d, d, vec_len) / 2;

Because the coordinates stored on disk are quantized int16 values, this recalculation introduces rounding noise.

This affects high dimensions datasets, and it is increasing as M increases.

Added hardcoded abs2=0.5 to FVectorNode::load_from_record and removed postprocess()
Sergei Golubchik
MDEV-40229 I_S.VECTOR_INDEXES: Unclear or inconsistent semantics of INDEX_SIZE

11.8 fix for I_S.TABLES.INDEX_LENGTH column

Take into account both data_file_length and index_file_length,
they both take space and both belong to the vercor index.
Lawrin Novitsky
ODBC-502 Driver can read and send to server memore past query end

This happened if UTF16 character with UnixODBC(i.e. Linux and not
Window) would have lower byte as the high byte in a surrogate pair
leading unit. One of helper functions erroneously took such character as
leading unit of a surrogate pair, and vise versa - would not recognize
real leading unit as such. That could lead to calculated byte length
would be longer, than the real one, it will be used for query text
transcoding to connection charset and thus memory past he end of query
read and sent to the server.
SQLNativeSqlW had the problem caused be similar reasons, but it did not
need length in char calculation, but in SQLWCHAR units.
The tast has been added and SQLNativeSqlW test thas been extended.
ParadoxV5
MDEV-39485 Heap-buffer-overflow upon read in `Rows_log_event` constructor

MariaDB recognizes Version 2 Rows Events from MySQL, including the
format of the “extra data” field added in this version. (MDEV-5115)

When parsing this extra data according to the format, whether this data
has sufficient length was only checked by assertions in the
`Rows_log_event` constructor and the `mariadb-binlog --verbose` printer.
When parsing an event with malformed extra data, these assertions
* would straight up terminate the program in debug builds.
* were stripped in non-debug (release) builds.
  This would render the parser defenseless to reading from erroneous
  memory locations outside of the containing event, which will either
  crash the program or, for `mariadb-binlog --verbose`, snapshot the
  running memory to be exposed when outputting the event.

This commit replaces those assertions with an actual validity check.

Since MariaDB does not generate v2 Rows Events,
the included test uses a handcrafted binlog file.

Reviewed-by: Kristian Nielsen <[email protected]>
Daniel Black
sql_test: mallinfo2 msan exclusion no longer needed

MSAN interceptor was added in clang-18.1.
ParadoxV5
MDEV-40366 OOB read on malformed `Format_description_log_event`

Neither the binary-parsing Format Description Event constructor
nor the constructor-bypassing `get_checksum_alg()` function
validated the content length of the passed event buffer.
If they receive an FDE with undersized contents,
they would obtain corrupt results from an erronous memory location,
if not outright crash the program with that memory error.

This commit fixes both sites by adding content length checks.
Because the goal is not to solve the existence of two binary parsers,
`get_checksum_alg()` receives a check duplicated from the Format
Description constructor and is no longer a function that never errors.
Note, it is implementation detail that `get_checksum_alg()`’s
fix catches the invalidity before any code reaches the
parser-contructor’s fix, though the latter would come
to effect if we refactor `get_checksum_alg()` away.

Reviewed-by: Kristian Nielsen <[email protected]>
Oleksandr Byelkin
Merge branch '11.4' into bb-11.4-release
Oleksandr Byelkin
Merge branch '11.8' into bb-11.8-release
Fariha Shaikh
MDEV-33660 Add note when setting AUTO_INCREMENT lower than next value

Currently, ALTER TABLE ... AUTO_INCREMENT = N silently fails when using
values lower than the next AUTO_INCREMENT value in the column.

Add a note when attempting to set AUTO_INCREMENT to a lower value than
the next AUTO_INCREMENT value, and maintain the higher value.

Update existing test suites and add a new test to the maria suite for
the newly added note.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
Daniel Black
MDEV-17846 Wrong result with grouping select (fix)

Prevent unused variable 'ref_type'  warnings on non-debug builds.
Yuchen Pei
MDEV-40486 Length check for vector fields in CREATE TABLE ... SELECT

The changes of MDEV-39558 2b6529426a7e7c65d286e093d84138be9dcc34a3
added length check assertion in Field_varstring constructors, and
length check in type inference for SELECT set operations, to emit
errors before reaching the assertions.

That change caused an error to turn into an assertion failure in a
separate path, when the length limit violation is not detected before
tripping the assertion. So in this patch we fix it by adding an
earlier length check in that path.

The reason that we place this check inside
Item_func_vec_fromtext::fix_length_and_dec rather than say
`create_field_for_create_select is for consistency:

If

create table t1 as select
vec_fromtext(concat('[',group_concat(1),']')) as c1 from seq_1_to_64;

fails due to length limit violation, then so should

create table t1 (v vector(64) not null);
insert into t1 select vec_fromtext(concat('[',group_concat(1),']'))
from seq_1_to_64;

Also use max_char_length() instead of max_length. This is a more
accurate length of characters. And add handling of empty string edge
case. Added testcases accordingly.

The change that uses max_char_length() causes side effects where
creating a table using a VEC_FROMTEXT(CHAR(1)) would result in a
0-dimensional vector field. This is accurate but 0-dim vector table
fields should not be allowed. So we make cases like this result in
a one-dimensional field.

Also fixed the underflow in (args[0]->max_length - 1) * 2 when the
arg's max length is 0. Previously this underflow would cause

create table t1 select vec_fromtext(NULL)

to fail with ER_TOO_BIG_FIELDLENGTH. Now it will be a VECTOR(1) field
Oleksandr Byelkin
Merge branch 'bb-10.11-release' into bb-11.4-release
drrtuy
chore: remove jemalloc extension from DuckDB CMake b/c since 1.5.4 is is a part of DuckDB core.
Daniel Black
MDEV-40243 Fix MEMORY_LEAK_C leaks in mariadb-dump  (rockdb tests)

With memory leaks fixed in the rocksdb.mysqldump/mysqldump2 no
longer need to run with leak detection disabled.

There tests are still disabled as there's no --rockdb arg to
mariadb-dump, but removing so there's no precidence to ignoring
leaks.

ref: 2217477fb8fc82f4921d9d13afcd24b1d86b34d6
Brandon Nesterenko
MDEV-40644: Slave SQL Thread Overflow on Malformed Table_map_log_event

A slave can crash when its master sends a Table_map event whose table
name is longer than an identifier can be. The SQL thread applying the
event copies the name into a fixed buffer and overruns it, and the SQL
thread crashes.

A Table_map event carries its database and table names each behind a
one-byte length. The constructor that parses the event sizes a buffer
for each name from its declared length, and do_apply_event() copies
both names out of the event into NAME_LEN+1 byte buffers with strmov().
strmov() copies up to the terminating null. A real identifier is at
most NAME_LEN bytes, so the name fits the buffer and carries that null.

The constructor never bounded the two lengths. Each length is read as a
byte, so each can reach 255. The constructor copied each name with
strncpy() over the declared length, which writes no terminating null
when the source holds none. do_apply_event() then copied the name into
its NAME_LEN+1 byte buffer with strmov(), which ran past the buffer
until it reached a null elsewhere in the heap. pack_info() read the
same names with %s for SHOW RELAYLOG EVENTS, past the buffer in the
same way.

Bound the lengths in the constructor. It now rejects a Table_map event
whose declared database name, or whose declared table name, exceeds
NAME_LEN, before it sizes the name buffers, so is_valid() returns
false. It also terminates each name it copies, so a source that holds
no null does not leave the name unterminated. A master naming a table
with an oversized name now stops the slave's SQL thread with
ER_SLAVE_RELAY_LOG_READ_FAILURE instead of crashing it.

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>
drrtuy
chore: add DuckDB version info function.
Sergei Golubchik
MDEV-40406 hide #mysql50# under old mode
Daniel Black
RocksDB: compile fix std::replace requires algorithm header

Otherwise it compile fails.

Found in clang-24.
drrtuy
feat: update DuckDB submodule to 1.5.5.
Alexander Barkov
MDEV-28498 Incorrect information in file: './test/t0.frm' on CREATE TABLE

Applying HEX encoding write writting an ENUM/SET TYPELIB to FRM
if the TYPELIB has 0x00 bytes in the value.

This HEX encoding was earlier used only to write UCS2/UTF16/UTF32 TYPELIBs.

A new flag FIELDFLAG_FRM_HEX_ENCODED_TYPELIB was added to indicate
that the TYPELIB is hex encoded. It's used only inside FRM.
Note, it's mangled with FIELDFLAG_TREAT_BIT_AS_CHAR.
This should not be harmful:
- BIT and ENUM/SET columns are handled by two separate code branches
  when opening an FRM
- The flag is unset immediately after decoding TYPELIB, so the rest
  of the code does not se an unexpected flag combination.
ParadoxV5
MDEV-40647 OOB read in IO Thread if the FDEv does not support Rotate Events

If the replication IO Thread receives a Rotate event following a Format
Description event (FDE) with no post-header length for Rotate events,
the Rotate event’s parser constructor indexes
the FDE’s post-header lengths array out of bounds.
This commit defends against this situation by checking before the
constructor that the FDE describes Rotate events as recognized at all.

In practice, because the Binlog Dump thread generates a Fake
`ROTATE_EVENT` **before** sending the FDE, it has pinned
Rotate events’ post-header length to 8 regardless of FDEs.
This fix solution considers that the FDE’s description should still be
respected, matching the constructor.

Reviewed-by: Kristian Nielsen <[email protected]>
ParadoxV5
MDEV-40544 Assertion fail / Memory leak in `mariadb-binlog --force-read`

A checksum error in
`mariadb-binlog --verify-binlog-checksum --force-read` previously
resulted in both an error message and an Unknown event substitute,
the latter of which failed an assertion in debug builds or
became forgotten (memory leak) in non-debug (release) builds.

Since `mariadb-binlog --force-read` outputs “Unknown event”s
rather than errors in other invalid event cases,
this commit removes the error status from this situation to match.

Reviewed-by: Kristian Nielsen <[email protected]>
Lawrin Novitsky
ODBC-502 Driver can read and send to server memore past query end

This happened if UTF16 character with UnixODBC(i.e. Linux and not
Window) would have lower byte as the high byte in a surrogate pair
leading unit. One of helper functions erroneously took such character as
leading unit of a surrogate pair, and vise versa - would not recognize
real leading unit as such. That could lead to calculated byte length
would be longer, than the real one, it will be used for query text
transcoding to connection charset and thus memory past he end of query
read and sent to the server.
SQLNativeSqlW had the problem caused be similar reasons, but it did not
need length in char calculation, but in SQLWCHAR units.
The tast has been added and SQLNativeSqlW test thas been extended.
Daniel Black
MDEV-37224 Remove UBSAN limitation from MTR tests

Having a not_ubsan.inc as a test case exclusion mechanism is
allowing developers to ignore UBSAN issues. As undefined behaviour
detected at runtime or compile time isn't acceptable in the
code base, remove the exclusion.

The lotofstack test, the only user of not_ubsan.inc, has this
exclusion because the stack size under UBSAN lacks predictability.

Adjust its exclusion because of this criteria, and not its UBSAN
status.

Reviewer: Jimmy Hu <[email protected]>
Daniel Black
MDEV-38405 Assertion `tbl->trn == 0' failed in _ma_set_trn_for_table

Aria bulk insert operations disables share->now_transaction meaning
a concurrent open of the stable table will have
trn == &dummy_transaction_object for its MARIA_HA object during opening.

As the _ma_set_trn_for_table is setting the trn, its harmless if the
current trn is the dummy_transaction_object.

Relax the assert to allow for this state.
Oleksandr Byelkin
Merge branch 'bb-11.4-release' into bb-11.8-release
Daniel Black
MDEV-34482 main.events_processlist test fix

As the test result is dependent of SHOW PROCESSLIST output,
adjust the wait condition to ensure the state is in sleeping
rather than "init" or another state.
Oleksandr Byelkin
Merge branch '10.11' into bb-10.11-release
Alexander Barkov
MDEV-39587 Package-wide TYPE for variable declarations

SET sql_mode=ORACLE;
DELIMITER $$
CREATE OR REPLACE PACKAGE pkg AS
  -- Declare a package public data type
  TYPE varchar_array IS TABLE OF VARCHAR(2000) INDEX BY INTEGER;
END;
$$
DELIMITER ;
DELIMITER $$

CREATE OR REPLACE PROCEDURE p1 AS
  v pkg.varchar_array; -- Use the package public data type
BEGIN
  v(0):='test';
  SELECT v(0);
END;
$$
DELIMITER ;

Note, the change is done only for sql_mode=ORACLE, because the TYPE
declaration is not available for the default mode.

Where package-wide types are available
--------------------------------------
- Variabe list type:
    DECLARE var pkg1.type1;

- RETURN type for a package routine:
    CREATE FUNCTION .. RETURN pkg1.type1 ...

- Parameter type for a package routine:
    PROCEDURE p1(param1 pkg1.type1);

- Assoc array element type:
    TYPE assoc1_t IS TABLE OF pkg1.type1 ...

- REF CURSOR RETURN type:
    TYPE cur1_t IS REF CURSOR RETURN pkg1.type1;

Change details
--------------

- Adding a member Lex_length_and_dec_st::m_foreign_module_type
  It's set to true when the data type was initialized from a TYPE
  in foreign routine (e.g. in PACKAGE spec).
  It's needed to prevent use of qualified identifiers in public contexts,
  i.e. in schema public routine parameter types and schema publuc function
  RETURN types.
  Adding a helper method sp_head::check_applicability() which prevents
  use of qualified types in public context.

- Adding a helper method sp_head::raise_unknown_data_type().

- Adding methods LEX::set_field_type_typedef_package_spec() for
  2-step and 3-step qualified indentifiers.
  It's used in field_type_all_with_typedefs which covers cases:
  - Variabe list type        : DECLARE var pkg1.type1;
  - RETURN type              : CREATE FUNCTION .. RETURN pkg1.type1 ...
  - Parameter type          : PROCEDURE p1(param1 pkg1.type1);
  - Assoc array element type : TYPE assoc1_t IS TABLE OF pkg1.type1 ...

- Adding a method LEX::declare_type_ref_cursor_return_typedef().
  It handles cases when a new TYPE REF CURSOR RETURN is declared,
  for both for qualified RETURN types and non-qualified RETURN types:
  - TYPE cur0_t IS REF CURSOR RETURN rec1_t;
  - TYPE cur0_t IS REF CURSOR RETURN pkg1.rec1_t;
  - TYPE cur0_t IS REF CURSOR RETURN db1.pkg1.rec1_t;

  The code was moved from LEX::declare_type_ref_cursor() into
  LEX::declare_type_ref_cursor_return_typedef() and extended
  to cover qualified RETURN types.

- Adding a method Sql_path::find_package_spec_type().
  It iterates through all schemas specified in @@path and searches
  for the given type in the given package.

- Adding a helper method sp_pcontext::type_defs_add_ref_cursor()
  to reuse the code.

- Adding a new method sp_package::get_typedef() to search
  for TYPE definitions in PACKAGE specifications.

- Adding a new method sp_head::get_typedef_package_spec()
  to search for TYPE definitions used by a PROCEDURE or FUNCTION.

- Adding a helper method
    Sp_handler::sp_cache_routine_reentrant_suppress_errors
  Adding a method Sp_handler::find_package_spec().
Thirunarayanan Balathandayuthapani
MDEV-40621 InnoDB: Failing assertion: doc_id == src_node->last_doc_id

AuxRecordReader::default_word_processor(): InnoDB fails to consider
the ilist data can be stored externally while decoding the auxiliary
table record.
Brandon Nesterenko
MDEV-40646: Undersized Table_map Metadata can Crash Slave

A slave can crash or leak heap content when its master sends a Table_map
event that declares columns whose types need more field metadata than
the event carries. Building its description of the table, the slave
reads past the end of the field metadata the event holds. The bytes
beyond the buffer are decoded as column metadata.

A Table_map event carries a column count, one type byte per column,
and a block of field metadata. A column's type fixes how many metadata
bytes it consumes: two for the string, enum, set, bit, varchar, and
decimal types, one for the blob, float, and high precision temporal
types, none for the rest. The master writes exactly that many bytes
per column, so the metadata block a well formed event carries is the
sum over its columns. The slave sizes the metadata block from the
event and, while building its table_def, walks the columns in order
and reads each type's metadata bytes from that block.

The slave checked only that the metadata block was no larger than two
bytes per column. It never checked the block against the column types
themselves. table_def() then walked every declared column and indexed
the metadata block for each, so a column whose type needed metadata
the block did not hold read past the end of it. A column count large
enough carried that read far past the allocation.

The Table_map reader now sums the metadata the declared column types
consume and rejects the event when the block is smaller than that
sum, so is_valid() reports the event as invalid and the SQL thread
stops with a relay log read failure before any column is decoded. A
block larger than the sum is still accepted, because every consumer
reads only the bytes the types call for. A column type this slave
does not recognize counts as zero in the sum, so Table_map events
from a newer master using such a type still parse. A Table_map whose
metadata cannot cover its column types now stops the slave with an
error instead of reading past the buffer.

Note that the regression test does not crash a regular debug build;
an ASAN build is required to show the invalid memory access.  The 24
column table map it injects overruns the metadata allocation by a
bounded amount that lands in mapped heap, which a debug build reads
without faulting and then stops the SQL thread with
ER_SLAVE_CONVERSION_FAILED on the column type mismatch.

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>
Fariha Shaikh
MDEV-39459 Fix bad sync pattern for chain replication MTR tests

In chain replication (1->2->3), syncing only server_3 after
save_master_gtid on server_1 does not guarantee server_2 has committed,
because server_2's binlog dump thread can send events to server_3 before
commit_ordered() completes on server_2.

Fix affected rpl tests by syncing server_2 before server_3, and update
result files accordingly.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.