Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
ParadoxV5
MDEV-40544 Assertion fail / Memory leak in `mariadb-binlog --force-read`

A checksum error in
`mariadb-binlog --verify-binlog-checksum --force-read` previously
resulted in both an error message and an Unknown event substitute,
the latter of which failed an assertion in debug builds or
became forgotten (memory leak) in non-debug (release) builds.

Since `mariadb-binlog --force-read` outputs “Unknown event”s
rather than errors in other invalid event cases,
this commit removes the error status from this situation to match.
drrtuy
fix: MDEV-40651 ALTER TABLE and DDL in general is now executed in an atomic manner b/c previously MariaDB DDL triggered multiple autocommit DDL statements in DuckDB some of those can fail leaving table in unsable state.
Sergei Golubchik
mariadb-test: don't use strlen/strend, support lines with embedded \0

add a test
Oleksandr Byelkin
Merge branch 'bb-11.4-release' into bb-11.8-release
Ahmad
MDEV-39858: Reloading COSINE metric index from disk degrades search recall due to abs2 quantization noise

When a vector is created in-memory using FVector::create() during normal inserts, its squared magnitude (abs2) under the COSINE metric is hardcoded to 0.5f.

However, when the index is reloaded from disk (after a server restart, FLUSH TABLES, or ALTER TABLE), the index uses FVectorNode::load_from_record(). This method reads the stored scale and quantized int16 coordinates from the database record, and runs postprocess(). Inside postprocess(), abs2 is dynamically recomputed using floating-point math:
abs2 = subabs2 + scale * scale * dot_product(d, d, vec_len) / 2;

Because the coordinates stored on disk are quantized int16 values, this recalculation introduces rounding noise.

This affects high dimensions datasets, and it is increasing as M increases.

Added hardcoded abs2=0.5 to FVectorNode::load_from_record and removed postprocess()
ParadoxV5
MDEV-40674 Include the checksum for non-corrupted Unknown events

This commit reörders code so the checksum is populated after
`mariadb-binlog --force` generates `Unknown_log_event` substitutes.
Previously, `mariadb-binlog --force` inconsistently omitted those
checksums from the output even if the checksum is persumably usable.

After merging to 11.4 (MDEV-31273), this commit will also fix «
MDEV-40542 MSAN use-of-uninitialized-value on Unknown_log_event::read_checksum_alg
», which was exposed by MDEV-31273’s removal
of the base `Log_event::checksum_alg` field.
drrtuy
fix: hardening against unknown SQL injection vectors running DuckDB.
Yuchen Pei
MDEV-24813 Signal full scan to storage engines.

When starting to do a full table/index scan without a WHERE or JOIN
condition, tell the storage engine so and the corresponding
ulong-truncated LIMIT.

Include an innodb implementation: added an innodb switch
table_lock_on_full_scan, so that when the switch is on, on receiving
the full scan signal from the sql layer, if the truncated LIMIT is
ULONG_MAX (likely no LIMIT), attempt to acquire a table lock.

Updated tests that have different results with the switch on.

The three deadlock_*_race tests cannot reach their DEBUG_SYNC race
under a table lock (it degenerates to a timeout), and the three I_S
tests only restate a lock-mode change already covered
by innodb_full_scan.test.

(Comment and code edited by Sergei Petrunia <[email protected]> and
Thirunarayanan Balathandayuthapani <[email protected]>)
Alexey Yurchenko
MDEV-40179 Found N prepared transactions after mariabackup SST

With log_bin=ON a transaction is committed via two-phase commit (the
binary log is the second participant), so it passes through the InnoDB
XA-prepare state. While a donor is held in BLOCK_COMMIT for a mariabackup
backup, its parallel appliers (wsrep_slave_threads > 1) leave one or more
such writesets prepared-but-not-yet-committed, and the snapshot captures
them. On a freshly SST'd joiner nothing resolves these prepared
transactions: binlog crash recovery does not run (the joiner has no in-use
binlog to recover from), and the wsrep continuity-based commit is inactive
because wsrep_emulate_bin_log is FALSE when log_bin is ON. The leftover
prepared transactions then abort startup with "Found <N> prepared
transactions!". Note this does not depend on the prepared set being
non-contiguous - even a contiguous run aborts, because nothing commits
or rolls it back.

Rollback these transactions in xarecover_handlerton(). If rollback fails
flag error to cause unireg_abort().
ParadoxV5
MDEV-40674 Include the checksum for non-corrupted Unknown events

This commit reörders code so the checksum is populated after
`mariadb-binlog --force` generates `Unknown_log_event` substitutes.
Previously, `mariadb-binlog --force` inconsistently omitted those
checksums from the output even if the checksum is presumably  usable.

After merging to 11.4 (MDEV-31273), this commit will also fix «
MDEV-40542 MSAN use-of-uninitialized-value on Unknown_log_event::read_checksum_alg
», which was exposed by MDEV-31273’s removal
of the base `Log_event::checksum_alg` field.
ParadoxV5
fixup MDEV-40674: `have_debug.inc`
Rex
MDEV-40465 table map consulted during setup fields on unfixed items

In setup_fields() we call item->update_used_tables() before split_sum_func
so that used_tables() is not consulted before the caches are set up on the
2nd execution of a prepared statement (fixes a 1st/2nd execution result
mismatch in main.subselect_nulls under --ps-protocol).

That recalculation exposes items whose caches are read while still being
(re)built, so add the guards it now depends on:
- Item_field::used_tables(): return 0 when field / field->table is not
  yet set, instead of dereferencing a null pointer.
- Item_direct_view_ref::used_tables(): return 0 when the item is not
  fixed yet, instead of asserting.
- Item_func::fix_fields(): reset used_tables_cache/const_item_cache at
  entry (assignment) instead of asserting they are already clear, so a
  re-fix is idempotent.

Remove select,ps.rdiff / select_jcl6,ps.rdiff / select_pkeycache,ps.rdiff:
the extra "resolved in SELECT #1" notes they recorded no longer appear
under --ps-protocol, so the select tests now match without an rdiff.
Yuchen Pei
MDEV-40486 Length check for vector fields in CREATE TABLE ... SELECT

The changes of MDEV-39558 2b6529426a7e7c65d286e093d84138be9dcc34a3
added length check assertion in Field_varstring constructors, and
length check in type inference for SELECT set operations, to emit
errors before reaching the assertions.

That change caused an error to turn into an assertion failure in a
separate path, when the length limit violation is not detected before
tripping the assertion. So in this patch we fix it by adding an
earlier length check in that path.

The reason that we place this check inside
Item_func_vec_fromtext::fix_length_and_dec rather than say
`create_field_for_create_select is for consistency:

If

create table t1 as select
vec_fromtext(concat('[',group_concat(1),']')) as c1 from seq_1_to_64;

fails due to length limit violation, then so should

create table t1 (v vector(64) not null);
insert into t1 select vec_fromtext(concat('[',group_concat(1),']'))
from seq_1_to_64;

Also use max_char_length() instead of max_length. This is a more
accurate length of characters. And add handling of empty string edge
case. Added testcases accordingly.

The change that uses max_char_length() causes side effects where
creating a table using a VEC_FROMTEXT(CHAR(1)) would result in a
0-dimensional vector field. This is accurate but 0-dim vector table
fields should not be allowed. So we make cases like this result in
a one-dimensional field.

Also fixed the underflow in (args[0]->max_length - 1) * 2 when the
arg's max length is 0. Previously this underflow would cause

create table t1 select vec_fromtext(NULL)

to fail with ER_TOO_BIG_FIELDLENGTH. Now it will be a VECTOR(1) field
Oleksandr Byelkin
Merge branch '10.6' into bb-10.11-release
drrtuy
fix: resolve SQL injection imposed by lexer semantics difference b/w
MariaDB and DuckDB.
Fariha Shaikh
MDEV-39459 Fix bad sync pattern for chain replication MTR tests

In chain replication (1->2->3), syncing only server_3 after
save_master_gtid on server_1 does not guarantee server_2 has committed,
because server_2's binlog dump thread can send events to server_3 before
commit_ordered() completes on server_2.

Fix affected rpl tests by syncing server_2 before server_3, and update
result files accordingly.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
Yuchen Pei
MDEV-40486 Length check for vector fields in CREATE TABLE ... SELECT

The changes of MDEV-39558 2b6529426a7e7c65d286e093d84138be9dcc34a3
added length check assertion in Field_varstring constructors, and
length check in type inference for SELECT set operations, to emit
errors before reaching the assertions.

That change caused an error to turn into an assertion failure in a
separate path, when the length limit violation is not detected before
tripping the assertion. So in this patch we fix it by adding an
earlier length check in that path.

The reason that we place this check inside
Item_func_vec_fromtext::fix_length_and_dec rather than say
`create_field_for_create_select is for consistency:

If

create table t1 as select
vec_fromtext(concat('[',group_concat(1),']')) as c1 from seq_1_to_64;

fails due to length limit violation, then so should

create table t1 (v vector(64) not null);
insert into t1 select vec_fromtext(concat('[',group_concat(1),']'))
from seq_1_to_64;

Also use max_char_length() instead of max_length. This is a more
accurate length of characters. And add handling of empty string edge
case. Added testcases accordingly.

The change that uses max_char_length() causes side effects where
creating a table using a VEC_FROMTEXT(CHAR(1)) would result in a
0-dimensional vector field. This is accurate but 0-dim vector table
fields should not be allowed. So we make cases like this result in
a one-dimensional field.

Also fixed the underflow in (args[0]->max_length - 1) * 2 when the
arg's max length is 0. Previously this underflow would cause

create table t1 select vec_fromtext(NULL)

to fail with ER_TOO_BIG_FIELDLENGTH. Now it will be a VECTOR(1) field
Yuchen Pei
MDEV-40632 Use my_fprintf in trx_print_low

To fix for windows %p in `fprintf(f, "TRANSACTION (%p)", trx);` is not
prefixed with 0x
Oleksandr Byelkin
Merge branch '10.6' into bb-10.11-release
Yuchen Pei
MDEV-24813 [to-squash] Fix avoid_deadlock_with_blocked failure in 32bit

Similar to the previous sub-block, could be a race condition
Oleksandr Byelkin
Merge branch '11.8' into bb-11.8-release
Yuchen Pei
MDEV-24813 [to-squash] disable full scan combination for a block

An alternative fix of the avoid_deadlock_with_blocked failure in 32bit
Alexey Yurchenko
MDEV-38147 error 1950 after mariabackup SST with gtid_strict_mode=ON

After a mariabackup SST the joiner could fail with

  ER_GTID_STRICT_OUT_OF_ORDER (error 1950)

while re-binlogging transactions received over IST.

The cause is that the binary log copied from the donor carries a
Gtid_list whose position can be ahead of the storage-engine snapshot
because getting snapshot and binlog are no mutually atomic, so transactions
can be present in the copied binlog that are not committed in the copied
engine snapshot.
After the SST the joiner reports the (committed) engine position to the
cluster, IST resends those transactions, and re-binlogging them under
gtid_strict_mode=ON collides with the ahead Gtid_list -> error 1950.
(MDEV-34483 made the engine snapshot stop short of the binlog, which is
what exposed this.)

The copied binary log carries no transactions the joiner needs - only a
Gtid_list - so instead of shipping and then having to truncate/reconcile
it, the joiner now starts a fresh binary log and seeds its GTID position
from the storage-engine checkpoint during recovery. That checkpoint is
the committed cluster position, i.e. exactly where IST resumes, so the
joiner's binary log stays in lockstep with the rest of the cluster and
no out-of-order GTID can occur.

This works for both wsrep_gtid_mode settings; only the binlog domain of
the cluster stream differs:

  - wsrep_gtid_mode=ON : wsrep_gtid_domain_id (cluster writes are
    re-tagged to it), which is the domain stored in the checkpoint;
  - wsrep_gtid_mode=OFF: gtid_domain_id (cluster writes keep the node's
    configured domain).

Async-replica positions (mysql.gtid_slave_pos) are part of the engine
snapshot and survive the SST unchanged, so a Galera node can still serve
as an async master or replica across the SST.

This commit:
- sql/log.cc: adds wsrep_seed_binlog_gtid_state(), called from
  do_binlog_recovery() when the joiner has no binary log, seeding the
  binlog GTID state for the cluster domain to the SE checkpoint position.
- scripts/wsrep_sst_mariabackup.sh: no longer moves the donor's binary
  log into place on the joiner.
- extra/mariabackup: backward compatibility: keep shipping binlog file
  in SST but
  - on donor fix the race between rotation and shipping so that the file
    shipped is the one that had been rotated;
  - on joiner discard shipped binlog in favour of one generated by
    wsrep_seed_binlog_gtid_state().
- sql/wsrep_sst.cc: logs the position actually adopted from storage
  (the authoritative post-SST position) rather than the script-reported
  one.
- sql/handler.cc: downgrades the "Discovered discontinuity in recovered
  wsrep transaction XIDs" message in wsrep_order_and_check_continuity()
  from warning to debug level. With parallel appliers a snapshot
  routinely captures prepared XIDs that are not contiguous with the
  engine checkpoint, so this is normal during SST recovery and of no
  value in regular operation; the transactions past the checkpoint are
  re-delivered by the cluster (IST/SST) regardless.
- Adds an MDEV-38147 MTR test reproducing the issue.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
ParadoxV5
MDEV-40544 Assertion fail / Memory leak in `mariadb-binlog --force-read`

A checksum error in
`mariadb-binlog --verify-binlog-checksum --force-read` previously
resulted in both an error message and an Unknown event substitute,
the latter of which fails assertion in debug builds or
becomes forgotten (memory leak) in non-debug (release) builds.

Since `mariadb-binlog --force-read` outputs “Unknown event”s
rather than errors in other invalid event cases.
this commit removes the error status from this situation to match.
Oleksandr Byelkin
Merge branch '10.11' into bb-10.11-release
Oleksandr Byelkin
Merge branch 'bb-11.4-release' into bb-11.8-release
Alexey Yurchenko
MDEV-40179 Found N prepared transactions after mariabackup SST

With log_bin=ON a transaction is committed via two-phase commit (the
binary log is the second participant), so it passes through the InnoDB
XA-prepare state. While a donor is held in BLOCK_COMMIT for a mariabackup
backup, its parallel appliers (wsrep_slave_threads > 1) leave one or more
such writesets prepared-but-not-yet-committed, and the snapshot captures
them. On a freshly SST'd joiner nothing resolves these prepared
transactions: binlog crash recovery does not run (the joiner has no in-use
binlog to recover from), and the wsrep continuity-based commit is inactive
because wsrep_emulate_bin_log is FALSE when log_bin is ON. The leftover
prepared transactions then abort startup with "Found <N> prepared
transactions!". Note this does not depend on the prepared set being
non-contiguous - even a contiguous run aborts, because nothing commits
or rolls it back.

Rollback these transactions in xarecover_handlerton(). If rollback fails
flag error to cause unireg_abort().
Oleksandr Byelkin
Merge branch '11.4' into bb-11.4-release
aleksei.bukhalov
docs: add public dataset tutorials for the DuckDB engine

Add two step-by-step tutorials:

- NYC Taxi Trips: load TLC Parquet via run_in_duckdb() + read_parquet(),
  keep the zone lookup in InnoDB, then join the two engines in a plain
  SELECT. Follows the structure of the ClickHouse taxi tutorial.

- OWID CO2 Emissions: load a commit-pinned CSV revision, run
  aggregations and a LAG() window function.

Both tutorials point at security-model.md before telling the reader to
enable duckdb_allow_run_in_duckdb, and both note that ordinary
INSERT/UPDATE/DELETE also work on ENGINE=DuckDB tables so that
run_in_duckdb() does not look like the only write path.

Also documents TIMESTAMPDIFF in mariadb-duckdb-incompatibilities.md and
adds a tutorials section to the README.

Every SQL block and every expected output was extracted from the
markdown, executed, and compared cell by cell on MariaDB 11.4.13 built
from this branch; both data sets are pinned so results stay
reproducible.
Yuchen Pei
MDEV-40486 Length check for vector fields in CREATE TABLE ... SELECT

The changes of MDEV-39558 2b6529426a7e7c65d286e093d84138be9dcc34a3
added length check assertion in Field_varstring constructors, and
length check in type inference for SELECT set operations, to emit
errors before reaching the assertions.

That change caused an error to turn into an assertion failure in a
separate path, when the length limit violation is not detected before
tripping the assertion. So in this patch we fix it by adding an
earlier length check in that path.

The reason that we place this check inside
Item_func_vec_fromtext::fix_length_and_dec rather than say
`create_field_for_create_select is for consistency:

If

create table t1 as select
vec_fromtext(concat('[',group_concat(1),']')) as c1 from seq_1_to_64;

fails due to length limit violation, then so should

create table t1 (v vector(64) not null);
insert into t1 select vec_fromtext(concat('[',group_concat(1),']'))
from seq_1_to_64;

Also use max_char_length() instead of max_length. This is a more
accurate length of characters. And add handling of empty string edge
case. Added testcases accordingly.

The change that uses max_char_length() causes side effects where
creating a table using a VEC_FROMTEXT(CHAR(1)) would result in a
0-dimensional vector field. This is accurate but 0-dim vector table
fields should not be allowed. So we make cases like this result in
a one-dimensional field.

Also fixed the underflow in (args[0]->max_length - 1) * 2 when the
arg's max length is 0. Previously this underflow would cause

create table t1 select vec_fromtext(NULL)

to fail with ER_TOO_BIG_FIELDLENGTH. Now it will be a VECTOR(1) field
Sergei Golubchik
MDEV-40670 qc_info OOB read if query contains \0

qc_info cannot parse query cache key, this cannot be done
without changing query cache key format.

let's limit the (possibly incorrect) db_length to make sure
it does not extend beyond the db name end, this way there will be no
OOB reads and no information leak.
Oleksandr Byelkin
Merge branch 'bb-10.11-release' into bb-11.4-release
Fariha Shaikh
MDEV-39459 Fix bad sync pattern for chain replication MTR tests

In chain replication (1->2->3), syncing only server_3 after
save_master_gtid on server_1 does not guarantee server_2 has committed,
because server_2's binlog dump thread can send events to server_3 before
commit_ordered() completes on server_2.

Fix affected rpl tests by syncing server_2 before server_3, and update
result files accordingly.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
Rex Johnston
MDEV-38801 Item_sum & Item_cache implement only shallow copy

We implement {Item_cache,Item_cache_row,Item_sum}::deep_copy, rather
than defaulting to shallow_copy_with_checks.