Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Sergei Golubchik
UBSAN: sql/table.h:237:16: runtime error: load of value 4, which is not a valid value for type 'bool'

the warning was about uninitialized `bool in_field_list`
let's initialize the whole ORDER when it's allocated.
drrtuy
fix: fix stack size warning.
Aleksey Midenkov
WITHOUT_ABI_CHECK followup

Followup for b337e14440b as info_src takes time too.
info_src does not make much sense without ABI check.
Monty
MDEV-25292 Atomic CREATE OR REPLACE TABLE

Atomic CREATE OR REPLACE allows to keep an old table intact if the
command fails or during the crash. That is done by renaming the
original table to temporary name, as a backup and restoring it if the
CREATE fails. When the command is complete and logged the backup
table is deleted.

Atomic replace algorithm

  Two DDL chains are used for CREATE OR REPLACE:
  ddl_log_state_create (C) and ddl_log_state_rm (D).

  1. (C) Log rename of ORIG to TMP table (Rename TMP to original).
  2. Rename orignal to TMP.
  3. (C) Log CREATE_TABLE_ACTION of ORIG (drops ORIG);
  4. Do everything with ORIG (like insert data)
  5. (D) Log drop of TMP
  6. Write query to binlog (this marks (C) to be closed in
    case of failure)
  7. Execute drop of TMP through (D)
  8. Close (C) and (D)

  If there is a failure before 6) we revert the changes in (C)
  Chain (D) is only executed if 6) succeded (C is closed on
  crash recovery).

Foreign key errors will be found at the 1) stage.

Additional notes

  - CREATE TABLE without REPLACE and temporary tables is not affected
    by this commit.
    set @@drop_before_create_or_replace=1 can be used to
    get old behaviour where existing tables are dropped
    in CREATE OR REPLACE.

  - CREATE TABLE is reverted if binlogging the query fails.

  - Engines having HTON_EXPENSIVE_RENAME flag set are not affected by
    this commit. Conflicting tables marked with this flag will be
    deleted with CREATE OR REPLACE.

  - Replication execution is not affected by this commit.
    - Replication will first drop the conflicting table and then
      creating the new one.

  - CREATE TABLE .. SELECT XID usage is fixed and now there is no need
    to log DROP TABLE via DDL_CREATE_TABLE_PHASE_LOG (see comments in
    do_postlock()). XID is now correctly updated so it disables
    DDL_LOG_DROP_TABLE_ACTION. Note that binary log is flushed at the
    final stage when the table is ready. So if we have XID in the
    binary log we don't need to drop the table.

  - Three variations of CREATE OR REPLACE handled:

    1. CREATE OR REPLACE TABLE t1 (..);
    2. CREATE OR REPLACE TABLE t1 LIKE t2;
    3. CREATE OR REPLACE TABLE t1 SELECT ..;

  - Test case uses 6 combinations for engines (aria, aria_notrans,
    myisam, ib, lock_tables, expensive_rename) and 2 combinations for
    binlog types (row, stmt). Combinations help to check differences
    between the results. Error failures are tested for the above three
    variations.

  - expensive_rename tests CREATE OR REPLACE without atomic
    replace. The effect should be the same as with the old behaviour
    before this commit.

  - Triggers mechanism is unaffected by this change. This is tested in
    create_replace.test.

  - LOCK TABLES is affected. Lock restoration must be done after new
    table is created or TMP is renamed back to ORIG

  - Moved ddl_log_complete() from send_eof() to finalize_ddl(). This
    checkpoint was not executed before for normal CREATE TABLE but is
    executed now.

  - CREATE TABLE will now rollback also if writing to the binary
    logging failed. See rpl_gtid_strict.test

backup ddl log changes

- In case of a successfull CREATE OR REPLACE we only log
  the CREATE event, not the DROP TABLE event of the old table.

ddl_log.cc changes

  ddl_log_execute_action() now properly return error conditions.
  ddl_log_disable_entry() added to allow one to disable one entry.
  The entry on disk is still reserved until ddl_log_complete() is
  executed.

On XID usage

  Like with all other atomic DDL operations XID is used to avoid
  inconsistency between master and slave in the case of a crash after
  binary log is written and before ddl_log_state_create is closed. On
  recovery XIDs are taken from binary log and corresponding DDL log
  events get disabled.  That is done by
  ddl_log_close_binlogged_events().

On linking two chains together

  Chains are executed in the ascending order of entry_pos of execute
  entries. But entry_pos assignment order is undefined: it may assign
  bigger number for the first chain and then smaller number for the
  second chain. So the execution order in that case will be reverse:
  second chain will be executed first.

  To avoid that we link one chain to another. While the base chain
  (ddl_log_state_create) is active the secondary chain
  (ddl_log_state_rm) is not executed. That is: only one chain can be
  executed in two linked chains.

  The interface ddl_log_link_chains() was defined in "MDEV-22166
  ddl_log_write_execute_entry() extension".

Atomic info parameters in HA_CREATE_INFO

  Many functions in CREATE TABLE pass the same parameters. These
  parameters are part of table creation info and should be in
  HA_CREATE_INFO (or whatever). Passing parameters via single
  structure is much easier for adding new data and
  refactoring.

InnoDB changes
  Added ha_innobase::can_be_renamed_to_backup() to check if
  a table with foreign keys can be renamed.

Aria changes:
- Fixed issue in Aria engine with CREATE + locked tables
  that data was not properly commited in some cases in
  case of crashes.

Other changes:
- Removed some auto variables in log.cc for better code readability.
- Fixed old bug that CREATE ... SELECT would not be able to auto repair
  a table that is part of the SELECT.
- Marked MyISAM that it does not support ROLLBACK (not required but
  done for better consistency with other engines).

Known issues:
- InnoDB tables with foreign key definitions are not fully supported
  with atomic create and replace:
  - ha_innobase::can_be_renamed_to_backup() can detect some cases
    where InnoDB does not support renaming table with foreign key
    constraints.  In this case MariaDB will drop the old table before
    creating the new one.
    The detected cases are:
    - The new and old table is using the same foreign key constraint
      name.
    - The old table has self referencing constraints.
  - If the old and new table uses the same name for a constraint the
    create of the new table will fail. The orignal table will be
    restored in this case.
  - The above issues will be fixed in a future commit.
- CREATE OR REPLACE TEMPORARY table is not full atomic. Any conflicting
  table will always be dropped before creating a new one. (Old behaviour).

Bug fixes related to this MDEV:

MDEV-36435 Assertion failure in finalize_locked_tables()
MDEV-36439 Assertion `thd_arg->lex->sql_command != SQLCOM_CREATE_SEQUENCE...
MDEV-36498 Failed CoR in non-atomic mode no longer generates DROP in RBR...
MDEV-36508 Temporary files #sql-create-....frm occasionally stay after
          crash recovery
MDEV-38479 Crash in CREATE OR REPLACE SEQUENCE when new sequence cannot
          be created
MDEV-36497 Assertion failure after atomic CoR with Aria under lock in
          transactional context
MDEV-36501 EITS data is lost after failed attempt to CREATE OR REPLACE
          table
MDEV-36493 Atomic CREATE OR REPLACE ... SELECT blocks InnoDB purge
MDEV-39367 MSAN/valgrind errors in temp_file_size_cb_func,
          main.tmp_space_usage fails
MDEV-39446 Atomic CREATE OR REPLACE fails if a table cannot be decrypted

InnoDB related changes:
- ha_innodb::rename_table() does not handle foreign key constraint
  when renaming an normal table to internal tempory tables. This
  causes problems for CREATE OR REPLACE as the old constraints causes
  failure when creating a new table with the same constraints.
  This is fixed inside InnoDB by not threating tempfiles (#sql-create-..),
  created as part of CREATE OR REPLACE, as temporary files.
- In ha_innobase::delete_table(), ignore checking of constraints when
  dropping a #sql-create temporary table.
- In tablename_to_filename() and filename_to_tablename(), don't do
  filename conversion for internal temporary tables (#sql-...)

Other things:
- maria_create_trn_for_mysql() does not register a new transaction
  handler for commits. This was needed to ensure create or replace
  will not end with an active transaction.
- We do not get anymore warnings about "Engine not supporting atomic
  create" when doing a legal CREATE OR REPLACE on a table with
  foreign key constraints.
- Updated VIDEX engine flags to disable CREATE SEQUENCE.

Reverted commits:
MDEV-36685 "CREATE-SELECT may lose in binlog side-effects of
stored-routine" as it did not take into account that it safe to clear
binlogs if the created table is non transactional and there are no
other non transactional tables used.
- This was done because it caused extra logging when it is not needed
  (not using any non transactional tables) and it also did not solve
  side effects when using statement based loggging.

Other things:
- EITS data is preserved if create or replace fails if
  drop_before_create_or_replace=OFF. If ON, then create or replace
  will drop EITS before the drop of the original table (as before).
- Using CREATE OR REPLACE on a encrypted table that the user cannot
  decrypt will fail instead of replacing the encrypted table.
  The encrypted table will unchanged.
forkfun
Merge branch '11.4' into '11.8'
Sergei Golubchik
fix sporadic galera test failures

* query @@datadir before audit is enabled, not directly before reading
  the log. just in case cat gets the log before SELECT is flushed.
* wait for a table to be dropped
Daniel Black
MDEV-35738 mariadb build -fsanitize=pointer-compare

invalid pointer pairs are when the length/memory of one string are
intermixed with another.

For comp_err, the end null pointer was compared to soffset within
my_strtoll10. As we didn't need the end position a NULL arg option
was compatble.

For uca-dump, Address Sanitizer raises invalid pointer pair because
argv options (opt) don't have an obvious correlation of having a start
at opt_X.length, even though the lstrncmp makes this true.

The DBUG_ASSERTS of strmov (added MDEV-11752) where incompatible with
pointer-compare. Replaced strmov with static inline version in
m_string.h using memmove that allows overlaps, and being inline allows
the uneeded parts of the implementation to be optimized away.
Daniel Black
MDEV-40488 disable connect.odbc_sqlite3 test on ASAN

Leak appears in libltdl which is opened by the unixodbc
driver. There doesn't appear to any mishandling at the
ODBC level of the connect storage engine.
Sergei Golubchik
MDEV-36147 MariaDB cannot open page-compressed InnoDB tables at startup if innodb_compression_algorithm other than zlib is specified

When innodb_compression_algorithm is set to a non-zlib algorithm (e.g.
lz4) and the provider plugin is loaded from mysql.plugin rather than
command line, InnoDB failed to start because it checked for the provider
at plugin initialization time, before plugin_load() reads mysql.plugin.

Fix:
* InnoDB returns HA_ERR_RETRY_INIT when the compression provider is
  missing.
* In sql_plugin.cc, the retry loop is changed to not reap until
  mysql.plugin has been loaded

Assisted-By: Claude:claude-4.6-sonnet
Aleksey Midenkov
MDEV-39063 Server crashes at Item_func_lastval and Item_func_setval with CTE alias

Pure aliases are not handled properly by Item_func_lastval::val_int()
and Item_func_setval::val_int().

This is followup fix for MDEV-33985 where it missed similar cases for
LASTVAL() and SETVAL().

add_table_to_list() does not create MDL request for pure aliases,
i.e. when there is no table_list->db set or TL_OPTION_ALIAS was
set. When the expression is not inside CTE the case with empty db is
handled by:

  else if (!lex->with_cte_resolution && lex->copy_db_to(&db))
    DBUG_RETURN(0);

So, table_list gets current database name and the query is failed with
ER_NO_SUCH_TABLE error.

The fix adds the case of is_pure_alias() check for val_int() methods
and fails it with ER_NOT_SEQUENCE2 error.

Note: semantics for TL_OPTION_ALIAS cannot be based on empty db, only
parser can set TL_OPTION_ALIAS as resolve_references_to_cte() relies
on TL_OPTION_ALIAS after copy_db_to().
drrtuy
fix: extra try-catch during plugin init.
Sergei Golubchik
MDEV-37840 Server crashes when executing FLUSH PRIVILEGES after starting with skip-grant-tables and creating mysql.host table

hash_filo's mutex was lazily initialized in clear(), but grant_reload()
locks acl_cache->lock without first calling clear(). With
--skip-grant-tables, acl_cache is created but clear() is never called
(acl_reload() skips it on error), leaving the mutex uninitialized.

Fix: initialize the mutex eagerly in the hash_filo constructor and
remove the now-redundant init flag.

Assisted-By: Claude:claude-4.6-sonnet
forkfun
Merge branch '13.0' into 'main'
Oleg Smirnov
MDEV-39491 Parallel Query: InnoDB clustered-index partitioning for parallel scan

Implement the InnoDB side of the handler parallel-scan API: partition the
clustered index into disjoint key-range chunks and serve them to the SQL layer
as pull-based scan jobs. Each chunk is read through the normal
row_search_mvcc() path, so MVCC visibility, AHI and the prefetch cache keep
working unchanged. InnoDB spawns no threads of its own.

- Parallel_coordinator (row0pcoord.{h,cc}): adapted from MySQL's
  Parallel_reader, reduced to partitioning and job distribution. Walks the
  index top-down into Exec_ctx chunks bounded by clustered-key tuples.
- ha_innobase: implements parallel_{init,end}_coordinator,
  parallel_get_worker_context, parallel_{init,end}_worker and
  parallel_get_next_row.
- Chunk-boundary clamp: row_prebuilt_t::m_pscan_end_tuple (NULL = unbounded)
  makes row_search_mvcc() stop before prefetching past the chunk's exclusive
  upper bound, treated as end-of-range so the next chunk is pulled.
- btr_pcur_open_on_user_rec(page_cur_t) overload to anchor chunk boundaries.
- Build: row0pcoord.cc added to CMakeLists.txt and auto_event_names[].
ParadoxV5
Test for MDEV-39788

MDEV-39788 found that the recent refactor on the `main` (now 12.3)
branch missed the (inconsistent) detail that, unlike `relay-log.info`,
`master.info`’s line count _includes_ the line-count line itself.

This commit extends and simplifies the test
`rpl.rpl_read_new_relay_log_info` to `main.rpl_new_info` so it
* Checks this detail to remind future changes of this type of mistake.
* Covers `@@master_info` as well.
* Covers the refactor’s buggy format as a downgrade/revert test.

While here, this commit also includes a new-format version
of MDEV-38020’s test to double as the value read check.

Reviewed-by: Brandon Nesterenko <[email protected]>
forkfun
MDEV-39522 Query with UNION fails in Oracle sql_mode with ER_BAD_FIELD_ERROR/ER_UNKNOWN_TABLE

MDEV-37325 unconditionally wrapped union subqueries in derived tables,
breaking name resolution of outer-scope/correlated references inside
the unions (producing ER_UNKNOWN_TABLE errors).

Delay the wrap until a following operator has a different linkage.
In Oracle mode all set operators share one priority and bind
left-to-right, so wrapping the accumulated prefix on each operator
change enforces it. This also corrects the row multiplicity of mixed
set operations toward left-to-right order.

create_priority_nest(): when the nest covers the whole prefix, point
the wrapper's first_nested at itself. Cut the prefix with
cut_next() before wrapping and re-register it on the outer unit;

Aleksey Midenkov:

In Oracle mode optimize_bag_operation() returns early, so union_distinct is
never recomputed and the stale value reaches execution.
Register the wrapper first, then run fix_distinct(): reset_distinct() now
scans the new outer chain (the wrapper alone) and correctly leaves
union_distinct NULL; a following DISTINCT operand re-establishes it.
Sergei Golubchik
MDEV-40571 insufficient validation of frm data when opening a table

numerous checks that the frm is valid, no OOB reads,
values make sense (number of keyparts not less than number of keys,
no keys means no keyparts, number of long unique fields is not larger than
number of fields, fields values in the record don't overlap and don't
go over record ends, and so on). most asserts were changed to if()'s
ParadoxV5
MDEV-39788: Remove added line in `master.info` format

The line-count lines in `master.info` and `relay-log.info`
have been inconsistent (off by one) since their introduction.
MDEV-37530 “fixed” this with its common code merger by chance,
changing `master.info` to use `relay-log.info`’s line-count definition.
This change, howëver, affected backward compatibility,
as `master.info` now expects an ignored MySQL-only line
where the first `key=value` option, `master_use_gtid`, is.

Since this legacy text-based format has limitations that make
it due for replacement, only code reüsablility is valuable,
and its consistency does not outweigh its compatibility.
Therefore, this commit solves this problem without reverting code by:
* Changing the writing code to be compatible with both interpretations
  (albeit inconsistent with the reading code)
* Adding a shim entry to `master.info`’s list
  to emulate prior versions’ reading behaviour
  * Although this solution can only restore upgrade compatibility with
    versions 10.0+, versions before MariaDB 10 have long been EOL.

While here, this commit also fixes code and
comments that contradict the actual effect.

[P.S.] The test for this regression is pushed to 10.11 in PR #5147.

Reviewed-by: Brandon Nesterenko <[email protected]>
Daniel Black
MDEV-40414 Server crash in Charset::charset upon JSON operations

JSON_EQUALS didn't check that ::val_str() of its arguments
where not-null before attempting to test their equality.

JSON_OVERLAPS also ensure that ::val_json() isn't null.
Sergei Golubchik
cleanup: get_item_copy<>(item)

make get_item_copy<T>(item) return T* not a generic Item*
helps to avoid casts when a copy needs to be fixed before returning.
Sergei Golubchik
MDEV-26910 mysqld_multi starts same instance multiple times with the risk to crash database

a group name may be present in a file more than once.
use hash to deduplicate.
Sergei Golubchik
MDEV-40571 insufficient validation of frm data when opening a table

numerous checks that the frm is valid, no OOB reads,
values make sense (number of keyparts not less than number of keys,
no keys means no keyparts, number of long unique fields is not larger than
number of fields, fields values in the record don't overlap and don't
go over record ends, and so on). most asserts were changed to if()'s
Sergei Golubchik
MDEV-25813 ASAN errors in err_conv / field_unpack upon multi-UPDATE causing ER_DUP_ENTRY

InnoDB always frees allocated in record[0] blobs even
when reading into record[1].
Let's read into record[0] for consistency.
Daniel Black
Remove no_valgrind_without_big.inc from *C* include dir

The identical file is in mysql-test/include where it is used.
Sergei Golubchik
MDEV-40571 insufficient validation of frm data when opening a table

numerous checks that the frm is valid, no OOB reads,
values make sense (number of keyparts not less than number of keys,
no keys means no keyparts, number of long unique fields is not larger than
number of fields, fields values in the record don't overlap and don't
go over record ends, and so on). most asserts were changed to if()'s
drrtuy
fix: MDEV-40452 ASAN builds complain about enourmous stack consumption b/c of the reference pointer used for stack usage calculations.
Aleksey Midenkov
MDEV-38854 Assertion table->vers_write fails upon ODKU into table with versioned column

In MDEV-25644 vers_check_update() sets vers_write to false in case it
returns false. It is ok for UPDATE but is not correct for ODKU is bulk
insert requires vers_write on next tuple.

The fix return vers_write value back when vers_check_update() and
related vers_insert_history_row() are done in ODKU.
drrtuy
fix: clean DeltaAppender after at rollback or disconnect.
Alexey (Holyfoot) Botchkov
MDEV-39654 No warning after CAST( AS xmltype).

Warning added.
Mohammad Tafzeel Shams
MDEV-39800: Assertion `!(mode & 2048U) || (mode & 512U) || is_supremum' failed

ISSUE:

Lock bypassing optimization allows an X-lock request to skip
waiting locks when the requesting transaction already holds an
S-lock on the same record. This optimization is designed for
regular B-tree record locks that use heap-number-based conflict
detection.

Spatial index predicate locks use different semantics. They
perform MBR (Minimum Bounding Rectangle) overlap checks for
conflict detection and must not participate in bypass
optimization.

The assertion failure occurred because predicate insert
intention locks is not considered inside lock bypass code,
which assumed all insert intention locks must be either gap
locks or on the supremum record (MDEV-34877).

ut_ad(!(insert_intention) || (gap) || is_supremum)

Additionally, predicate locks could incorrectly enable
bypass_mode because the existing checks did not explicitly
exclude LOCK_PREDICATE locks.

FIX:

- lock_t::is_predicate(): Add a helper to identify spatial
  index locks.

- lock_t::can_be_bypassed(): Return false for predicate
  locks.

- lock_rec_has_to_wait_in_queue(): Update the assertion to
  allow predicate insert intention locks. Also add a
  !is_predicate() check to bypass_mode calculation.

- lock_rec_queue_validate_bypass(): Add an early return to
  skip bypass validation for predicate locks.
drrtuy
chore: renaming, extra docs and bump DuckDB to gamma.
Sergei Golubchik
MDEV-40571 insufficient validation of frm data when opening a table

numerous checks that the frm is valid, no OOB reads,
values make sense (number of keyparts not less than number of keys,
no keys means no keyparts, number of long unique fields is not larger than
number of fields, fields values in the record don't overlap and don't
go over record ends, and so on). most asserts were changed to if()'s
forkfun
Merge branch '11.8' into '12.3'
Sergei Golubchik
MDEV-31024 Server crash / ASAN use-after-poison in Binary_string::free_buffer / Item_func_sformat::~Item_func_sformat

re-allocate Item_func_sformat::val_arg in shallow_copy()
to keep it in the same memroot as the item.
Alexey (Holyfoot) Botchkov
MDEV-39683 Numeric aggregates should end up with an error for xmltype.

Appropriate xxx_fix_length_and_dec() added to the Type_handler_xmltype.
forkfun
Merge branch '12.3' into '13.0'
drrtuy
chore: DuckDB build.sh exposes build with ASAN flag.
forkfun
Merge branch '10.11' into '11.4'
Daniel Black
MDEV-39113 MSAN/ADDR addr2line stack resolver detrimental

MSAN/ASAN test environment, the addr2line was so high in memory
utilization that it was the pick of the OOM killer to resolve the OOM
situation. Once this occurred there wasn't a saved core or gdb backtrace
of the core to resolve the issue.

To resolve this, make stack-trace default to 0 (off) for the addr2line
base implementation under memory sanitizer and address sanitizer.

MariaDB-backup also forces the enabling of stack-trace. Disabling this
unconditionally reduces the risk of a user operational impact if a
lengthy stack trace starting in a mariadb-backup critical locked period.

The mysqld--help test now excludes the stack-trace as its result is
environment dependant. The "Defaults to..." output for suppressed
variables, currently only stack-trace, is excluded.

Since thread-stack is an excluded varable, the ubsan/asan exclusions
from commits dfa6fba9595a and dfa6fba9595a aren't required.
Mohammad Tafzeel Shams
MDEV-40504: Fix memory leak in mariabackup incremental copy

ibx_copy_incremental_over_full() : Replace die() with
proper error handling to avoid memory leaks when RocksDB
backup directory operations fail.