Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Sergei Golubchik
test fixes
Sergei Golubchik
MDEV-39292 fix incorrect merge
Sergei Golubchik
MDEV-40059 too long character_set_collations crash
Sergei Golubchik
MDEV-39292 fix incorrect merge
Monty
MDEV-25292 Atomic CREATE OR REPLACE TABLE

Atomic CREATE OR REPLACE allows to keep an old table intact if the
command fails or during the crash. That is done by renaming the
original table to temporary name, as a backup and restoring it if the
CREATE fails. When the command is complete and logged the backup
table is deleted.

Atomic replace algorithm

  Two DDL chains are used for CREATE OR REPLACE:
  ddl_log_state_create (C) and ddl_log_state_rm (D).

  1. (C) Log rename of ORIG to TMP table (Rename TMP to original).
  2. Rename orignal to TMP.
  3. (C) Log CREATE_TABLE_ACTION of ORIG (drops ORIG);
  4. Do everything with ORIG (like insert data)
  5. (D) Log drop of TMP
  6. Write query to binlog (this marks (C) to be closed in
    case of failure)
  7. Execute drop of TMP through (D)
  8. Close (C) and (D)

  If there is a failure before 6) we revert the changes in (C)
  Chain (D) is only executed if 6) succeded (C is closed on
  crash recovery).

Foreign key errors will be found at the 1) stage.

Additional notes

  - CREATE TABLE without REPLACE and temporary tables is not affected
    by this commit.
    set @@drop_before_create_or_replace=1 can be used to
    get old behaviour where existing tables are dropped
    in CREATE OR REPLACE.

  - CREATE TABLE is reverted if binlogging the query fails.

  - Engines having HTON_EXPENSIVE_RENAME flag set are not affected by
    this commit. Conflicting tables marked with this flag will be
    deleted with CREATE OR REPLACE.

  - Replication execution is not affected by this commit.
    - Replication will first drop the conflicting table and then
      creating the new one.

  - CREATE TABLE .. SELECT XID usage is fixed and now there is no need
    to log DROP TABLE via DDL_CREATE_TABLE_PHASE_LOG (see comments in
    do_postlock()). XID is now correctly updated so it disables
    DDL_LOG_DROP_TABLE_ACTION. Note that binary log is flushed at the
    final stage when the table is ready. So if we have XID in the
    binary log we don't need to drop the table.

  - Three variations of CREATE OR REPLACE handled:

    1. CREATE OR REPLACE TABLE t1 (..);
    2. CREATE OR REPLACE TABLE t1 LIKE t2;
    3. CREATE OR REPLACE TABLE t1 SELECT ..;

  - Test case uses 6 combinations for engines (aria, aria_notrans,
    myisam, ib, lock_tables, expensive_rename) and 2 combinations for
    binlog types (row, stmt). Combinations help to check differences
    between the results. Error failures are tested for the above three
    variations.

  - expensive_rename tests CREATE OR REPLACE without atomic
    replace. The effect should be the same as with the old behaviour
    before this commit.

  - Triggers mechanism is unaffected by this change. This is tested in
    create_replace.test.

  - LOCK TABLES is affected. Lock restoration must be done after new
    table is created or TMP is renamed back to ORIG

  - Moved ddl_log_complete() from send_eof() to finalize_ddl(). This
    checkpoint was not executed before for normal CREATE TABLE but is
    executed now.

  - CREATE TABLE will now rollback also if writing to the binary
    logging failed. See rpl_gtid_strict.test

backup ddl log changes

- In case of a successfull CREATE OR REPLACE we only log
  the CREATE event, not the DROP TABLE event of the old table.

ddl_log.cc changes

  ddl_log_execute_action() now properly return error conditions.
  ddl_log_disable_entry() added to allow one to disable one entry.
  The entry on disk is still reserved until ddl_log_complete() is
  executed.

On XID usage

  Like with all other atomic DDL operations XID is used to avoid
  inconsistency between master and slave in the case of a crash after
  binary log is written and before ddl_log_state_create is closed. On
  recovery XIDs are taken from binary log and corresponding DDL log
  events get disabled.  That is done by
  ddl_log_close_binlogged_events().

On linking two chains together

  Chains are executed in the ascending order of entry_pos of execute
  entries. But entry_pos assignment order is undefined: it may assign
  bigger number for the first chain and then smaller number for the
  second chain. So the execution order in that case will be reverse:
  second chain will be executed first.

  To avoid that we link one chain to another. While the base chain
  (ddl_log_state_create) is active the secondary chain
  (ddl_log_state_rm) is not executed. That is: only one chain can be
  executed in two linked chains.

  The interface ddl_log_link_chains() was defined in "MDEV-22166
  ddl_log_write_execute_entry() extension".

Atomic info parameters in HA_CREATE_INFO

  Many functions in CREATE TABLE pass the same parameters. These
  parameters are part of table creation info and should be in
  HA_CREATE_INFO (or whatever). Passing parameters via single
  structure is much easier for adding new data and
  refactoring.

InnoDB changes
  Added ha_innobase::can_be_renamed_to_backup() to check if
  a table with foreign keys can be renamed.

Aria changes:
- Fixed issue in Aria engine with CREATE + locked tables
  that data was not properly commited in some cases in
  case of crashes.

Other changes:
- Removed some auto variables in log.cc for better code readability.
- Fixed old bug that CREATE ... SELECT would not be able to auto repair
  a table that is part of the SELECT.
- Marked MyISAM that it does not support ROLLBACK (not required but
  done for better consistency with other engines).

Known issues:
- InnoDB tables with foreign key definitions are not fully supported
  with atomic create and replace:
  - ha_innobase::can_be_renamed_to_backup() can detect some cases
    where InnoDB does not support renaming table with foreign key
    constraints.  In this case MariaDB will drop the old table before
    creating the new one.
    The detected cases are:
    - The new and old table is using the same foreign key constraint
      name.
    - The old table has self referencing constraints.
  - If the old and new table uses the same name for a constraint the
    create of the new table will fail. The orignal table will be
    restored in this case.
  - The above issues will be fixed in a future commit.
- CREATE OR REPLACE TEMPORARY table is not full atomic. Any conflicting
  table will always be dropped before creating a new one. (Old behaviour).

Bug fixes related to this MDEV:

MDEV-36435 Assertion failure in finalize_locked_tables()
MDEV-36439 Assertion `thd_arg->lex->sql_command != SQLCOM_CREATE_SEQUENCE...
MDEV-36498 Failed CoR in non-atomic mode no longer generates DROP in RBR...
MDEV-36508 Temporary files #sql-create-....frm occasionally stay after
          crash recovery
MDEV-38479 Crash in CREATE OR REPLACE SEQUENCE when new sequence cannot
          be created
MDEV-36497 Assertion failure after atomic CoR with Aria under lock in
          transactional context
MDEV-36501 EITS data is lost after failed attempt to CREATE OR REPLACE
          table
MDEV-36493 Atomic CREATE OR REPLACE ... SELECT blocks InnoDB purge
MDEV-39367 MSAN/valgrind errors in temp_file_size_cb_func,
          main.tmp_space_usage fails
MDEV-39446 Atomic CREATE OR REPLACE fails if a table cannot be decrypted

InnoDB related changes:
- ha_innodb::rename_table() does not handle foreign key constraint
  when renaming an normal table to internal tempory tables. This
  causes problems for CREATE OR REPLACE as the old constraints causes
  failure when creating a new table with the same constraints.
  This is fixed inside InnoDB by not threating tempfiles (#sql-create-..),
  created as part of CREATE OR REPLACE, as temporary files.
- In ha_innobase::delete_table(), ignore checking of constraints when
  dropping a #sql-create temporary table.
- In tablename_to_filename() and filename_to_tablename(), don't do
  filename conversion for internal temporary tables (#sql-...)

Other things:
- maria_create_trn_for_mysql() does not register a new transaction
  handler for commits. This was needed to ensure create or replace
  will not end with an active transaction.
- We do not get anymore warnings about "Engine not supporting atomic
  create" when doing a legal CREATE OR REPLACE on a table with
  foreign key constraints.
- Updated VIDEX engine flags to disable CREATE SEQUENCE.

Reverted commits:
MDEV-36685 "CREATE-SELECT may lose in binlog side-effects of
stored-routine" as it did not take into account that it safe to clear
binlogs if the created table is non transactional and there are no
other non transactional tables used.
- This was done because it caused extra logging when it is not needed
  (not using any non transactional tables) and it also did not solve
  side effects when using statement based loggging.

Other things:
- EITS data is preserved if create or replace fails if
  drop_before_create_or_replace=OFF. If ON, then create or replace
  will drop EITS before the drop of the original table (as before).
- Using CREATE OR REPLACE on a encrypted table that the user cannot
  decrypt will fail instead of replacing the encrypted table.
  The encrypted table will unchanged.
Sergei Golubchik
MDEV-39292 fix incorrect merge
Sergei Golubchik
MDEV-40059 too long character_set_collations crash
Sergei Golubchik
test at the end
Sergei Golubchik
update CODING_STANDARDS.md for agent era

* moved human-oriented hopefully eventually consistent codiing style
  document to https://mariadb.org/about/coding-style/
* rewrote the document in an agent friendly way: main rule first,
  don't repeat rules that the agent would follow by default anyway,
  highlight differences with the defaults, don't overdo explaining.
* asked claude to look through sql/ and mysys/ extract common patterns and
  add them here as rules
Georg Richter
Fix CONC-825: Fix unaligned pointer cast in int2store macro

Replaced the unsafe `uint16*` pointer cast in the x86/Win32 path
of `int2store` with a portable byte-shifting implementation.

The previous typed cast violated strict alignment rules and caused
modern optimizing compilers (e.g., GCC -O2) to emit aligned vector
instructions (like `movaps`) on unaligned packet buffer offsets,
leading to runtime crashes
Oleg Smirnov
MDEV-39491 Parallel Query: InnoDB clustered-index partitioning for parallel scan

Implement the InnoDB side of the handler parallel-scan API: partition the
clustered index into disjoint key-range chunks and serve them to the SQL layer
as pull-based scan jobs. Each chunk is read through the normal
row_search_mvcc() path, so MVCC visibility, AHI and the prefetch cache keep
working unchanged. InnoDB spawns no threads of its own.

- Parallel_coordinator (row0pcoord.{h,cc}): adapted from MySQL's
  Parallel_reader, reduced to partitioning and job distribution. Walks the
  index top-down into Exec_ctx chunks bounded by clustered-key tuples.
- ha_innobase: implements parallel_{init,end}_coordinator,
  parallel_get_worker_context, parallel_{init,end}_worker and
  parallel_get_next_row.
- Chunk-boundary clamp: row_prebuilt_t::m_pscan_end_tuple (NULL = unbounded)
  makes row_search_mvcc() stop before prefetching past the chunk's exclusive
  upper bound, treated as end-of-range so the next chunk is pulled.
- btr_pcur_open_on_user_rec(page_cur_t) overload to anchor chunk boundaries.
- Build: row0pcoord.cc added to CMakeLists.txt and auto_event_names[].
Sergei Petrunia
Code cleanup (3).
Oleg Smirnov
MDEV-39845 Introduce parallel scan API

Add a handler-level interface that engines can implement to support
parallel table scans, with a serial-scan fallback when unsupported..

- HA_CAN_PARALLEL_SCAN table flag and handler::is_parallel_scan_supported()
- Coordinator-side methods (parallel_init_coordinator / parallel_end_coordinator,
  parallel_get_worker_context) driven by the master thread
- Worker-side methods (parallel_init_worker / parallel_get_next_row /
  parallel_end_worker) driven by child threads, with the
  ha_parallel_get_next_row() wrapper doing the usual bookkeeping
- Parallel_worker_ctx, an opaque per-worker context subclass
Sergei Petrunia
Code cleanup in JSON array-of-object reading, add unit tests.
Sergei Golubchik
MDEV-40058 cached_sha2_password crashes on zero-length password

valid encrypted password cannot have zero length and must end with '\0'
Oleg Smirnov
Do a full table scan in chunks in one thread (pseudo-parallel)
Sergei Golubchik
update CODING_STANDARDS.md for agent era

* moved human-oriented hopefully eventually consistent codiing style
  document to https://mariadb.org/about/coding-style/
* rewrote the document in an agent friendly way: main rule first,
  don't repeat rules that the agent would follow by default anyway,
  highlight differences with the defaults, don't overdo explaining.
* asked claude to look through sql/ and mysys/ extract common patterns and
  add them here as rules
Georg Richter
caching_sha2 fixes:

- check that filesize doesn't exceed INT_MAX
- Fixed parameter order typo in fread
- terminate public key by trailing zero
Sergei Golubchik
update CODING_STANDARDS.md for agent era

* moved human-oriented hopefully eventually consistent codiing style
  document to https://mariadb.org/about/coding-style/
* rewrote the document in an agent friendly way: main rule first,
  don't repeat rules that the agent would follow by default anyway,
  highlight differences with the defaults, don't overdo explaining.
* asked claude to look through sql/ and mysys/ extract common patterns and
  add them here as rules
Sergei Golubchik
MDEV-32362 post-review changes

* don't handle generated columns specially in --xml mode
* test header must be MDEV summary
* add test version markers
* add a test for generated/invisible mix
bsrikanth-mariadb
Replace List<> with Mem_root_dynamic_array for double and uint types
Sergei Petrunia
More code cleanups.
Fariha Shaikh
MDEV-32362 Handle generated columns in mariadb-dump INSERT statements

Currently, generated column names and values are present in INSERT
statements created by the dump tool. While this doesn't break
restorability, it is unnecessary as values for generated columns are
calculated based on other column values and need not be explicitly
inserted.

Modify mariadb-dump to handle generated columns in INSERT statements:

- In default mode, generated column values are replaced with DEFAULT,
  preserving the user's chosen insert style.
- When --complete-insert is enabled (either explicitly or due to
  INVISIBLE columns), generated columns are omitted entirely from
  the column list and values.

The server computes the generated column values automatically in
both cases.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
Sergei Golubchik
MDEV-39292 fix incorrect merge
Daniel Black
MDEV-39585 mariadb bootstrap fails to perform plugin deinitalization

mariadbd under --bootstrap failed to preform plugin deinitialization.

The sleep(2);exit is removed and replaced to a goto termination label
to perform the same shutdown procedure of the server after all the
connection closing.

To prevent a compile error about char *user being uninitialized
this sql_print_information(ER_DEFAULT(ER_NORMAL_SHUTDOWN)) is moved to
its own block. The memory free did need to occur in the bootstrap mode
too to avoid memory leak errors.

wait_for_signal_thread_to_end(), was previously in close_connections()
however its required too for --bootstrap.
Sergei Golubchik
MDEV-40059 too long character_set_collations crash
Sergei Golubchik
update CODING_STANDARDS.md for agent era

* moved human-oriented hopefully eventually consistent codiing style
  document to https://mariadb.org/about/coding-style/
* rewrote the document in an agent friendly way: main rule first,
  don't repeat rules that the agent would follow by default anyway,
  highlight differences with the defaults, don't overdo explaining.
* asked claude to look through sql/ and mysys/ extract common patterns and
  add them here as rules
Sergei Golubchik
MDEV-40058 cached_sha2_password crashes on zero-length password

valid encrypted password cannot have zero length and must end with '\0'
Sergei Golubchik
MDEV-40059 too long character_set_collations crash
Sergei Golubchik
MDEV-40058 cached_sha2_password crashes on zero-length password

valid encrypted password cannot have zero length and must end with '\0'
Daniel Black
MDEV-39585: Support SHUTDOWN command on Windows in bootstrap

Don't perform mysqld_win_initiate_shutdown under --bootstrap when
triggered by SHUTDOWN. With this we don't perform any service
interactions.

Then the shutdown can proceeded without then hard process termination
in mysqld_win_initiate_shutdown. This previously occurred because the
handle_connections_win() was never called in --bootstrap and therefore
startup_complete() was false.

Thanks Vladislav Vaintroub for investigation and providing
implementation guidance.
Oleg Smirnov
MDEV-39845 Introduce parallel scan API

Add a handler-level interface that engines can implement to support
parallel table scans, with a serial-scan fallback when unsupported..

- HA_CAN_PARALLEL_SCAN table flag and handler::is_parallel_scan_supported()
- Coordinator-side methods (parallel_init_coordinator / parallel_end_coordinator,
  parallel_get_worker_context) driven by the master thread
- Worker-side methods (parallel_init_worker / parallel_get_next_row /
  parallel_end_worker) driven by child threads, with the
  ha_parallel_get_next_row() wrapper doing the usual bookkeeping
- Parallel_worker_ctx, an opaque per-worker context subclass
Oleg Smirnov
MDEV-39491 Parallel Query: InnoDB clustered-index partitioning for parallel scan

Implement the InnoDB side of the handler parallel-scan API: partition the
clustered index into disjoint key-range chunks and serve them to the SQL layer
as pull-based scan jobs. Each chunk is read through the normal
row_search_mvcc() path, so MVCC visibility, AHI and the prefetch cache keep
working unchanged. InnoDB spawns no threads of its own.

- Parallel_coordinator (row0pcoord.{h,cc}): adapted from MySQL's
  Parallel_reader, reduced to partitioning and job distribution. Walks the
  index top-down into Exec_ctx chunks bounded by clustered-key tuples.
- ha_innobase: implements parallel_{init,end}_coordinator,
  parallel_get_worker_context, parallel_{init,end}_worker and
  parallel_get_next_row.
- Chunk-boundary clamp: row_prebuilt_t::m_pscan_end_tuple (NULL = unbounded)
  makes row_search_mvcc() stop before prefetching past the chunk's exclusive
  upper bound, treated as end-of-range so the next chunk is pulled.
- btr_pcur_open_on_user_rec(page_cur_t) overload to anchor chunk boundaries.
- Build: row0pcoord.cc added to CMakeLists.txt and auto_event_names[].
Sergei Golubchik
MDEV-40058 cached_sha2_password crashes on zero-length password

valid encrypted password cannot have zero length and must end with '\0'
Vladislav Vaintroub
MDEV-40028 Assertion `rights.allow_bits() == merged->cols' failed

Fixed mismatched memroot in new operator, in apply_deny_column.

MDEV-40014 Crash in SHOW GRANTS FOR user after REVOKE

Fixed premature my_hash_reset in update_role_columns, should not be
done if only DENYs are present

Also fixed unintended GRANT on column privileges when DENY should be
applied.
Vladislav Vaintroub
MDEV-40028 Assertion `rights.allow_bits() == merged->cols' failed

Fixed mismatched memroot in new operator, in apply_deny_column.

MDEV-40014 Crash in SHOW GRANTS FOR user after REVOKE

Fixed premature my_hash_reset in update_role_columns, should not be
done if only DENYs are present

Also fixed unintended GRANT on column privileges when DENY should be
applied.