Console View

Categories: connectors experimental galera main

Legend:

Passed

Failed

Warnings

Failed Again

Running

Exception

Offline

No data

connectors

experimental

galera

main

fe0c06a232c5...

fe0c06a232c54f1054778dd59126abc2be451f44

Alessandro Vetere

iminelink@gmail.com

MDEV-40408 btr_page_reorganize_low() uses the buffer pool just to obtain a scratch block

Add buf_pool.scratch_buf, a pool of page frames that the page
reorganization operations use instead of taking a block from the global
buffer pool.

buf_pool_t::scratch_buffer: A singly linked list of chunks of
buf_tmp_buffer_t slots. A thread holds at most one slot at a time, and
it holds a page latch while doing so, which is why reserve() must not
wait for the buffer pool or for I/O. When every slot is in use,
reserve() appends a chunk that holds twice the slots of the last one, up
to a limit on the size of one chunk. Only grow() waits, on the mutex
that serializes it and on the allocator. A chunk is never moved or freed
before close(), so a thread can keep using the slot that it reserved
while another thread appends a chunk. Debug builds start with a single
slot, so that a second concurrent page reorganization exercises grow().

buf_pool_t::scratch_buffer::shrink(): Free the page frames of the slots
that are not in use. The slots and the chunks are kept, because a slot
costs 32 bytes while a page frame costs srv_page_size. A frame survives
the first pass, because that pass only clears the used flag. The master
thread calls this often while the server is idle and rarely while it is
active, and buf_pool_t::garbage_collect() calls it under memory
pressure, where it ignores the used flag, because releasing these frames
is much cheaper than shrinking the buffer pool.

buf_pool_t::io_buf_t::acquire(): Factor out the scan for an unreserved
slot, which io_buf_t::reserve() ran twice and the scratch buffer reuses.

btr_page_reorganize_low(), page_zip_reorganize(): Obtain the scratch
page frame from buf_pool.scratch_buf. This removes the buf_pool.mutex
acquisition and the free block wait that buf_block_alloc() could
perform while at least a page X-latch was being held.
btr_page_reorganize_low() also used to leak the block on its error
paths; a single exit now releases the slot.

page_copy_rec_list_end_no_locks(), lock_move_reorganize_page(): Take
the source page frame instead of a source buf_block_t, because the
source is no longer a buffer pool block. In
page_copy_rec_list_end_no_locks() the source page is page_align(rec) at
every call site, so only the record is passed.

buf_tmp_buffer_t::acquire(), buf_tmp_buffer_t::release(): Use acquire
and release memory ordering, so that the page frame pointer of a slot
is published to the next thread that reserves the slot. acquire() also
reads the flag before the exchange, so that a scan across an array of
slots does not write to the slots that it finds reserved. release()
also marks the page frame undefined for Valgrind and MSan, because the
frame stays allocated for the next reserver and no deallocation marks
it.

fb834b1f1ee7...

fb834b1f1ee7850347b57704e17c8e15f392c820

forkfun

alice.sherepa@gmail.com

MDEV-39566 fix status_by_thread crash on live thread-count change

PFS_table_context snapshots the live thread/user/host/account
count at scan start and again on restore (filesort's second
rnd_init). If the count changed between the two, m_map_size
mismatched and the server aborted.

Skip the wasted re-sample on restore, bound each table's scan by
the frozen snapshot instead of the container's live count.

eeb8cb8587d9...

eeb8cb8587d94956d814e2e0465fc27b8f8280c0

Abdelrahman Hedia

bodyhadia44@gmail.com

MDEV-29803: Change mariadb-binlog --gtid-strict-mode default to OFF

The --gtid-strict-mode option in mariadb-binlog was introduced in MDEV-4989
with a default of ON. This causes mariadb-binlog to refuse to display
events when it encounters out-of-order GTIDs, which commonly happens
when replaying a remote binlog into a server and then reading back the
resulting local binlog files.

This is overly restrictive for a diagnostic/display tool. While the
server's gtid_strict_mode makes sense as a safety mechanism, applying
the same strict validation by default in the client tool prevents users
from even inspecting problematic binlog files.

Change the default to OFF so that mariadb-binlog processes binlog files
without erroring on out-of-order GTIDs by default. Users who want strict
validation can still explicitly pass --gtid-strict-mode.

Added regression test binlog.mdev_29803 that verifies:
- Default (OFF): reading binlog files with replayed events succeeds
- Explicit --gtid-strict-mode: still produces the expected error

e61cabc0a2ae...

e61cabc0a2ae1a58c401fc24bb9f25caab6dc9b5

Monty

monty@mariadb.org

MDEV-25292 Atomic CREATE OR REPLACE TABLE

Atomic CREATE OR REPLACE allows to keep an old table intact if the
command fails or during the crash. That is done by renaming the
original table to temporary name, as a backup and restoring it if the
CREATE fails. When the command is complete and logged the backup
table is deleted.

Atomic replace algorithm

Two DDL chains are used for CREATE OR REPLACE:
ddl_log_state_create (C) and ddl_log_state_rm (D).

1. (C) Log rename of ORIG to TMP table (Rename TMP to original).
2. Rename orignal to TMP.
3. (C) Log CREATE_TABLE_ACTION of ORIG (drops ORIG);
4. Do everything with ORIG (like insert data)
5. (D) Log drop of TMP
6. Write query to binlog (this marks (C) to be closed in
case of failure)
7. Execute drop of TMP through (D)
8. Close (C) and (D)

If there is a failure before 6) we revert the changes in (C)
Chain (D) is only executed if 6) succeded (C is closed on
crash recovery).

Foreign key errors will be found at the 1) stage.

Additional notes

- CREATE TABLE without REPLACE and temporary tables is not affected
by this commit.
set @@drop_before_create_or_replace=1 can be used to
get old behaviour where existing tables are dropped
in CREATE OR REPLACE.

- CREATE TABLE is reverted if binlogging the query fails.

- Engines having HTON_EXPENSIVE_RENAME flag set are not affected by
this commit. Conflicting tables marked with this flag will be
deleted with CREATE OR REPLACE.

- Replication execution is not affected by this commit.
- Replication will first drop the conflicting table and then
creating the new one.

- CREATE TABLE .. SELECT XID usage is fixed and now there is no need
to log DROP TABLE via DDL_CREATE_TABLE_PHASE_LOG (see comments in
do_postlock()). XID is now correctly updated so it disables
DDL_LOG_DROP_TABLE_ACTION. Note that binary log is flushed at the
final stage when the table is ready. So if we have XID in the
binary log we don't need to drop the table.

- Three variations of CREATE OR REPLACE handled:

1. CREATE OR REPLACE TABLE t1 (..);
2. CREATE OR REPLACE TABLE t1 LIKE t2;
3. CREATE OR REPLACE TABLE t1 SELECT ..;

- Test case uses 6 combinations for engines (aria, aria_notrans,
myisam, ib, lock_tables, expensive_rename) and 2 combinations for
binlog types (row, stmt). Combinations help to check differences
between the results. Error failures are tested for the above three
variations.

- expensive_rename tests CREATE OR REPLACE without atomic
replace. The effect should be the same as with the old behaviour
before this commit.

- Triggers mechanism is unaffected by this change. This is tested in
create_replace.test.

- LOCK TABLES is affected. Lock restoration must be done after new
table is created or TMP is renamed back to ORIG

- Moved ddl_log_complete() from send_eof() to finalize_ddl(). This
checkpoint was not executed before for normal CREATE TABLE but is
executed now.

- CREATE TABLE will now rollback also if writing to the binary
logging failed. See rpl_gtid_strict.test

backup ddl log changes

- In case of a successfull CREATE OR REPLACE we only log
the CREATE event, not the DROP TABLE event of the old table.

ddl_log.cc changes

ddl_log_execute_action() now properly return error conditions.
ddl_log_disable_entry() added to allow one to disable one entry.
The entry on disk is still reserved until ddl_log_complete() is
executed.

On XID usage

Like with all other atomic DDL operations XID is used to avoid
inconsistency between master and slave in the case of a crash after
binary log is written and before ddl_log_state_create is closed. On
recovery XIDs are taken from binary log and corresponding DDL log
events get disabled. That is done by
ddl_log_close_binlogged_events().

On linking two chains together

Chains are executed in the ascending order of entry_pos of execute
entries. But entry_pos assignment order is undefined: it may assign
bigger number for the first chain and then smaller number for the
second chain. So the execution order in that case will be reverse:
second chain will be executed first.

To avoid that we link one chain to another. While the base chain
(ddl_log_state_create) is active the secondary chain
(ddl_log_state_rm) is not executed. That is: only one chain can be
executed in two linked chains.

The interface ddl_log_link_chains() was defined in "MDEV-22166
ddl_log_write_execute_entry() extension".

Atomic info parameters in HA_CREATE_INFO

Many functions in CREATE TABLE pass the same parameters. These
parameters are part of table creation info and should be in
HA_CREATE_INFO (or whatever). Passing parameters via single
structure is much easier for adding new data and
refactoring.

InnoDB changes
Added ha_innobase::can_be_renamed_to_backup() to check if
a table with foreign keys can be renamed.

Aria changes:
- Fixed issue in Aria engine with CREATE + locked tables
that data was not properly commited in some cases in
case of crashes.

Other changes:
- Removed some auto variables in log.cc for better code readability.
- Fixed old bug that CREATE ... SELECT would not be able to auto repair
a table that is part of the SELECT.
- Marked MyISAM that it does not support ROLLBACK (not required but
done for better consistency with other engines).

Known issues:
- InnoDB tables with foreign key definitions are not fully supported
with atomic create and replace:
- ha_innobase::can_be_renamed_to_backup() can detect some cases
where InnoDB does not support renaming table with foreign key
constraints. In this case MariaDB will drop the old table before
creating the new one.
The detected cases are:
- The new and old table is using the same foreign key constraint
name.
- The old table has self referencing constraints.
- If the old and new table uses the same name for a constraint the
create of the new table will fail. The orignal table will be
restored in this case.
- The above issues will be fixed in a future commit.
- CREATE OR REPLACE TEMPORARY table is not full atomic. Any conflicting
table will always be dropped before creating a new one. (Old behaviour).

Bug fixes related to this MDEV:

MDEV-36435 Assertion failure in finalize_locked_tables()
MDEV-36439 Assertion `thd_arg->lex->sql_command != SQLCOM_CREATE_SEQUENCE...
MDEV-36498 Failed CoR in non-atomic mode no longer generates DROP in RBR...
MDEV-36508 Temporary files #sql-create-....frm occasionally stay after
crash recovery
MDEV-38479 Crash in CREATE OR REPLACE SEQUENCE when new sequence cannot
be created
MDEV-36497 Assertion failure after atomic CoR with Aria under lock in
transactional context
MDEV-36501 EITS data is lost after failed attempt to CREATE OR REPLACE
table
MDEV-36493 Atomic CREATE OR REPLACE ... SELECT blocks InnoDB purge
MDEV-39367 MSAN/valgrind errors in temp_file_size_cb_func,
main.tmp_space_usage fails
MDEV-39446 Atomic CREATE OR REPLACE fails if a table cannot be decrypted

InnoDB related changes:
- ha_innodb::rename_table() does not handle foreign key constraint
when renaming an normal table to internal tempory tables. This
causes problems for CREATE OR REPLACE as the old constraints causes
failure when creating a new table with the same constraints.
This is fixed inside InnoDB by not threating tempfiles (#sql-create-..),
created as part of CREATE OR REPLACE, as temporary files.
- In ha_innobase::delete_table(), ignore checking of constraints when
dropping a #sql-create temporary table.
- In tablename_to_filename() and filename_to_tablename(), don't do
filename conversion for internal temporary tables (#sql-...)

Other things:
- maria_create_trn_for_mysql() does not register a new transaction
handler for commits. This was needed to ensure create or replace
will not end with an active transaction.
- We do not get anymore warnings about "Engine not supporting atomic
create" when doing a legal CREATE OR REPLACE on a table with
foreign key constraints.
- Updated VIDEX engine flags to disable CREATE SEQUENCE.

Reverted commits:
MDEV-36685 "CREATE-SELECT may lose in binlog side-effects of
stored-routine" as it did not take into account that it safe to clear
binlogs if the created table is non transactional and there are no
other non transactional tables used.
- This was done because it caused extra logging when it is not needed
(not using any non transactional tables) and it also did not solve
side effects when using statement based loggging.

Other things:
- EITS data is preserved if create or replace fails if
drop_before_create_or_replace=OFF. If ON, then create or replace
will drop EITS before the drop of the original table (as before).
- Using CREATE OR REPLACE on a encrypted table that the user cannot
decrypt will fail instead of replacing the encrypted table.
The encrypted table will unchanged.

e582992ea4b9...

e582992ea4b935b079496a99d1574cc11882beab

Sergei Petrunia

sergey@mariadb.com

Review input 2: re-word comments

e2323e583eae...

e2323e583eae8553f343fbab14cde5788bd8b3bc

bsrikanth-mariadb

srikanth.bondalapati@mariadb.com

MDEV-39226: Push whole multi-table update/delete down into engines

Give storage engines a way to take over an entire multi-table
UPDATE/DELETE, the way they can already take over a SELECT. Without it the
join, the row matching and every modification run in the SQL layer even
when an engine could do the whole statement itself in one step; a
single-table UPDATE/DELETE already avoids this via
direct_update_rows()/direct_delete_rows(), but a multi-table statement has
no primary handler object to drive that path.

This adds a generic, engine-agnostic pushdown interface: the SQL layer
offers the statement to the engine, and if the engine accepts it, it
performs the whole thing and reports only the row counts.

- Split select_handler into a pushdown_handler base with select_handler
(result set) and a new multi_upddel_handler (runs a whole UPDATE/DELETE,
reports row counts, reported as PUSHED UPDATE/PUSHED DELETE); add
handlerton::create_multi_upddel, looked up in Sql_cmd_dml::execute_inner().
- multi_update/multi_delete gain direct_update_delete_done(), which records
the engine's counts so send_eof() binlogs and replies without the
SQL-layer loop; it forces statement-format binlogging so the change still
replicates under binlog_format=ROW, and errors out instead of silently
dropping counts for an unsupported result object.
- FederatedX implements the interface as the reference engine used to test
correctness: it prints the statement back and runs it remotely, passes
the engine's error code/SQLSTATE through, reads the matched count from the
remote info string, executes IGNORE locally, and only pushes down when all
tables share one remote server (same as SELECT/derived/unit pushdown).

Test: federated.federatedx_pushdown_upd_del.

e029d7897326...

e029d7897326a9d1f7fbfeb6ef269ea84101efd9

sjaakola

seppo.jaakola@mariadb.com

MDEV-38869 sequence conflicts with streaming replication

Sequence access conflicts with streaming replication could cause the
server to hang, as shown in MDEV-38869.

A sequence table is written from SEQUENCE::next_value() while
SEQUENCE::mutex is held. For a streaming transaction the row write in
handler::ha_write_row() would then replicate a fragment and block waiting
for certification and commit order, while an applier may be waiting for
the same mutex in SEQUENCE::set_value(). Neither side can proceed, the
node deadlocks and the BF abort of the local transaction can never be
delivered.

This commit avoids the deadlock by skipping the streaming replication
step for sequence table rows. The row is already in the write set and is
replicated with the following fragment, or at commit.

Only that one step is skipped. The skip is passed down as a parameter to
wsrep_after_row() and wsrep_after_row_internal() rather than by not
calling them at all, so the row is still counted against
wsrep_max_ws_rows and wsrep_check_pk() still runs. A transaction using
sequences heavily therefore cannot silently exceed the configured write
set row limit.

The commit has also a new mtr test for three sequence/SR conflict
scenarios: galera.galera_sequences_bf_kill_sr

- a streaming transaction and an applier competing for SEQUENCE::mutex,
where both are expected to commit

- the same, but with the applier also BF aborting the local transaction
over a gap lock. A streaming transaction cannot be replayed, so it is
rolled back and the client gets ER_LOCK_DEADLOCK. The applier is held
at the abort_trx_end sync point until the abort has been issued, so
that the local transaction cannot finish its fragment first

- twelve row inserts on both nodes with wsrep_trx_fragment_unit=rows, so
that each node reserves several sequence cache ranges and the sequence
table writes land inside fragments carrying several rows. The values
the two nodes hand out must not overlap

bc44b0fc9864...

bc44b0fc986487fab0bc4152adaece8fa45ba0d8

forkfun

alice.sherepa@gmail.com

MDEV-40303 SIGSEGV in Item_field::type_handler() on PS re-execution

Re-executing a prepared statement that reads a derived-table column
holding a scalar UNION subquery crashed the server: between executions
st_select_lex_unit::cleanup() frees union_result and sets 'cleaned' but
leaves 'prepared' set, so on re-fix set_row() walks a stale item_list
whose Item_field::field are NULL.

Fix: prepare() re-prepares a cleaned unit instead of treating it as
still prepared.

b89e5c654688...

b89e5c654688c4c9c8db12d98d6d75a097699eb2

Sergei Petrunia

sergey@mariadb.com

Review input 3: don't check for is_supported_update_delete()

* we only get into create_federatedx_multi_upddel_handler for UPDATE/DELETE

* Use of is_supported_update_delete() implies that this checks
for UPDATEs/DELETEs of which only some are supported.
The function actually returned TRUE for any UPDATE/DELETE.

b80038d965fb...

b80038d965fb652926ee424985c6d699d5d5e8de

Marko Mäkelä

marko.makela@mariadb.com

squash! 208ce0ae56e1ddd990555028b82aabd96909484f

InnoDB_backup::commit(): Enqueue the remaining log.

InnoDB_backup::checkpoint_complete(): If backup is running and
commit() has not been called, add each completed innodb_archive_log=ON
file to InnoDB_backup::queue. Else, skip or delete, as appropriate.

b722cc53ed4d...

b722cc53ed4da0b58fa1e4cf88deabf166abb1da

Val Doroshchuk

valbok@gmail.com

Rename duckdb file name to allow to use duckdb as schema

If duckdb is used as schema, DuckDB requires to use it in queries explicitly since the name conflicts with the catalog.

This fixes
Ambiguous reference to catalog or schema "duckdb" - use a fully qualified path like '.duckdb'

aba1a16b6dd6...

aba1a16b6dd68415b0091493314105edef29cd1e

Thirunarayanan Balathandayuthapani

thiru@mariadb.com

MDEV-39091 BACKUP SERVER of ENGINE=RocksDB to the local file system

Wire the RocksDB (MyRocks) engine into BACKUP SERVER TO and
BACKUP SERVER WITH (streaming). RocksDB SST files are immutable,
so a consistent point-in-time snapshot is obtained cheaply
with a rocksdb::Checkpoint. The checkpoint is frozen at
BACKUP_PHASE_NO_COMMIT because the checkpoint files are
immutable, the actual copy is deferred to BACKUP_PHASE_FINISH,
after the backup MDL has been released, and the server-side
checkpoint is removed at end of FINISH. No user-level lock
is needed: MDL_BACKUP_START already serializes concurrent BACKUP SERVER.

Files are copied into the target's "#rocksdb" subdirectory.
The default rocksdb_datadir ("./#rocksdb") makes restore
transparent: on restore the files land at <datadir>/#rocksdb, so
pointing a server at the extracted backup just works, with
no copy-back and no RocksDB prepare step.

storage/rocksdb/rdb_backup_server.{h,cc}:
New RocksDB_backup context and the three handlerton hooks.
The checkpoint file list is drained across N CONCURRENT step threads
via a lock-free atomic cursor; each file is copied with
copy_entire_file() (directory target) or backup_stream_*().
Since SSTs are immutable, the sendfile(2) fast path
(backup_stream_append_async) is used for the stream target.

rdb_create_checkpoint(): Factor the checkpoint creation
rdb_remove_checkpoint(): Remove the checkpoint creation logic.
rdb_get_datadir(): Get the rocksdb data directory.

a6c90a944d26...

a6c90a944d26ee3980b79e564f1f7d0cfeef05d3

Monty

monty@mariadb.org

MDEV-40765 Assertion `"unexpected references" == 0' failed upon failing CREATE OR REPLACE

There was a few reasons for this failure:
- There was no timeout for the mdl_lock in the ddl log in
execute_rename_table. This is needed to protect against a concurrent
purge in InnoDB. If the purge would be running, the ddl would call
rename_table without a ddl protection the assert could happen.
- InnoDB locked the original table name in purge, not the temporary
name used to cache the table in case of rollback. The effect is
that if a purge happens during rename the assert could happen.
- The table was a partitioned and the detection of partitioned tables
did not take into account temporary #sql-create- prefixed tables.

Other things:
- The code in execute_rename_table() assumes that InnoDB never holds
a MDL lock for more more than a few milliseconds. I am using a timeout
for 10 seconds, just in case. In practice this timeout should be very
short.
- Added a 10 second timeout for the MDL lock in execute_drop_table()
- Removed mysql_mutex_unlock(&LOCK_gdl) / mysql_mutex_lock(&LOCK_gdl)
around calls to binlog as these are unsafe. The binlog code uses
global variables that needs protection from other caller.
- Fixed a few compiler warnings related to tmp_file_prefix.

a1784a4af078...

a1784a4af078da5d47f3150216e9abb68d66955b

Aleksey Midenkov

midenok@gmail.com

MDEV-20865 Fix tests and system scripts to foreign key requirements

Storing foreign key metadata in TABLE_SHARE makes opening a foreign table
preopen its referenced tables, so a referenced table must be created
before the foreign one and dropped after it. Rearrange the order of
CREATE/DROP commands in tests and system scripts accordingly.

Affected: the help tables in mariadb_system_tables{,_fix}.sql (with
system_mysql_db_fix* results), fetch_first, union, insert_notembedded,
instant_alter_index_rename and opt_context_store_ddls; some also index a
referenced column or add a missing referenced table, and
opt_context_store_ddls re-records the now-shown MyISAM foreign key.

a10a74d4864b...

a10a74d4864be7031b2505663de2b6ae25e7e182

Aleksey Midenkov

midenok@gmail.com

MDEV-34392 Check foreign key column nullability in the server layer

Making a foreign key column NOT NULL must be refused when a referential
action can still write NULL into it -- ON UPDATE/DELETE SET NULL, or ON
UPDATE CASCADE from a NULLable parent. Until now this was enforced through a
per-column nullability bitmap in FK_info (fields_nullable) that the storage
engine allocated and populated, and that the server read back while checking
an ALTER.

Since MDEV-20865 the server keeps the foreign key definitions in TABLE_SHARE,
so this round trip through the engine is redundant. Perform the check
directly in mysql_prepare_alter_table() from the stored referential actions
and the old/new column definitions, and drop the FK_info bitmap along with
its assign_nullable()/set_nullable()/is_nullable() helpers. Incompatible
changes are still rejected with ER_FK_COLUMN_NOT_NULL.

Fixes foreign_null test.

a106f2d6079b...

a106f2d6079b29816aa53b8d4dc6ec31572e65c8

Aleksey Midenkov

midenok@gmail.com

MDEV-20865 Store foreign key info in TABLE_SHARE

1. Access foreign keys via TABLE_SHARE::foreign_keys and
TABLE_SHARE::referenced_keys;

foreign_keys and referenced_keys are lists in TABLE_SHARE.

2. Remove handler FK interface:

- get_foreign_key_list()
- get_parent_foreign_key_list()
- referenced_by_foreign_key()

3. Invalidate referenced shares on:

- RENAME TABLE
- DROP TABLE
- RENAME COLUMN
- ADD FOREIGN KEY

When foreign table is created or altered by the above operations
all referenced shares are closed. This blocks the operation while
any referenced shares are used (when at least one its TABLE
instance is locked).

4. Update referenced shares on:

- CREATE TABLE

On CREATE TABLE add items to referenced_keys of referenced
shares. States of referenced shares are restored in case of errors.

5. Invalidate foreign shares on:

- RENAME TABLE
- RENAME COLUMN

The above-mentioned blocking takes effect.

6. Check foreign/referenced shares consistency on:

- CHECK TABLE

7. Temporary change until MDEV-21051:

InnoDB fill foreign key info at handler open().

FOREIGN_KEY_INFO is refactored to FK_info holding Lex_cstring.

On first TABLE open FK_info is loaded from storage engine into
TABLE_SHARE. All referenced shares (if any exists) are closed. This
leads to blocking of first time foreign table open while referenced
tables are used.

MDEV-21311 Converge Foreign_key and supplemental generated Key together

mysql_prepare_create_table() does data validation and such utilities
as automatic name generation. But it does that only for indexes and
ignores Foreign_key objects. Now as Foreign_key data needs to be
stored in FRM files as well this processing must be done for it like
for any other Key objects.

Replace Key::FOREIGN_KEY type with Key::foreign flag of type
Key::MULTIPLE and Key::generated set to true. Construct one object
with Key::foreign == true instead of two objects of type
Key::FOREIGN_KEY and Key::MULTIPLE.

MDEV-21051 datadict refactorings

- Move read_extra2() to datadict.cc
- Refactored extra2_fields to Extra2_info
- build_frm_image() readability

MDEV-21051 build_table_shadow_filename() refactoring

mysql_prepare_alter_table() leaks fixes

MDEV-21051 amend system tables locking restriction

Table mysql.help_relation has foreign key to mysql.help_keyword. On
bootstrap when help_relation is opened, it preopens help_keyword for
READ and fails in lock_tables_check().

If system table is opened for write then fk references are opened for
write.

Related to: Bug#25422, WL#3984
Tests: main.lock

MDEV-21051 Store and read foreign key info into/from FRM files

1. Introduce Foreign_key_io class which creates/parses binary stream
containing foreign key structures. Referenced tables store there only
hints about foreign tables (their db and name), they restore full info
from the corresponding tables.

Foreign_key_io is stored under new EXTRA2_FOREIGN_KEY_INFO field in
extra2 section of FRM file.

2. Modify mysql_prepare_create_table() to generate names for foreign
keys. Until InnoDB storage of foreign keys is removed, FK names must
be unique across the database: the FK name must be based on table
name.

3. Keep stored data in sync on DDL changes. Referenced tables update
their foreign hints after following operations on foreign tables:

- RENAME TABLE
- DROP TABLE
- CREATE TABLE
- ADD FOREIGN KEY
- DROP FOREIGN KEY

Foreign tables update their foreign info after following operations on
referenced tables:

- RENAME TABLE
- RENAME COLUMN

4. To achieve 3. there must be ability to rewrite extra2 section of
FRM file without full reparse. FRM binary is built from primary
structures like HA_CREATE_INFO and cannot be built from TABLE_SHARE.

Use shadow write and rename like fast_alter_partition_table()
does. Shadow FRM is new FRM file that replaces the old one.

CREATE TABLE workflow:

1. Foreign_key is constructed in parser, placed into
alter_info->key_list;

2. mysql_prepare_create_table() translates them to FK_info, assigns
foreign_id if needed;

3. build_frm_image() writes two FK_info lists into FRM's extra2
section, for referenced keys it stores only table names (hints);

4. init_from_binary_frm_image() parses extra2 section and fills
foreign_keys and referenced_keys of TABLE_SHARE.

It restores referenced_keys by reading hint list of table names,
opening corresponding shares and restoring FK_info from their
foreign_keys. Hints resolution is done only when initializing
non-temporary shares. Usually temporary share has different
(temporary) name and it is impossible to resolve foreign keys by
that name (as we identify them by both foreign and referenced
table names). Another not unimportant reason is performance: this
saves spare share acquisitions.

ALTER TABLE workflow:

1. Foreign_key is constructed in parser, placed into
alter_info->key_list;

2. mysql_prepare_alter_table() prepares action lists and share list
of foreigns/references;

3. mysql_prepare_alter_table() locks list of foreigns/references by
MDL_INTENTION_EXCLUSIVE, acquires shares;

4. prepare_create_table() converts key_list into FK_list, assigns
foreign_id;

5. shadow FRM of altered table is created;

6. data is copied;

7. altered table is locked by MDL_EXCLUSIVE;

8. fk_handle_alter() processes action lists, creates FK backups,
modifies shares, writes shadow FRMs;

9. altered table is closed;

10. shadow FRMs are installed;

11. altered table is renamed, FRM backup deleted;

12. (TBD in MDEV-21053) shadow FRMs installation log closed, backups
deleted;

On FK backup system:

In case of failed DDL operation all shares that was modified must be
restored into original state. This is done by FK_ddl_backup (CREATE,
DROP), FK_rename_backup (RENAME), FK_alter_backup (ALTER).

On STL usage:

STL is used for utility not performance-critical algorithms, core
structures hold native List. A wrapper was made to convert STL
exception into bool error status or NULL value.

MDEV-20865 fk_check_consistency() in CHECK TABLE

Self-refs fix

Test table_flags fix: "debug" deviation is now gone.

FIXMEs: +16 -1

8f08b2994a30...

8f08b2994a30d1a4f64e2652b3bd05696148a7ae

Aleksey Midenkov

midenok@gmail.com

MDEV-20865 extra2_fields refactored to Extra2_info class

read_extra2() is now Extra2_info::read()
Additional assertions for checking size consistency.

Extra2_info::write() is used by further MDEV-20865 development.

7cfc549d474b...

7cfc549d474b26de4e107a84c1d4a4bcd05befe3

Andrei Elkin

andrei.elkin@pp.inet.fi

explore deeper, incl the 2nd problematic DROP.

7503495047ef...

7503495047ef192ebcfc83164496321e903173e6

Aleksey Midenkov

midenok@gmail.com

MDEV-20865 Extra2_info::write(): gate fields by length, not str

store_size() counts a field only when its length is non-zero, but write()
emitted a field whenever its str was set. A {str != NULL, length == 0} field
would then reach extra2_write_len(0) (bogus zero length / assert) and break
the write_size == store_size() invariant. Gate the writes on length so both
sides agree.

FIXME: fixup to 8f08b2994a3

731566bfab19...

731566bfab192616742897daa1083a991ce8fdb2

Monty

monty@mariadb.org

56193378fa56...

56193378fa56e33fda05512adc7a4d943d668c5e

Thirunarayanan B

thiru@mariadb.com

Merge branch 'MDEV-14992' into MDEV-39091

56064d47afb4...

56064d47afb489b2a30c09915158d3ad808f4afb

Alexander Barkov

bar@mariadb.com

MDEV-39518 Allow prepared statements in stored functions in assignment right hand

TODO: ps_in_func-stack3-pfp.test crashes with InnoDB!!!

Allowing prepared statements in stored functions when
a stored function is used in an assignment right hand.

Both DEFAULT clause of a variable initialization and
the right side of the SET statement are supported:

CREATE PROCEDURE p1()
BEGIN
-- case 1: DEFAULT clause
DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK

-- case 2: SP variable assignment statement
DECLARE spvar2 INT;
SET spvar= f1_with_ps(); -- OK
END;

- The parser now does not reject PS statements in stored functions.
PS applicability in stored functions is now detected at run time.
Note, PS statements in triggers are still prohibited by the parser.

- Functions with PS do not acquire MDL locks on tables.
They work like procedures in terms of table opening.

- Functions with PS are not replicated as a single `SELECT f1()` call.
They are replicated per-statement, like procedures.

- Only bare function calls are supported for now. Using a function in
an expression does not make it PS-safe yet yet:
SET v= f1()+0;

4ba823c552df...

4ba823c552dfc1dd5cfcff860bf07b0bd9ec53eb

Aleksey Midenkov

midenok@gmail.com

MDEV-40799 Runtime plugin/UDF load errors lost under --silent-startup

Regression from MDEV-32745 (7828fb475b0), which guarded the
plugin-load my_error() calls with opt_silent_startup. That option is
a lifetime global, set once at startup and never reset, so the guard
suppressed the SQL error for the whole server lifetime, not just
during startup. Runtime operations (INSTALL PLUGIN, CREATE FUNCTION
... SONAME) then skipped my_error(), never set the diagnostics area
and wrongly succeeded - e.g. main.ps's "call proc_1()" no longer
failed with ER_CANT_OPEN_LIBRARY.

Startup callers pass MYF(ME_ERROR_LOG); runtime callers pass MYF(0).
Gate the silencing on that flag via silent_plugin_startup() so it
applies only to the startup error-log path, and runtime errors always
reach the client.

No new test case: the runtime failure path is already covered by
existing tests (e.g. main.ps's ER_CANT_OPEN_LIBRARY check). The
regression stayed invisible only because stock MTR does not start
servers with --silent-startup. A dedicated test would have to restart
the server with --silent-startup solely to assert that a startup-only
option does not affect runtime, which adds little over the restored
invariant.

4785cef01d57...

4785cef01d57f98edd51b3a3a5f4e588189b9e20

Daniel Bartholomew

db@dbart.us

Merge branch 'bb-10.11-bumpversion' of github.com:MariaDB/server into bb-10.11-bumpversion

4618ed066934...

4618ed066934f91605307d433a05552bba8474c2

Monty

monty@mariadb.org

MDEV-40776 Atomic CREATE OR REPLACE silently breaks the foreign key

Give an error if one tries to drop a table referenced by a foreign keys
This is needed as innodb will keep the reference to the origina table
even when it is renamed to a temporary name as part of create or replace.

Other things:
- Changed the error message for ER_TRUNCATE_ILLEGAL_FK to say
"Cannot drop or truncate a table ..."

3a52eac4fee8...

3a52eac4fee8f8b57091124226ccea72c09de0e4

Daniel Bartholomew

db@dbart.us

bump the VERSION

37d51f40b808...

37d51f40b808c7ea8f576e8ebb20794fce518e73

Aleksey Midenkov

midenok@gmail.com

34b5c272e7b0...

34b5c272e7b0717c2defc1d703b8230001f03303

Andrei Elkin

andrei.elkin@pp.inet.fi

explore deeper, incl the 2nd problematic DROP.

349e9bf0feb9...

349e9bf0feb976ae8f5e5ec18f1e64ac3c090416

Andrei Elkin

andrei.elkin@pp.inet.fi

explore bb-13.2-MDEV-22992:rpl.rpl_xa

2891f618cd43...

2891f618cd43ac789f0be49ec942f79d84b71986

Thirunarayanan Balathandayuthapani

thiru@mariadb.com

213b013d7619...

213b013d76199120d243d7bb9173014b0c46f571

Georgi (Joro) Kodinov

joro@mariadb.org

MDEV-40909: main.user_limits is unstable

The cleanup of the old connection goes on in the background.
It can take longer on a busy server and this trigger the active sessions
warning in DROP user.
Stabilized the test by disabling warnings around DROP USER.

208ce0ae56e1...

208ce0ae56e1ddd990555028b82aabd96909484f

Marko Mäkelä

marko.makela@mariadb.com

squash! b625be27ca0a0b24730dd19d37d8bbf85917156c

InnoDB_backup::context: Remove the pointer indirection and do not allow
two overlapping backup operations.

InnoDB_backup::init(): Wait for a possible previous BACKUP SERVER
operation to reach the very end of InnoDB_backup::context::cleanup()
so that the context can be safely reused.

148f7e48f284...

148f7e48f28473063239fee2d8477e71e9a1bd86

Aleksey Midenkov

midenok@gmail.com

Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

10114ecb5dd1...

10114ecb5dd197317b8e2f4e12b907b74c9eb4fc

Aleksey Midenkov

midenok@gmail.com

MDEV-20865 Refactor Share_acquire::fk_error() into acquire()

Share_acquire::fk_error() inspected thd->is_error() after the acquisition to
decide whether a missing referenced table is tolerable. That breaks when an
outer Internal_error_handler consumes the error first, e.g. the
Postponed_error_handler installed by mysql_rename_tables() (MDEV-27027).

Move the decision into Share_acquire::acquire(): push a
No_such_table_error_handler for the acquisition when foreign key checks are
not enforced, so ER_NO_SUCH_TABLE is trapped above any outer handler rather
than deferred or leaked into SHOW WARNINGS. The non-tolerated error outcome
is recorded in Share_acquire::error; the four consumers check it instead of
calling fk_error().

FIXME: Squash into the main "MDEV-20865 Store foreign key info in
TABLE_SHARE" commit.

0a72b234ea59...

0a72b234ea59ba0f2a639e9a2104e65b95b853a4

Aleksey Midenkov

midenok@gmail.com

MDEV-40311 Fix mysqldump-nl leaving slave connection behind

The trailing CHANGE MASTER left master.info in the datadir, so a
later test that restarts the server auto-started the slave and broke
its check-testcase. Use RESET SLAVE ALL to drop the connection.

MTR's internal check of the test case 'sys_vars.default_master_connection_basic' failed.
This means that the test case does not preserve the state that existed
before the test case was executed. Most likely the test case did not
do a proper clean-up. It could also be caused by the previous test run
by this thread, if the server wasn't restarted.
This is the diff of the states of the servers before and after the
test case was executed:
-Slave_IO_Running No
-Slave_SQL_Running No
+Slave_IO_Running Connecting
+Slave_SQL_Running Yes
...
-Last_IO_Errno 0
-Last_IO_Error
+Last_IO_Errno 1045
+Last_IO_Error error connecting to master '[email protected]:3306' - retry-time: 60 maximum-retries: 100000 message: Access denied for user 'root'@'localhost' (using password: NO)

05feba501ae8...

05feba501ae85bc7605dd7bbe662f384ac9e7804

Alexander Barkov

bar@mariadb.com

012e9de65db2...

012e9de65db2a56b844eb24016ae114563acee28

Aleksey Midenkov

midenok@gmail.com

MDEV-20865 Reuse share in fk_handle_drop()

mysql_rm_table_no_locks() already opens the dropped table's share, so
pass it to fk_handle_drop() instead of re-acquiring it with
GTS_FK_SHALLOW_HINTS.

Skip fk_handle_drop() when the share is unreadable, letting the engine
report the error. This restores ER_GET_ERRNO for
main.partition_not_blackhole, which clear_error() used to mask by
zeroing my_errno.

New Share_acquire::inexistent_t lets DROP keep seeing ER_NO_SUCH_TABLE
independent of foreign_key_checks (INEXISTENT_ALWAYS).

Fixes main.partition_not_blackhole

FIXME: Squash into the main "MDEV-20865 Store foreign key info in
TABLE_SHARE" commit.

009e81eec550...

009e81eec5509b80ba4973e368433b36009807bf

Andrei Elkin

andrei.elkin@pp.inet.fi

explore bb-13.2-MDEV-22992:rpl.rpl_xa

0019c8dcc961...

0019c8dcc9615be1316f2af7d4f5081e4c2d8734

Aleksey Midenkov

midenok@gmail.com