Console View

Categories: connectors experimental galera main

Legend:

Passed

Failed

Warnings

Failed Again

Running

Exception

Offline

No data

connectors

experimental

galera

main

ff09eefbe4e8...

ff09eefbe4e8b57846435ee4a928f17e49f0fdfc

Georgi (Joro) Kodinov

joro@mariadb.org

MDEV-40815: resolveip is not built for the minbuild cmake target

Added the resolveip target to the minbuild target.

fe0c06a232c5...

fe0c06a232c54f1054778dd59126abc2be451f44

Alessandro Vetere

iminelink@gmail.com

MDEV-40408 btr_page_reorganize_low() uses the buffer pool just to obtain a scratch block

Add buf_pool.scratch_buf, a pool of page frames that the page
reorganization operations use instead of taking a block from the global
buffer pool.

buf_pool_t::scratch_buffer: A singly linked list of chunks of
buf_tmp_buffer_t slots. A thread holds at most one slot at a time, and
it holds a page latch while doing so, which is why reserve() must not
wait for the buffer pool or for I/O. When every slot is in use,
reserve() appends a chunk that holds twice the slots of the last one, up
to a limit on the size of one chunk. Only grow() waits, on the mutex
that serializes it and on the allocator. A chunk is never moved or freed
before close(), so a thread can keep using the slot that it reserved
while another thread appends a chunk. Debug builds start with a single
slot, so that a second concurrent page reorganization exercises grow().

buf_pool_t::scratch_buffer::shrink(): Free the page frames of the slots
that are not in use. The slots and the chunks are kept, because a slot
costs 32 bytes while a page frame costs srv_page_size. A frame survives
the first pass, because that pass only clears the used flag. The master
thread calls this often while the server is idle and rarely while it is
active, and buf_pool_t::garbage_collect() calls it under memory
pressure, where it ignores the used flag, because releasing these frames
is much cheaper than shrinking the buffer pool.

buf_pool_t::io_buf_t::acquire(): Factor out the scan for an unreserved
slot, which io_buf_t::reserve() ran twice and the scratch buffer reuses.

btr_page_reorganize_low(), page_zip_reorganize(): Obtain the scratch
page frame from buf_pool.scratch_buf. This removes the buf_pool.mutex
acquisition and the free block wait that buf_block_alloc() could
perform while at least a page X-latch was being held.
btr_page_reorganize_low() also used to leak the block on its error
paths; a single exit now releases the slot.

page_copy_rec_list_end_no_locks(), lock_move_reorganize_page(): Take
the source page frame instead of a source buf_block_t, because the
source is no longer a buffer pool block. In
page_copy_rec_list_end_no_locks() the source page is page_align(rec) at
every call site, so only the record is passed.

buf_tmp_buffer_t::acquire(), buf_tmp_buffer_t::release(): Use acquire
and release memory ordering, so that the page frame pointer of a slot
is published to the next thread that reserves the slot. acquire() also
reads the flag before the exchange, so that a scan across an array of
slots does not write to the slots that it finds reserved. release()
also marks the page frame undefined for Valgrind and MSan, because the
frame stays allocated for the next reserver and no deallocation marks
it.

fb834b1f1ee7...

fb834b1f1ee7850347b57704e17c8e15f392c820

forkfun

alice.sherepa@gmail.com

MDEV-39566 fix status_by_thread crash on live thread-count change

PFS_table_context snapshots the live thread/user/host/account
count at scan start and again on restore (filesort's second
rnd_init). If the count changed between the two, m_map_size
mismatched and the server aborted.

Skip the wasted re-sample on restore, bound each table's scan by
the frozen snapshot instead of the container's live count.

f09e3b61e3a0...

f09e3b61e3a03bc4a1773f98a901acd11740c583

Monty

monty@mariadb.org

Removed some not needed checks and add a DBUG_ASSERT() for not covered code

- In ha_partition.cc:check_parallel_search(), remove check if
item_field->field is null. This is not needed as the function is run
after fix_field() which guarnatees that the field is always set.
- Added DBUG_ASSERT(new_field) to Item_field::fix_fields() to check if a
select-list item, found by name or alias when resolving ORDER BY/GROUP
BY/HAVING, can have field == 0. This error path is not covered by any
mtr test.

e61cabc0a2ae...

e61cabc0a2ae1a58c401fc24bb9f25caab6dc9b5

Monty

monty@mariadb.org

MDEV-25292 Atomic CREATE OR REPLACE TABLE

Atomic CREATE OR REPLACE allows to keep an old table intact if the
command fails or during the crash. That is done by renaming the
original table to temporary name, as a backup and restoring it if the
CREATE fails. When the command is complete and logged the backup
table is deleted.

Atomic replace algorithm

Two DDL chains are used for CREATE OR REPLACE:
ddl_log_state_create (C) and ddl_log_state_rm (D).

1. (C) Log rename of ORIG to TMP table (Rename TMP to original).
2. Rename orignal to TMP.
3. (C) Log CREATE_TABLE_ACTION of ORIG (drops ORIG);
4. Do everything with ORIG (like insert data)
5. (D) Log drop of TMP
6. Write query to binlog (this marks (C) to be closed in
case of failure)
7. Execute drop of TMP through (D)
8. Close (C) and (D)

If there is a failure before 6) we revert the changes in (C)
Chain (D) is only executed if 6) succeded (C is closed on
crash recovery).

Foreign key errors will be found at the 1) stage.

Additional notes

- CREATE TABLE without REPLACE and temporary tables is not affected
by this commit.
set @@drop_before_create_or_replace=1 can be used to
get old behaviour where existing tables are dropped
in CREATE OR REPLACE.

- CREATE TABLE is reverted if binlogging the query fails.

- Engines having HTON_EXPENSIVE_RENAME flag set are not affected by
this commit. Conflicting tables marked with this flag will be
deleted with CREATE OR REPLACE.

- Replication execution is not affected by this commit.
- Replication will first drop the conflicting table and then
creating the new one.

- CREATE TABLE .. SELECT XID usage is fixed and now there is no need
to log DROP TABLE via DDL_CREATE_TABLE_PHASE_LOG (see comments in
do_postlock()). XID is now correctly updated so it disables
DDL_LOG_DROP_TABLE_ACTION. Note that binary log is flushed at the
final stage when the table is ready. So if we have XID in the
binary log we don't need to drop the table.

- Three variations of CREATE OR REPLACE handled:

1. CREATE OR REPLACE TABLE t1 (..);
2. CREATE OR REPLACE TABLE t1 LIKE t2;
3. CREATE OR REPLACE TABLE t1 SELECT ..;

- Test case uses 6 combinations for engines (aria, aria_notrans,
myisam, ib, lock_tables, expensive_rename) and 2 combinations for
binlog types (row, stmt). Combinations help to check differences
between the results. Error failures are tested for the above three
variations.

- expensive_rename tests CREATE OR REPLACE without atomic
replace. The effect should be the same as with the old behaviour
before this commit.

- Triggers mechanism is unaffected by this change. This is tested in
create_replace.test.

- LOCK TABLES is affected. Lock restoration must be done after new
table is created or TMP is renamed back to ORIG

- Moved ddl_log_complete() from send_eof() to finalize_ddl(). This
checkpoint was not executed before for normal CREATE TABLE but is
executed now.

- CREATE TABLE will now rollback also if writing to the binary
logging failed. See rpl_gtid_strict.test

backup ddl log changes

- In case of a successfull CREATE OR REPLACE we only log
the CREATE event, not the DROP TABLE event of the old table.

ddl_log.cc changes

ddl_log_execute_action() now properly return error conditions.
ddl_log_disable_entry() added to allow one to disable one entry.
The entry on disk is still reserved until ddl_log_complete() is
executed.

On XID usage

Like with all other atomic DDL operations XID is used to avoid
inconsistency between master and slave in the case of a crash after
binary log is written and before ddl_log_state_create is closed. On
recovery XIDs are taken from binary log and corresponding DDL log
events get disabled. That is done by
ddl_log_close_binlogged_events().

On linking two chains together

Chains are executed in the ascending order of entry_pos of execute
entries. But entry_pos assignment order is undefined: it may assign
bigger number for the first chain and then smaller number for the
second chain. So the execution order in that case will be reverse:
second chain will be executed first.

To avoid that we link one chain to another. While the base chain
(ddl_log_state_create) is active the secondary chain
(ddl_log_state_rm) is not executed. That is: only one chain can be
executed in two linked chains.

The interface ddl_log_link_chains() was defined in "MDEV-22166
ddl_log_write_execute_entry() extension".

Atomic info parameters in HA_CREATE_INFO

Many functions in CREATE TABLE pass the same parameters. These
parameters are part of table creation info and should be in
HA_CREATE_INFO (or whatever). Passing parameters via single
structure is much easier for adding new data and
refactoring.

InnoDB changes
Added ha_innobase::can_be_renamed_to_backup() to check if
a table with foreign keys can be renamed.

Aria changes:
- Fixed issue in Aria engine with CREATE + locked tables
that data was not properly commited in some cases in
case of crashes.

Other changes:
- Removed some auto variables in log.cc for better code readability.
- Fixed old bug that CREATE ... SELECT would not be able to auto repair
a table that is part of the SELECT.
- Marked MyISAM that it does not support ROLLBACK (not required but
done for better consistency with other engines).

Known issues:
- InnoDB tables with foreign key definitions are not fully supported
with atomic create and replace:
- ha_innobase::can_be_renamed_to_backup() can detect some cases
where InnoDB does not support renaming table with foreign key
constraints. In this case MariaDB will drop the old table before
creating the new one.
The detected cases are:
- The new and old table is using the same foreign key constraint
name.
- The old table has self referencing constraints.
- If the old and new table uses the same name for a constraint the
create of the new table will fail. The orignal table will be
restored in this case.
- The above issues will be fixed in a future commit.
- CREATE OR REPLACE TEMPORARY table is not full atomic. Any conflicting
table will always be dropped before creating a new one. (Old behaviour).

Bug fixes related to this MDEV:

MDEV-36435 Assertion failure in finalize_locked_tables()
MDEV-36439 Assertion `thd_arg->lex->sql_command != SQLCOM_CREATE_SEQUENCE...
MDEV-36498 Failed CoR in non-atomic mode no longer generates DROP in RBR...
MDEV-36508 Temporary files #sql-create-....frm occasionally stay after
crash recovery
MDEV-38479 Crash in CREATE OR REPLACE SEQUENCE when new sequence cannot
be created
MDEV-36497 Assertion failure after atomic CoR with Aria under lock in
transactional context
MDEV-36501 EITS data is lost after failed attempt to CREATE OR REPLACE
table
MDEV-36493 Atomic CREATE OR REPLACE ... SELECT blocks InnoDB purge
MDEV-39367 MSAN/valgrind errors in temp_file_size_cb_func,
main.tmp_space_usage fails
MDEV-39446 Atomic CREATE OR REPLACE fails if a table cannot be decrypted

InnoDB related changes:
- ha_innodb::rename_table() does not handle foreign key constraint
when renaming an normal table to internal tempory tables. This
causes problems for CREATE OR REPLACE as the old constraints causes
failure when creating a new table with the same constraints.
This is fixed inside InnoDB by not threating tempfiles (#sql-create-..),
created as part of CREATE OR REPLACE, as temporary files.
- In ha_innobase::delete_table(), ignore checking of constraints when
dropping a #sql-create temporary table.
- In tablename_to_filename() and filename_to_tablename(), don't do
filename conversion for internal temporary tables (#sql-...)

Other things:
- maria_create_trn_for_mysql() does not register a new transaction
handler for commits. This was needed to ensure create or replace
will not end with an active transaction.
- We do not get anymore warnings about "Engine not supporting atomic
create" when doing a legal CREATE OR REPLACE on a table with
foreign key constraints.
- Updated VIDEX engine flags to disable CREATE SEQUENCE.

Reverted commits:
MDEV-36685 "CREATE-SELECT may lose in binlog side-effects of
stored-routine" as it did not take into account that it safe to clear
binlogs if the created table is non transactional and there are no
other non transactional tables used.
- This was done because it caused extra logging when it is not needed
(not using any non transactional tables) and it also did not solve
side effects when using statement based loggging.

Other things:
- EITS data is preserved if create or replace fails if
drop_before_create_or_replace=OFF. If ON, then create or replace
will drop EITS before the drop of the original table (as before).
- Using CREATE OR REPLACE on a encrypted table that the user cannot
decrypt will fail instead of replacing the encrypted table.
The encrypted table will unchanged.

e582992ea4b9...

e582992ea4b935b079496a99d1574cc11882beab

Sergei Petrunia

sergey@mariadb.com

Review input 2: re-word comments

e2323e583eae...

e2323e583eae8553f343fbab14cde5788bd8b3bc

bsrikanth-mariadb

srikanth.bondalapati@mariadb.com

MDEV-39226: Push whole multi-table update/delete down into engines

Give storage engines a way to take over an entire multi-table
UPDATE/DELETE, the way they can already take over a SELECT. Without it the
join, the row matching and every modification run in the SQL layer even
when an engine could do the whole statement itself in one step; a
single-table UPDATE/DELETE already avoids this via
direct_update_rows()/direct_delete_rows(), but a multi-table statement has
no primary handler object to drive that path.

This adds a generic, engine-agnostic pushdown interface: the SQL layer
offers the statement to the engine, and if the engine accepts it, it
performs the whole thing and reports only the row counts.

- Split select_handler into a pushdown_handler base with select_handler
(result set) and a new multi_upddel_handler (runs a whole UPDATE/DELETE,
reports row counts, reported as PUSHED UPDATE/PUSHED DELETE); add
handlerton::create_multi_upddel, looked up in Sql_cmd_dml::execute_inner().
- multi_update/multi_delete gain direct_update_delete_done(), which records
the engine's counts so send_eof() binlogs and replies without the
SQL-layer loop; it forces statement-format binlogging so the change still
replicates under binlog_format=ROW, and errors out instead of silently
dropping counts for an unsupported result object.
- FederatedX implements the interface as the reference engine used to test
correctness: it prints the statement back and runs it remotely, passes
the engine's error code/SQLSTATE through, reads the matched count from the
remote info string, executes IGNORE locally, and only pushes down when all
tables share one remote server (same as SELECT/derived/unit pushdown).

Test: federated.federatedx_pushdown_upd_del.

da8bf475e0e9...

da8bf475e0e99d14240081930845d019bbd29fc6

Vladislav Vaintroub

vvaintroub@gmail.com

MDEV-22991 Support SSL over named pipes

Update libmariadb, to remove named pipe SSL check.
Update client.c to remove that check as well.

Add MTR test for named pipe + SSL:
- for Connector/C client, new test named_pipe_ssl
- for in-server clent, run mariabackup with user created as
"IDENTIFIED WITH named_pipe REQUIRE SSL"

c7eaa9192189...

c7eaa91921890d15d1fee397389d5cf65d072998

Georgi (Joro) Kodinov

joro@mariadb.org

Merge remote-tracking branch 'origin/main' into main-mdbf-1224

bc44b0fc9864...

bc44b0fc986487fab0bc4152adaece8fa45ba0d8

forkfun

alice.sherepa@gmail.com

MDEV-40303 SIGSEGV in Item_field::type_handler() on PS re-execution

Re-executing a prepared statement that reads a derived-table column
holding a scalar UNION subquery crashed the server: between executions
st_select_lex_unit::cleanup() frees union_result and sets 'cleaned' but
leaves 'prepared' set, so on re-fix set_row() walks a stale item_list
whose Item_field::field are NULL.

Fix: prepare() re-prepares a cleaned unit instead of treating it as
still prepared.

bab03b0fc44e...

bab03b0fc44e7aeac869b20b0055c6a8f2083eb6

Monty

monty@mariadb.org

Added proper cleanup of main.cte_update_delete.test

b89e5c654688...

b89e5c654688c4c9c8db12d98d6d75a097699eb2

Sergei Petrunia

sergey@mariadb.com

Review input 3: don't check for is_supported_update_delete()

* we only get into create_federatedx_multi_upddel_handler for UPDATE/DELETE

* Use of is_supported_update_delete() implies that this checks
for UPDATEs/DELETEs of which only some are supported.
The function actually returned TRUE for any UPDATE/DELETE.

b80038d965fb...

b80038d965fb652926ee424985c6d699d5d5e8de

Marko Mäkelä

marko.makela@mariadb.com

squash! 208ce0ae56e1ddd990555028b82aabd96909484f

InnoDB_backup::commit(): Enqueue the remaining log.

InnoDB_backup::checkpoint_complete(): If backup is running and
commit() has not been called, add each completed innodb_archive_log=ON
file to InnoDB_backup::queue. Else, skip or delete, as appropriate.

a6c90a944d26...

a6c90a944d26ee3980b79e564f1f7d0cfeef05d3

Monty

monty@mariadb.org

MDEV-40765 Assertion `"unexpected references" == 0' failed upon failing CREATE OR REPLACE

There was a few reasons for this failure:
- There was no timeout for the mdl_lock in the ddl log in
execute_rename_table. This is needed to protect against a concurrent
purge in InnoDB. If the purge would be running, the ddl would call
rename_table without a ddl protection the assert could happen.
- InnoDB locked the original table name in purge, not the temporary
name used to cache the table in case of rollback. The effect is
that if a purge happens during rename the assert could happen.
- The table was a partitioned and the detection of partitioned tables
did not take into account temporary #sql-create- prefixed tables.

Other things:
- The code in execute_rename_table() assumes that InnoDB never holds
a MDL lock for more more than a few milliseconds. I am using a timeout
for 10 seconds, just in case. In practice this timeout should be very
short.
- Added a 10 second timeout for the MDL lock in execute_drop_table()
- Removed mysql_mutex_unlock(&LOCK_gdl) / mysql_mutex_lock(&LOCK_gdl)
around calls to binlog as these are unsafe. The binlog code uses
global variables that needs protection from other caller.
- Fixed a few compiler warnings related to tmp_file_prefix.

a4bce4afad3b...

a4bce4afad3bc9effb0ec289809b4c44b6c2ecd8

Alexander Barkov

bar@mariadb.com

MDEV-39518 Allow prepared statements in stored functions in assignment right hand

TODO: ps_in_func-stack3-pfp.test crashes with InnoDB!!!

Allowing prepared statements in stored functions when
a stored function is used in an assignment right hand.

Both DEFAULT clause of a variable initialization and
the right side of the SET statement are supported:

CREATE PROCEDURE p1()
BEGIN
-- case 1: DEFAULT clause
DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK

-- case 2: SP variable assignment statement
DECLARE spvar2 INT;
SET spvar= f1_with_ps(); -- OK
END;

- The parser now does not reject PS statements in stored functions.
PS applicability in stored functions is now detected at run time.
Note, PS statements in triggers are still prohibited by the parser.

- Functions with PS do not acquire MDL locks on tables.
They work like procedures in terms of table opening.

- Functions with PS are not replicated as a single `SELECT f1()` call.
They are replicated per-statement, like procedures.

- Only bare function calls are supported for now. Using a function in
an expression does not make it PS-safe yet yet:
SET v= f1()+0;

a1784a4af078...

a1784a4af078da5d47f3150216e9abb68d66955b

Aleksey Midenkov

midenok@gmail.com

MDEV-20865 Fix tests and system scripts to foreign key requirements

Storing foreign key metadata in TABLE_SHARE makes opening a foreign table
preopen its referenced tables, so a referenced table must be created
before the foreign one and dropped after it. Rearrange the order of
CREATE/DROP commands in tests and system scripts accordingly.

Affected: the help tables in mariadb_system_tables{,_fix}.sql (with
system_mysql_db_fix* results), fetch_first, union, insert_notembedded,
instant_alter_index_rename and opt_context_store_ddls; some also index a
referenced column or add a missing referenced table, and
opt_context_store_ddls re-records the now-shown MyISAM foreign key.

a10a74d4864b...

a10a74d4864be7031b2505663de2b6ae25e7e182

Aleksey Midenkov

midenok@gmail.com

MDEV-34392 Check foreign key column nullability in the server layer

Making a foreign key column NOT NULL must be refused when a referential
action can still write NULL into it -- ON UPDATE/DELETE SET NULL, or ON
UPDATE CASCADE from a NULLable parent. Until now this was enforced through a
per-column nullability bitmap in FK_info (fields_nullable) that the storage
engine allocated and populated, and that the server read back while checking
an ALTER.

Since MDEV-20865 the server keeps the foreign key definitions in TABLE_SHARE,
so this round trip through the engine is redundant. Perform the check
directly in mysql_prepare_alter_table() from the stored referential actions
and the old/new column definitions, and drop the FK_info bitmap along with
its assign_nullable()/set_nullable()/is_nullable() helpers. Incompatible
changes are still rejected with ER_FK_COLUMN_NOT_NULL.

Fixes foreign_null test.

a106f2d6079b...

a106f2d6079b29816aa53b8d4dc6ec31572e65c8

Aleksey Midenkov

midenok@gmail.com

MDEV-20865 Store foreign key info in TABLE_SHARE

1. Access foreign keys via TABLE_SHARE::foreign_keys and
TABLE_SHARE::referenced_keys;

foreign_keys and referenced_keys are lists in TABLE_SHARE.

2. Remove handler FK interface:

- get_foreign_key_list()
- get_parent_foreign_key_list()
- referenced_by_foreign_key()

3. Invalidate referenced shares on:

- RENAME TABLE
- DROP TABLE
- RENAME COLUMN
- ADD FOREIGN KEY

When foreign table is created or altered by the above operations
all referenced shares are closed. This blocks the operation while
any referenced shares are used (when at least one its TABLE
instance is locked).

4. Update referenced shares on:

- CREATE TABLE

On CREATE TABLE add items to referenced_keys of referenced
shares. States of referenced shares are restored in case of errors.

5. Invalidate foreign shares on:

- RENAME TABLE
- RENAME COLUMN

The above-mentioned blocking takes effect.

6. Check foreign/referenced shares consistency on:

- CHECK TABLE

7. Temporary change until MDEV-21051:

InnoDB fill foreign key info at handler open().

FOREIGN_KEY_INFO is refactored to FK_info holding Lex_cstring.

On first TABLE open FK_info is loaded from storage engine into
TABLE_SHARE. All referenced shares (if any exists) are closed. This
leads to blocking of first time foreign table open while referenced
tables are used.

MDEV-21311 Converge Foreign_key and supplemental generated Key together

mysql_prepare_create_table() does data validation and such utilities
as automatic name generation. But it does that only for indexes and
ignores Foreign_key objects. Now as Foreign_key data needs to be
stored in FRM files as well this processing must be done for it like
for any other Key objects.

Replace Key::FOREIGN_KEY type with Key::foreign flag of type
Key::MULTIPLE and Key::generated set to true. Construct one object
with Key::foreign == true instead of two objects of type
Key::FOREIGN_KEY and Key::MULTIPLE.

MDEV-21051 datadict refactorings

- Move read_extra2() to datadict.cc
- Refactored extra2_fields to Extra2_info
- build_frm_image() readability

MDEV-21051 build_table_shadow_filename() refactoring

mysql_prepare_alter_table() leaks fixes

MDEV-21051 amend system tables locking restriction

Table mysql.help_relation has foreign key to mysql.help_keyword. On
bootstrap when help_relation is opened, it preopens help_keyword for
READ and fails in lock_tables_check().

If system table is opened for write then fk references are opened for
write.

Related to: Bug#25422, WL#3984
Tests: main.lock

MDEV-21051 Store and read foreign key info into/from FRM files

1. Introduce Foreign_key_io class which creates/parses binary stream
containing foreign key structures. Referenced tables store there only
hints about foreign tables (their db and name), they restore full info
from the corresponding tables.

Foreign_key_io is stored under new EXTRA2_FOREIGN_KEY_INFO field in
extra2 section of FRM file.

2. Modify mysql_prepare_create_table() to generate names for foreign
keys. Until InnoDB storage of foreign keys is removed, FK names must
be unique across the database: the FK name must be based on table
name.

3. Keep stored data in sync on DDL changes. Referenced tables update
their foreign hints after following operations on foreign tables:

- RENAME TABLE
- DROP TABLE
- CREATE TABLE
- ADD FOREIGN KEY
- DROP FOREIGN KEY

Foreign tables update their foreign info after following operations on
referenced tables:

- RENAME TABLE
- RENAME COLUMN

4. To achieve 3. there must be ability to rewrite extra2 section of
FRM file without full reparse. FRM binary is built from primary
structures like HA_CREATE_INFO and cannot be built from TABLE_SHARE.

Use shadow write and rename like fast_alter_partition_table()
does. Shadow FRM is new FRM file that replaces the old one.

CREATE TABLE workflow:

1. Foreign_key is constructed in parser, placed into
alter_info->key_list;

2. mysql_prepare_create_table() translates them to FK_info, assigns
foreign_id if needed;

3. build_frm_image() writes two FK_info lists into FRM's extra2
section, for referenced keys it stores only table names (hints);

4. init_from_binary_frm_image() parses extra2 section and fills
foreign_keys and referenced_keys of TABLE_SHARE.

It restores referenced_keys by reading hint list of table names,
opening corresponding shares and restoring FK_info from their
foreign_keys. Hints resolution is done only when initializing
non-temporary shares. Usually temporary share has different
(temporary) name and it is impossible to resolve foreign keys by
that name (as we identify them by both foreign and referenced
table names). Another not unimportant reason is performance: this
saves spare share acquisitions.

ALTER TABLE workflow:

1. Foreign_key is constructed in parser, placed into
alter_info->key_list;

2. mysql_prepare_alter_table() prepares action lists and share list
of foreigns/references;

3. mysql_prepare_alter_table() locks list of foreigns/references by
MDL_INTENTION_EXCLUSIVE, acquires shares;

4. prepare_create_table() converts key_list into FK_list, assigns
foreign_id;

5. shadow FRM of altered table is created;

6. data is copied;

7. altered table is locked by MDL_EXCLUSIVE;

8. fk_handle_alter() processes action lists, creates FK backups,
modifies shares, writes shadow FRMs;

9. altered table is closed;

10. shadow FRMs are installed;

11. altered table is renamed, FRM backup deleted;

12. (TBD in MDEV-21053) shadow FRMs installation log closed, backups
deleted;

On FK backup system:

In case of failed DDL operation all shares that was modified must be
restored into original state. This is done by FK_ddl_backup (CREATE,
DROP), FK_rename_backup (RENAME), FK_alter_backup (ALTER).

On STL usage:

STL is used for utility not performance-critical algorithms, core
structures hold native List. A wrapper was made to convert STL
exception into bool error status or NULL value.

MDEV-20865 fk_check_consistency() in CHECK TABLE

Self-refs fix

Test table_flags fix: "debug" deviation is now gone.

FIXMEs: +16 -1

9b4a8958024d...

9b4a8958024d205556cdabe398483c08ebb1d4ae

Sergei Petrunia

sergey@mariadb.com

Review input 1: simplify code in EXPLAIN handling

remove is_pushed_down_select_type(), keep just
get_pushed_down_select_text().

8f08b2994a30...

8f08b2994a30d1a4f64e2652b3bd05696148a7ae

Aleksey Midenkov

midenok@gmail.com

MDEV-20865 extra2_fields refactored to Extra2_info class

read_extra2() is now Extra2_info::read()
Additional assertions for checking size consistency.

Extra2_info::write() is used by further MDEV-20865 development.

89326d4d3fb0...

89326d4d3fb069de0e4f764554a454a7a2410ac7

Georgi (Joro) Kodinov

joro@mariadb.org

Merge remote-tracking branch 'origin/main' into main-mdbf-1224

7503495047ef...

7503495047ef192ebcfc83164496321e903173e6

Aleksey Midenkov

midenok@gmail.com

MDEV-20865 Extra2_info::write(): gate fields by length, not str

store_size() counts a field only when its length is non-zero, but write()
emitted a field whenever its str was set. A {str != NULL, length == 0} field
would then reach extra2_write_len(0) (bogus zero length / assert) and break
the write_size == store_size() invariant. Gate the writes on length so both
sides agree.

FIXME: fixup to 8f08b2994a3

731566bfab19...

731566bfab192616742897daa1083a991ce8fdb2

Monty

monty@mariadb.org

61209b15a5f9...

61209b15a5f91d8c1093fd1eca3dd963a17ed069

Alessandro Vetere

iminelink@gmail.com

MDEV-38056 Assertion 'bpage->state() >= buf_page_t::UNFIXED' in buf_page_get_zip()

Three callers reconstruct old versions of a clustered index record, derive
secondary index entries from them, and dereference the BLOB pointers of
externally stored columns: row_vers_impl_x_locked_low() for the implicit lock
check, row_undo_mod_sec_is_unsafe() for rollback, and row_check_index() for
CHECK TABLE ... EXTENDED. purge_sys.view is what keeps those pages allocated,
but purge_sys_t::view_guard froze it only for the duration of
trx_undo_prev_version_build(), and the dereference happens after. On
ROW_FORMAT=COMPRESSED this trips the assertion above; in a release build the
freed page is read anyway, which is silent corruption.

All three now hold purge_sys.latch across the dereference, and only where the
version has externally stored columns, since row_build() dereferences nothing
otherwise. In row_check_index() the freeze spans the whole comparison, which
fetches one field at a time and may evaluate a virtual column expression; for
the latest version it is also confined to a delete-marked record, the only
kind that need not own what it points at. Freezing that late means the view
may have advanced since the walk decided a version was reachable, so each
caller re-establishes that under the freeze; trx_undo_prev_version() states
the condition and why testing the oldest writer applied suffices.

The first two callers can treat that test as an invariant and end the walk
where it fails. row_check_index() cannot, because it decides reachability from
the lagging purge_sys.end_view on purpose, and that lag is how it finds orphan
secondary index records: where the test fails it stops and reports nothing
rather than treating the record as an orphan, and no report is lost for good,
end_view eventually reaching the same point. Its two purge_sys.is_purgeable()
tests now read the frozen view wherever a fetch follows, which makes them
atomic with the fetch they guard. purge_sys_t::view_guard and
purge_sys_t::is_purgeable() assert that the thread holds no purge_sys.latch,
because a nested shared acquisition would hang behind a waiting writer.

trx_undo_prev_version(): remove the gate that was meant to stop CHECK TABLE
... EXTENDED from fetching BLOBs it may no longer own, and
view_guard::is_extended() with it, which no guard mode could satisfy. That
decision belongs to the caller, the only one that knows whether it will
dereference anything.

row_log_table_get_pk(): document why the online ALTER path may dereference
without a freeze.

Debug-only keywords. purge_hold_cleanup parks a purge batch between its last
purged record and purge_sys_t::batch_cleanup(), the window in which a reader
that goes by purge_sys.end_view can still reach history the batch has removed;
a batch opens and closes it without ever returning to the test.
purge_no_blob_freeze sends a version that does have externally stored columns
down the path one without any takes, which is what all four call sites did
before this change, and reproduces the assertion above.
row_vers_impl_x_locked_purgeable, row_undo_mod_sec_is_unsafe_purgeable and
row_check_index_purgeable force the re-validation to fail, reaching exits that
purge_sys.view advancing mid-walk otherwise produces. The first reports no
implicit lock for a row that a live transaction still holds, so a test may
only check that nothing breaks; the other two make the server more cautious.

Tests. old_blob and old_blob_updel cover the implicit lock check,
old_blob_rollback the rollback, old_blob_check CHECK TABLE ... EXTENDED. Each
parks a walk at a dereference, makes the BLOB freeable, and asserts that the
counter of purged update records, which is what would free it, stays at zero
while parked and advances once the walk is over. old_blob_updel covers an undo
log record that stores only the 20-byte reference and needs purge_hold_cleanup
to reach it; old_blob_rollback parks at a reference that the version merely
inherited. All four fail with the original assertion under
debug_dbug=+d,purge_no_blob_freeze, and skip above a 16k page size, which
ROW_FORMAT=COMPRESSED requires. old_blob_purgeable drives the three
re-validation exits, needs no synchronisation, and runs at every page size.

564aeccaf700...

564aeccaf700c9d72d39cac286a0ab7ad6fada45

Aleksey Midenkov

midenok@gmail.com

4ba823c552df...

4ba823c552dfc1dd5cfcff860bf07b0bd9ec53eb

Aleksey Midenkov

midenok@gmail.com

MDEV-40799 Runtime plugin/UDF load errors lost under --silent-startup

Regression from MDEV-32745 (7828fb475b0), which guarded the
plugin-load my_error() calls with opt_silent_startup. That option is
a lifetime global, set once at startup and never reset, so the guard
suppressed the SQL error for the whole server lifetime, not just
during startup. Runtime operations (INSTALL PLUGIN, CREATE FUNCTION
... SONAME) then skipped my_error(), never set the diagnostics area
and wrongly succeeded - e.g. main.ps's "call proc_1()" no longer
failed with ER_CANT_OPEN_LIBRARY.

Startup callers pass MYF(ME_ERROR_LOG); runtime callers pass MYF(0).
Gate the silencing on that flag via silent_plugin_startup() so it
applies only to the startup error-log path, and runtime errors always
reach the client.

No new test case: the runtime failure path is already covered by
existing tests (e.g. main.ps's ER_CANT_OPEN_LIBRARY check). The
regression stayed invisible only because stock MTR does not start
servers with --silent-startup. A dedicated test would have to restart
the server with --silent-startup solely to assert that a startup-only
option does not affect runtime, which adds little over the restored
invariant.

4618ed066934...

4618ed066934f91605307d433a05552bba8474c2

Monty

monty@mariadb.org

MDEV-40776 Atomic CREATE OR REPLACE silently breaks the foreign key

Give an error if one tries to drop a table referenced by a foreign keys
This is needed as innodb will keep the reference to the origina table
even when it is renamed to a temporary name as part of create or replace.

Other things:
- Changed the error message for ER_TRUNCATE_ILLEGAL_FK to say
"Cannot drop or truncate a table ..."

3e3664a8a53c...

3e3664a8a53c2d3197cdd141b31794ae3cf4538f

Georgi (Joro) Kodinov

joro@mariadb.org

Days work: started doing grouping and doxygen conversion of plugin.h

37d51f40b808...

37d51f40b808c7ea8f576e8ebb20794fce518e73

Aleksey Midenkov

midenok@gmail.com

349e9bf0feb9...

349e9bf0feb976ae8f5e5ec18f1e64ac3c090416

Andrei Elkin

andrei.elkin@pp.inet.fi

explore bb-13.2-MDEV-22992:rpl.rpl_xa

208ce0ae56e1...

208ce0ae56e1ddd990555028b82aabd96909484f

Marko Mäkelä

marko.makela@mariadb.com

squash! b625be27ca0a0b24730dd19d37d8bbf85917156c

InnoDB_backup::context: Remove the pointer indirection and do not allow
two overlapping backup operations.

InnoDB_backup::init(): Wait for a possible previous BACKUP SERVER
operation to reach the very end of InnoDB_backup::context::cleanup()
so that the context can be safely reused.

111ea6c17717...

111ea6c177170df8d53efffca82c44c194eaa07c

Georgi (Joro) Kodinov

joro@mariadb.org

day's work: added more plugin types and clarified the top level group structure

10114ecb5dd1...

10114ecb5dd197317b8e2f4e12b907b74c9eb4fc

Aleksey Midenkov

midenok@gmail.com

MDEV-20865 Refactor Share_acquire::fk_error() into acquire()

Share_acquire::fk_error() inspected thd->is_error() after the acquisition to
decide whether a missing referenced table is tolerable. That breaks when an
outer Internal_error_handler consumes the error first, e.g. the
Postponed_error_handler installed by mysql_rename_tables() (MDEV-27027).

Move the decision into Share_acquire::acquire(): push a
No_such_table_error_handler for the acquisition when foreign key checks are
not enforced, so ER_NO_SUCH_TABLE is trapped above any outer handler rather
than deferred or leaked into SHOW WARNINGS. The non-tolerated error outcome
is recorded in Share_acquire::error; the four consumers check it instead of
calling fk_error().

FIXME: Squash into the main "MDEV-20865 Store foreign key info in
TABLE_SHARE" commit.

0b096e574f23...

0b096e574f23d3dee8b82454bd1109d9f1a5209f

Alessandro Vetere

iminelink@gmail.com

0a72b234ea59...

0a72b234ea59ba0f2a639e9a2104e65b95b853a4

Aleksey Midenkov

midenok@gmail.com

MDEV-40311 Fix mysqldump-nl leaving slave connection behind

The trailing CHANGE MASTER left master.info in the datadir, so a
later test that restarts the server auto-started the slave and broke
its check-testcase. Use RESET SLAVE ALL to drop the connection.

MTR's internal check of the test case 'sys_vars.default_master_connection_basic' failed.
This means that the test case does not preserve the state that existed
before the test case was executed. Most likely the test case did not
do a proper clean-up. It could also be caused by the previous test run
by this thread, if the server wasn't restarted.
This is the diff of the states of the servers before and after the
test case was executed:
-Slave_IO_Running No
-Slave_SQL_Running No
+Slave_IO_Running Connecting
+Slave_SQL_Running Yes
...
-Last_IO_Errno 0
-Last_IO_Error
+Last_IO_Errno 1045
+Last_IO_Error error connecting to master '[email protected]:3306' - retry-time: 60 maximum-retries: 100000 message: Access denied for user 'root'@'localhost' (using password: NO)

02e031059600...

02e031059600c4cd6bc9c90fda8c121ae4499718

Aleksey Midenkov

midenok@gmail.com

01db440f3803...

01db440f38039df24d0a9480b6c0627147df823e

Georgi (Joro) Kodinov

joro@mariadb.org

MDEV-40909: main.user_limits is unstable

The cleanup of the old connection goes on in the background.
It can take longer on a busy server and this trigger the active sessions
warning in DROP user.
Stabilized the test by disabling warnings around DROP USER.

012e9de65db2...

012e9de65db2a56b844eb24016ae114563acee28

Aleksey Midenkov

midenok@gmail.com

MDEV-20865 Reuse share in fk_handle_drop()

mysql_rm_table_no_locks() already opens the dropped table's share, so
pass it to fk_handle_drop() instead of re-acquiring it with
GTS_FK_SHALLOW_HINTS.

Skip fk_handle_drop() when the share is unreadable, letting the engine
report the error. This restores ER_GET_ERRNO for
main.partition_not_blackhole, which clear_error() used to mask by
zeroing my_errno.

New Share_acquire::inexistent_t lets DROP keep seeing ER_NO_SUCH_TABLE
independent of foreign_key_checks (INEXISTENT_ALWAYS).

Fixes main.partition_not_blackhole

FIXME: Squash into the main "MDEV-20865 Store foreign key info in
TABLE_SHARE" commit.

0019c8dcc961...

0019c8dcc9615be1316f2af7d4f5081e4c2d8734

Aleksey Midenkov

midenok@gmail.com