Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
fa7498dded54...
fa7498dded54548dcec1942dbe311ca2fe794e40
Sergei Golubchik
workaround for https://bugzilla.redhat.com/show_bug.cgi?id=2390105
f033f5a9ec56...
f033f5a9ec569742597c36e8ac0cc446de00f1c3
Mohammad Tafzeel Shams
MDEV-35154 : dict_sys_t::load_table() is holding exclusive dict_sys.latch
for unnecessarily long time

Issue:

dict_load_table_one() was invoked with the exclusive dict_sys.latch held
and never released it. The whole load ran inside that one critical
section: reading the SYS_TABLES record, opening the .ibd file, and
reading SYS_COLUMNS, SYS_VIRTUAL, SYS_INDEXES, SYS_FIELDS, the clustered
index root page, and SYS_FOREIGN, as well as loading every table related
by FOREIGN KEY constraints.

Almost all of that is I/O. Because dict_sys.latch is the single latch
that guards every dictionary lookup, one cold table open blocked every
other session from opening any table, including tables that were already
cached. A slow enough load could trigger the fatal semaphore wait check.

Fix:

Split the load into three phases. A short latched phase creates the
table object and publishes it as an incomplete "stub"; the I/O runs with
no latch held; a final latched phase links the FOREIGN KEY constraints.
The stub is marked with the new dict_table_t::loading, which hides it
from dict_sys_t::find_table(), so other threads treat it as not cached.
A thread that needs exactly that table waits on a condition variable and
retries its lookup, while all other tables can be looked up and loaded
concurrently.

dict_table_t::loading has three states. A table remains hidden
(LOADING_FK) until every table related to it by FOREIGN KEY constraints
has been loaded as well, and dict_sys_t::load_table() makes them visible
in one step. The intermediate state is visible to constraint linking via
the new dict_sys_t::find_table_fk(), so that two threads loading tables
that reference each other can still link the constraint between them.

Tables that InnoDB internal SQL can refer to are loaded without ever
releasing the latch. The internal SQL parser is not reentrant and is
serialized only by the exclusive dict_sys.latch, and it opens tables in
the middle of parsing; releasing the latch there let another thread run
the parser concurrently and corrupt its state.

Changes:

-  dict_load_table_one() : publish the table as a LOADING_DEF stub in
  table_non_LRU and release the latch before loading the tablespace,
  the columns and the indexes; reacquire it, move the table to
  table_LRU and mark it LOADING_FK before loading the foreign key
  constraints. Failures now remove the published stub and signal the
  waiting threads instead of only freeing the object. Add the
  hold_latch parameter and the dict_load_table_one_no_latch debug sync
  point.

-  dict_sys_t::load_table() : wait for a concurrent load of the same
  table; allocate the foreign key table names on a local heap, because
  the latch is released while draining them; in the drain loop, wait
  for a table whose definition is still being loaded by another thread
  and skip one that is only waiting for its own related tables; make
  all tables loaded by this invocation visible in one step and signal
  the waiting threads.

-  dict_load_hold_latch() : Whether a table may be referenced by
  InnoDB internal SQL, and therefore must be loaded without releasing
  the latch: InnoDB system tables, FULLTEXT INDEX auxiliary tables and
  the persistent statistics tables.

-  dict_load_table_on_id() : copy the table name and release the
  SYS_TABLES page latch before calling load_table(), which may now
  block. Restore the cursor position only if the scan has to continue.

-  dict_load_foreign(), dict_load_foreigns() : add the fk_heap
  parameter and allocate the names appended to fk_tables on it; the
  heap of the referenced table would not survive the release of the
  latch. Resolve both sides with find_table_fk().

-  dict_table_t : add NOT_LOADING, LOADING_DEF, LOADING_FK and the
  Atomic_relaxed<byte> loading member, plus debug load_thread and
  is_loader(). loading is a separate atomic and not a bit-field,
  because the loader modifies the neighbouring bit-fields without
  holding any latch.

-  dict_sys_t : add load_mutex, load_cond, find_table_any(),
  find_table_fk(), load_wait(), load_done(), allow_eviction(), and
  document that load_table() may release and reacquire the latch.

-  dict_load_foreigns() : add a fk_heap parameter for the names appended
  to fk_tables.

-  dict_sys_t::find_table_any() : the previous body of find_table(),
  returning tables that are being loaded. Only the name is read.

-  dict_sys_t::find_table() : hide tables that are being loaded, both in
  the by-name and the by-id variant. In the by-id variant, loading is
  checked before any bit-field, to avoid a torn read.

-  dict_sys_t::find_table_fk() : like find_table(), but LOADING_FK
  tables are returned, so that constraints can be linked into them
  while holding the exclusive latch.

-  dict_sys_t::load_wait(), dict_sys_t::load_done() : wait for and
  signal the completion of a load. load_wait() acquires load_mutex
  before releasing the latch, so that the wakeup cannot be lost.

-  dict_sys_t::create(), dict_sys_t::close() : initialise and destroy
  load_mutex and load_cond.

-  dict_table_can_be_evicted() : a table that is being loaded may only
  be freed by the thread that is loading it.

-  dict_foreign_add_to_cache() : resolve both sides with
  find_table_fk().

-  btr_search_disable() : skip tables that are being loaded. Walking
  their indexes would race with the loading thread, which appends to
  that list without holding the latch. Such tables cannot have any
  adaptive hash index references.

-  create_table_info_t::create_foreign_keys() : prevent the eviction of
  a referenced table as soon as it is resolved. The pointer is stored
  in a dict_foreign_t that is not in any referenced_set yet, so
  dict_sys_t::remove() would not clear it.

-  create_table_info_t::create_table() : acquire a reference to the
  created table around the foreign key handling, and use a local heap
  for the names of the foreign key related tables.

-  assertion fix : The load path may now run without dict_sys.latch, but only in the
  thread that is loading the table, and a cached table is no longer
  necessarily fully loaded. dict_sys.locked() is relaxed to
  "dict_sys.locked() || table->is_loader()" in dict_load_columns(),
  dict_load_virtual_col(), dict_load_fields(), dict_load_indexes(),
  dict_index_add_to_cache(), dict_index_find_cols(),
  dict_index_build_internal_clust(),
  dict_index_build_internal_non_clust() and
  dict_index_build_internal_fts(). Checks of dict_table_t::cached are
  relaxed and reordered after the atomic loading in
  dict_table_add_system_columns(), dict_sys_t::add(),
  dict_table_rename_in_cache() and hash_insert().

-  innodb.dict_load_concurrent
  A load is parked at dict_load_table_one_no_latch while
  holding no latch; another table can be loaded meanwhile, and a
  second opener of the same table waits. A second case checks that a
  table is not made visible while a table related to it by a FOREIGN
  KEY constraint is still being loaded by another thread.
de89a1b08d30...
de89a1b08d30554a807cb4c1a4186aaeb105ab19
Oleg Smirnov
MDEV-40003 Parallel Query: allow parallel scans on ranges in InnoDB clustered index

This commit implements partitioning of the InnoDB clustered index for
range scans, i.e., when the scan is performed on one or more intervals
of the primary key values.

It complements the previously introduced functionality of partitioning
of the whole clustered InnoDB index.

Each interval is handed to the parallel scan coordinator as a separate
scan, so each range is chunked the same way as the full index.
That means holding an S-latch on the whole index tree and a recursive
descent from the root node for each range, so it makes sense to
parallelize only when the number of ranges is relatively small,
and the ranges are large enough.
d66529ab8f86...
d66529ab8f86d39d61f83bbbe31aadfa2eace229
sjaakola
MDEV-38869 sequence conflicts with streaming replication

Sequence access conflicts with streaming replication could cause the
server to hang, as shown in MDEV-38869.

A sequence table is written from SEQUENCE::next_value() while
SEQUENCE::mutex is held. For a streaming transaction the row write in
handler::ha_write_row() would then replicate a fragment and block waiting
for certification and commit order, while an applier may be waiting for
the same mutex in SEQUENCE::set_value(). Neither side can proceed, the
node deadlocks and the BF abort of the local transaction can never be
delivered.

This commit avoids the deadlock by skipping the streaming replication
step for sequence table rows. The row is already in the write set and is
replicated with the following fragment, or at commit.

Only that one step is skipped. The skip is passed down as a parameter to
wsrep_after_row() and wsrep_after_row_internal() rather than by not
calling them at all, so the row is still counted against
wsrep_max_ws_rows and wsrep_check_pk() still runs. A transaction using
sequences heavily therefore cannot silently exceed the configured write
set row limit.

The commit has also a new mtr test for three sequence/SR conflict
scenarios: galera.galera_sequences_bf_kill_sr

- a streaming transaction and an applier competing for SEQUENCE::mutex,
  where both are expected to commit

- the same, but with the applier also BF aborting the local transaction
  over a gap lock. A streaming transaction cannot be replayed, so it is
  rolled back and the client gets ER_LOCK_DEADLOCK. The applier is held
  at the abort_trx_end sync point until the abort has been issued, so
  that the local transaction cannot finish its fragment first

- twelve row inserts on both nodes with wsrep_trx_fragment_unit=rows, so
  that each node reserves several sequence cache ranges and the sequence
  table writes land inside fragments carrying several rows. The values
  the two nodes hand out must not overlap
cd7b230d901e...
cd7b230d901e90995b9143a052f5f92525fead59
drrtuy
feat: MDEV-40672 implement basic support for the pluggable aggregate functions
cbaec9b89fb1...
cbaec9b89fb1ea88b2e7f6f77ab1a981827dd21f
Sergei Petrunia
Make sql_parallel_thread self-contained

- Move 'reaped' and 'kill_signal' OUT to sql_parallel_workers
- Move 'workers' and 'nworkers' OUT to sql_parallel_workers
- Move server_threads update logic IN.
afc7105ae8db...
afc7105ae8db8a9d1bab089a89755ae9fdceeb7b
bsrikanth-mariadb
MDEV-39226: Push whole multi-table update/delete down into engines

Give storage engines a way to take over an entire multi-table
UPDATE/DELETE, the way they can already take over a SELECT. Without it the
join, the row matching and every modification run in the SQL layer even
when an engine could do the whole statement itself in one step; a
single-table UPDATE/DELETE already avoids this via
direct_update_rows()/direct_delete_rows(), but a multi-table statement has
no primary handler object to drive that path.

This adds a generic, engine-agnostic pushdown interface: the SQL layer
offers the statement to the engine, and if the engine accepts it, it
performs the whole thing and reports only the row counts.

- Split select_handler into a pushdown_handler base with select_handler
  (result set) and a new multi_upddel_handler (runs a whole UPDATE/DELETE,
  reports row counts, reported as PUSHED UPDATE/PUSHED DELETE); add
  handlerton::create_multi_upddel, looked up in Sql_cmd_dml::execute_inner().
- multi_update/multi_delete gain direct_update_delete_done(), which records
  the engine's counts so send_eof() binlogs and replies without the
  SQL-layer loop; it forces statement-format binlogging so the change still
  replicates under binlog_format=ROW, and errors out instead of silently
  dropping counts for an unsupported result object.
- FederatedX implements the interface as the reference engine used to test
  correctness: it prints the statement back and runs it remotely, passes
  the engine's error code/SQLSTATE through, reads the matched count from the
  remote info string, executes IGNORE locally, and only pushes down when all
  tables share one remote server (same as SELECT/derived/unit pushdown).

Test: federated.federatedx_pushdown_upd_del.
ac53b14bc4b8...
ac53b14bc4b848ffc4643da0d98d787f06441f90
Yuchen Pei
MDEV-40168 wip

works:

SET SESSION debug = '+d,test_invisible_index,test_completely_invisible';
create table t1 (c int, j json, key idx ((CAST(j->'$.tags' AS CHAR(6) ARRAY))))engine=innodb;
SHOW CREATE TABLE t1;
Table Create Table
t1 CREATE TABLE `t1` (
  `c` int(11) DEFAULT NULL,
  `j` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL CHECK (json_valid(`j`))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci
show index from t1;
Table Non_unique Key_name Seq_in_index Column_name Collation Cardinality Sub_part Packed Null Index_type Comment Index_comment Ignored
t1 1 invisible1 1 invisible1 A 0 NULL NULL YES BTREE NO
t1 1 idx 1 DB_MVI_1 NULL NULL NULL NULL YES FULLTEXT NO
set @old_innodb_ft_aux_table=@@global.innodb_ft_aux_table;
set global innodb_ft_aux_table='test/t1';
insert into t1 values (1, '{"tags": ["1", "abcde", "34567"]}');
SELECT * FROM INFORMATION_SCHEMA.INNODB_FT_INDEX_CACHE;
WORD FIRST_DOC_ID LAST_DOC_ID DOC_COUNT DOC_ID POSITION
34567 1 1 1 1 2
abcde 1 1 1 1 1
DROP TABLE t1;
set global innodb_ft_aux_table=@old_innodb_ft_aux_table;
select mvi_encode('[1, 42, 3]', int);
mvi_encode('[1, 42, 3]', int)
8000000000000001 800000000000002a 8000000000000003
select mvi_encode('[1, 42, 3]', unsigned);
mvi_encode('[1, 42, 3]', unsigned)
0000000000000001 000000000000002a 0000000000000003
select mvi_encode('[1, 42, "3  "]', char(6));
mvi_encode('[1, 42, "3  "]', char(6))
31 3432 33
a514a746b81e...
a514a746b81e428ae38a758f645a34e1e364eabd
sjaakola
MDEV-38243 Write binlog row events for changes done by cascading FK operations

This commit implements a feature which changes the handling of cascading foreign
key operations to write the changes of cascading operations into binlog.
The applying of such transaction, in the slave node, will apply just the binlog
events, and does not execute the actual foreign key cascade operation.
This will simplify the slave side replication applying and make it more predictable
in terms of potential interference with other parallel applying happning
in the node.

This feature can be turned ON/OFF by new variable:
rpl_use_binlog_events_for_fk_cascade, with default value OFF

The actual implementation is largely by windsurf.

The commit has also mtr tests for testing rpl_use_binlog_events_for_fk_cascade
feature:  rpl.rpl_fk_cascade_binlog_row, rpl.rpl_fk_set_null_binlog_row and
rpl.fk_cascade_binlog_row_rollback
a1d102cf3372...
a1d102cf33722eed4dd500a405231994a7305499
Sergei Petrunia
Move all thread creation/cleanup into sql_parallel_thread.*

The cleanup API is not yet very pretty.
a07d7a3e846e...
a07d7a3e846e412c74e6e4bcf2425f35c69c0563
Oleksandr Byelkin
fix maturity
9da8bfff1662...
9da8bfff16626f7fa01aeacdc60974c0f6678a06
Yuchen Pei
MDEV-40168 wip

works:

SET SESSION debug = '+d,test_invisible_index,test_completely_invisible';
create table t1 (c int, j json, key idx ((CAST(j->'$.tags' AS CHAR(6) ARRAY))))engine=innodb;
SHOW CREATE TABLE t1;
Table Create Table
t1 CREATE TABLE `t1` (
  `c` int(11) DEFAULT NULL,
  `j` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL CHECK (json_valid(`j`))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci
show index from t1;
Table Non_unique Key_name Seq_in_index Column_name Collation Cardinality Sub_part Packed Null Index_type Comment Index_comment Ignored
t1 1 invisible1 1 invisible1 A 0 NULL NULL YES BTREE NO
t1 1 idx 1 DB_MVI_1 NULL NULL NULL NULL YES FULLTEXT NO
set @old_innodb_ft_aux_table=@@global.innodb_ft_aux_table;
set global innodb_ft_aux_table='test/t1';
insert into t1 values (1, '{"tags": ["1", "abcde", "34567"]}');
SELECT * FROM INFORMATION_SCHEMA.INNODB_FT_INDEX_CACHE;
WORD FIRST_DOC_ID LAST_DOC_ID DOC_COUNT DOC_ID POSITION
34567 1 1 1 1 2
abcde 1 1 1 1 1
DROP TABLE t1;
set global innodb_ft_aux_table=@old_innodb_ft_aux_table;
select mvi_encode('[1, 42, 3]', int);
mvi_encode('[1, 42, 3]', int)
8000000000000001 800000000000002a 8000000000000003
9ce6dbe72839...
9ce6dbe7283993e89c89ccefd8a265e9cff94d3b
Oleg Smirnov
MDEV-39491 Parallel Query: InnoDB clustered-index partitioning for parallel scan

Implement the InnoDB side of the handler parallel-scan API: partition the
clustered index into disjoint key-range chunks and serve them to the SQL layer
as pull-based scan jobs. Each chunk is read through the normal
row_search_mvcc() path, so MVCC visibility, AHI and the prefetch cache keep
working unchanged. InnoDB spawns no threads of its own.

This commit introduces only the whole index partitioning for a subsequent
full table scan in parallel. Ranges of the index are not supported
and will be implemented later.

- Parallel_coordinator (row0pcoord.{h,cc}): adapted from MySQL's
  Parallel_reader, reduced to partitioning and job distribution. Walks the
  index top-down into Exec_ctx chunks bounded by clustered-key tuples.
- ha_innobase: implements parallel_{init,end}_coordinator,
  parallel_get_worker_context, parallel_{init,end}_worker and
  parallel_get_next_row.
- Chunk-boundary clamp: row_prebuilt_t::m_pscan_end_tuple (NULL = unbounded)
  makes row_search_mvcc() stop before prefetching past the chunk's exclusive
  upper bound, treated as end-of-range so the next chunk is pulled.
- btr_pcur_open_on_user_rec(page_cur_t) overload to anchor chunk boundaries.
- Build: row0pcoord.cc added to CMakeLists.txt and auto_event_names[].
9a0cdcdaef7f...
9a0cdcdaef7fc5a6ef8ec7b26b99f73967c62e95
Oleg Smirnov
MDEV-40005 Parallel Query: allow parallel scans on secondary indexes in InnoDB

    This commit extends parallel scanning to InnoDB secondary indexes, both
    whole-index scans and scans of ranges of the key. It complements the
    previously introduced partitioning of the clustered index.

    Both unique and non-unique secondary indexes are supported, as well as
    covering and non-covering scans.
9760800bd042...
9760800bd042381325745c41642547c065de05e7
Sergei Petrunia
Make sql_parallel_thread module isolated.

This required adding
virtual pwt_worker_base::on_fatal_error()

which pwt_worker overrides.

pwt_manager_base doesn't track if fatal_error has occurred
Should it? fatal_error is in sql_parallel_workers...
9425851a6917...
9425851a6917817f176bd1dde23d2a657f394a1f
Oleg Smirnov
MDEV-40003 Parallel Query: allow parallel scans on ranges in InnoDB clustered index

This commit implements partitioning of the InnoDB clustered index for
range scans, i.e., when the scan is performed on one or more intervals
of the primary key values.

It complements the previously introduced functionality of partitioning
of the whole clustered InnoDB index.

Each interval is handed to the parallel scan coordinator as a separate
scan, so each range is chunked the same way as the full index.
That means holding an S-latch on the whole index tree and a recursive
descent from the root node for each range, so it makes sense to
parallelize only when the number of ranges is relatively small,
and the ranges are large enough.
93a9398a0ca6...
93a9398a0ca6531c176fc04c8bf6d9d83b206f8d
Oleg Smirnov
MDEV-40005 Parallel Query: allow parallel scans on secondary indexes in InnoDB

    This commit extends parallel scanning to InnoDB secondary indexes, both
    whole-index scans and scans of ranges of the key. It complements the
    previously introduced partitioning of the clustered index.

    Both unique and non-unique secondary indexes are supported, as well as
    covering and non-covering scans.
9247d4d2589b...
9247d4d2589b144c904b36ab18d707ab931f764d
bsrikanth-mariadb
MDEV-39226: Push whole multi-table update/delete down into engines

Give storage engines a way to take over an entire multi-table
UPDATE/DELETE, the way they can already take over a SELECT. Without it the
join, the row matching and every modification run in the SQL layer even
when an engine could do the whole statement itself in one step; a
single-table UPDATE/DELETE already avoids this via
direct_update_rows()/direct_delete_rows(), but a multi-table statement has
no primary handler object to drive that path.

This adds a generic, engine-agnostic pushdown interface: the SQL layer
offers the statement to the engine, and if the engine accepts it, it
performs the whole thing and reports only the row counts.

- Split select_handler into a pushdown_handler base with select_handler
  (result set) and a new multi_upddel_handler (runs a whole UPDATE/DELETE,
  reports row counts, reported as PUSHED UPDATE/PUSHED DELETE); add
  handlerton::create_multi_upddel, looked up in Sql_cmd_dml::execute_inner().
- multi_update/multi_delete gain direct_update_delete_done(), which records
  the engine's counts so send_eof() binlogs and replies without the
  SQL-layer loop; it forces statement-format binlogging so the change still
  replicates under binlog_format=ROW, and errors out instead of silently
  dropping counts for an unsupported result object.
- FederatedX implements the interface as the reference engine used to test
  correctness: it prints the statement back and runs it remotely, passes
  the engine's error code/SQLSTATE through, reads the matched count from the
  remote info string, executes IGNORE locally, and only pushes down when all
  tables share one remote server (same as SELECT/derived/unit pushdown).

Test: federated.federatedx_pushdown_upd_del.
8d4fbdb05710...
8d4fbdb05710f83caad4fce335d6bc90ec860c1c
Oleksandr Byelkin
fixed maturity
80f148702140...
80f148702140af15eb384ea51b6b618a10b39ddf
Yuchen Pei
MDEV-40168 wip

works:

SET SESSION debug = '+d,test_invisible_index,test_completely_invisible';
create table t1 (c int, j json, key idx ((CAST(j->'$.tags' AS CHAR(6) ARRAY))))engine=innodb;
SHOW CREATE TABLE t1;
Table Create Table
t1 CREATE TABLE `t1` (
  `c` int(11) DEFAULT NULL,
  `j` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL CHECK (json_valid(`j`))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci
show index from t1;
Table Non_unique Key_name Seq_in_index Column_name Collation Cardinality Sub_part Packed Null Index_type Comment Index_comment Ignored
t1 1 invisible1 1 invisible1 A 0 NULL NULL YES BTREE NO
t1 1 idx 1 DB_MVI_1 NULL NULL NULL NULL YES FULLTEXT NO
set @old_innodb_ft_aux_table=@@global.innodb_ft_aux_table;
set global innodb_ft_aux_table='test/t1';
insert into t1 values (1, '{"tags": ["1", "abcde", "34567"]}');
SELECT * FROM INFORMATION_SCHEMA.INNODB_FT_INDEX_CACHE;
WORD FIRST_DOC_ID LAST_DOC_ID DOC_COUNT DOC_ID POSITION
34567 1 1 1 1 2
abcde 1 1 1 1 1
DROP TABLE t1;
set global innodb_ft_aux_table=@old_innodb_ft_aux_table;
select mvi_encode('[1, 42, 3]', int);
mvi_encode('[1, 42, 3]', int)
8000000000000001 800000000000002a 8000000000000003
select mvi_encode('[1, 42, 3]', unsigned);
mvi_encode('[1, 42, 3]', unsigned)
0000000000000001 000000000000002a 0000000000000003
71fe571600d2...
71fe571600d20d27ed9d91bd9a194f732b98f209
PranavKTiwari
MDEV-38633: Row events in statement based binlog: optimization possible?
A failing multi-table UPDATE or DELETE marked every target temporary table as not up to date in the binary log, even when nothing had been changed.
Any later statement reading such a table was then forced to use row logging.
Only mark the tables when something was actually changed—that is, when rows were updated/deleted or a non-transactional table was modified.
If nothing changed, set THD::tmp_table_binlog_handled so that mark_tmp_table_as_free_for_reuse() does not mark them either.
718dc0812ebb...
718dc0812ebb19d72f823809c1ad0a3eea9c7644
Daniel Black
deb: add Ubuntu stonking as next release
673d3e5d0860...
673d3e5d0860d2ff974d9f17456e5541b737114c
Oleg Smirnov
MDEV-39491 Parallel Query: InnoDB clustered-index partitioning for parallel scan

Implement the InnoDB side of the handler parallel-scan API: partition the
clustered index into disjoint key-range chunks and serve them to the SQL layer
as pull-based scan jobs. Each chunk is read through the normal
row_search_mvcc() path, so MVCC visibility, AHI and the prefetch cache keep
working unchanged. InnoDB spawns no threads of its own.

This commit introduces only the whole index partitioning for a subsequent
full table scan in parallel. Ranges of the index are not supported
and will be implemented later.

- Parallel_coordinator (row0pcoord.{h,cc}): adapted from MySQL's
  Parallel_reader, reduced to partitioning and job distribution. Walks the
  index top-down into Exec_ctx chunks bounded by clustered-key tuples.
- ha_innobase: implements parallel_{init,end}_coordinator,
  parallel_get_worker_context, parallel_{init,end}_worker and
  parallel_get_next_row.
- Chunk-boundary clamp: row_prebuilt_t::m_pscan_end_tuple (NULL = unbounded)
  makes row_search_mvcc() stop before prefetching past the chunk's exclusive
  upper bound, treated as end-of-range so the next chunk is pulled.
- btr_pcur_open_on_user_rec(page_cur_t) overload to anchor chunk boundaries.
- Build: row0pcoord.cc added to CMakeLists.txt and auto_event_names[].
670173f40ffb...
670173f40ffbbcb0c17e5a0fb802eb603ea80626
Sergei Petrunia
Small cleanups: set thd->userstat_running in pwt_worker_base, comments.
5a71adc3f771...
5a71adc3f771f338365ce05a06ab12269079edc1
sjaakola
MDEV-38243 Write binlog row events for changes done by cascading FK operations

Refactoring according to Serg's review. In this version, SE/server API now
narrows the SE role to just report the changes done by foreign key cascading,
and server side does most of the work after that.

Added a design document MDEV-38243-design.md
5992b3d00bec...
5992b3d00bec6bdaee4c7dcd3efd9bb8e39e4911
bsrikanth-mariadb
MDEV-39226: Push whole multi-table update/delete down into engines

Give storage engines a way to take over an entire multi-table
UPDATE/DELETE, the way they can already take over a SELECT. Without it the
join, the row matching and every modification run in the SQL layer even
when an engine could do the whole statement itself in one step; a
single-table UPDATE/DELETE already avoids this via
direct_update_rows()/direct_delete_rows(), but a multi-table statement has
no primary handler object to drive that path.

This adds a generic, engine-agnostic pushdown interface: the SQL layer
offers the statement to the engine, and if the engine accepts it, it
performs the whole thing and reports only the row counts.

- Split select_handler into a pushdown_handler base with select_handler
  (result set) and a new multi_upddel_handler (runs a whole UPDATE/DELETE,
  reports row counts, reported as PUSHED UPDATE/PUSHED DELETE); add
  handlerton::create_multi_upddel, looked up in Sql_cmd_dml::execute_inner().
- multi_update/multi_delete gain direct_update_delete_done(), which records
  the engine's counts so send_eof() binlogs and replies without the
  SQL-layer loop; it forces statement-format binlogging so the change still
  replicates under binlog_format=ROW, and errors out instead of silently
  dropping counts for an unsupported result object.
- FederatedX implements the interface as the reference engine used to test
  correctness: it prints the statement back and runs it remotely, passes
  the engine's error code/SQLSTATE through, reads the matched count from the
  remote info string, executes IGNORE locally, and only pushes down when all
  tables share one remote server (same as SELECT/derived/unit pushdown).

Test: federated.federatedx_pushdown_upd_del.
50761fc82a2c...
50761fc82a2c4d1c591c91d2bcb3bcdc97272049
Alexander Barkov
MDEV-39518 Allow prepared statements in stored functions in assignment right hand

Allowing prepared statements in stored functions when
a stored function is used in an assignment right hand.

Both DEFAULT clause of a variable initialization and
the right side of the SET statement are supported:

  CREATE PROCEDURE p1()
  BEGIN
    -- case 1: DEFAULT clause
    DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK

    -- case 2: SP variable assignment statement
    DECLARE spvar2 INT;
    SET spvar2= f1_with_ps(); -- OK
  END;

- Only assignments to SP variables works for now:
  * SET spvar= func_with_ps(); -- OK
  * SET @uvar= func_with_ps(); -- Error

- Only bare function calls are supported for now. Using a function in
  an expression does not make it PS-safe yet:
    SET v= f1()+0;

- The parser now does not reject PS statements in stored functions.
  PS applicability in stored functions is now detected at run time.
  Note, PS statements in triggers are still prohibited by the parser.

- Functions with PS do not acquire MDL locks on tables, and no MDL is
  taken on the routines themselves either. They work like procedures in
  terms of table opening and routine locking: a concurrent DROP FUNCTION
  can complete while such a function is executing.

- Functions with PS are not replicated as a single `SELECT f1()` call.
  They are replicated per-statement, like procedures.
3deb33d367dc...
3deb33d367dcd0f5af038621d728f20e45361053
bsrikanth-mariadb
MDEV-39226: Push whole multi-table update/delete down into engines

Give storage engines a way to take over an entire multi-table
UPDATE/DELETE, the way they can already take over a SELECT. Without it the
join, the row matching and every modification run in the SQL layer even
when an engine could do the whole statement itself in one step; a
single-table UPDATE/DELETE already avoids this via
direct_update_rows()/direct_delete_rows(), but a multi-table statement has
no primary handler object to drive that path.

This adds a generic, engine-agnostic pushdown interface: the SQL layer
offers the statement to the engine, and if the engine accepts it, it
performs the whole thing and reports only the row counts.

- Split select_handler into a pushdown_handler base with select_handler
  (result set) and a new multi_upddel_handler (runs a whole UPDATE/DELETE,
  reports row counts, reported as PUSHED UPDATE/PUSHED DELETE); add
  handlerton::create_multi_upddel, looked up in Sql_cmd_dml::execute_inner().
- multi_update/multi_delete gain direct_update_delete_done(), which records
  the engine's counts so send_eof() binlogs and replies without the
  SQL-layer loop; it forces statement-format binlogging so the change still
  replicates under binlog_format=ROW, and errors out instead of silently
  dropping counts for an unsupported result object.
- FederatedX implements the interface as the reference engine used to test
  correctness: it prints the statement back and runs it remotely, passes
  the engine's error code/SQLSTATE through, reads the matched count from the
  remote info string, executes IGNORE locally, and only pushes down when all
  tables share one remote server (same as SELECT/derived/unit pushdown).

Test: federated.federatedx_pushdown_upd_del.
3a60cc8cb561...
3a60cc8cb561d9df7a67910620108426f24c73d2
sjaakola
MDEV-38243 Write binlog row events for changes done by cascading FK operations

Fixes according to Kristian Nielsen's review:
* Removed obsolete checks for slave thread
* Supporting slave with old MariaDB version.
  Events logged in cascade operation are additionally flagged with
  the long-standing NO_FOREIGN_KEY_CHECKS_F, so a replica that does
  not understand FK_CASCADE_EVENTS_F still disables foreign key checks
  and  does not re-execute the cascade

Also, thee are now binlog event flags to mark both original and derived
events. This will make it possible for the slave to choose whether to use
the derived events in applying or to execute the cascade operation

There is a new test rpl.rpl_fk_cascade_binlog_row_old_slave, for checking
compatibility with replication slave of old mariadb version
38438ca1913a...
38438ca1913ae469d2c19a3b75ef9f64a3d73c5b
Oleg Smirnov
MDEV-39845 Introduce parallel scan API

Add a handler-level interface that engines can implement to support
parallel table scans, with a serial-scan fallback when unsupported..

- HA_CAN_PARALLEL_SCAN table flag and handler::is_parallel_scan_supported()
- Coordinator-side methods (parallel_init_coordinator / parallel_end_coordinator,
  parallel_get_worker_context) driven by the master thread
- Worker-side methods (parallel_init_worker / parallel_get_next_row /
  parallel_end_worker) driven by child threads, with the
  ha_parallel_get_next_row() wrapper doing the usual bookkeeping
- Parallel_worker_ctx, an opaque per-worker context subclass
29f88336bc13...
29f88336bc13500a5f871a7c3cbf094886d502bd
Alexander Barkov
MDEV-39518 Allow prepared statements in stored functions in assignment right hand

Allowing prepared statements in stored functions when
a stored function is used in an assignment right hand.

Both DEFAULT clause of a variable initialization and
the right side of the SET statement are supported:

  CREATE PROCEDURE p1()
  BEGIN
    -- case 1: DEFAULT clause
    DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK

    -- case 2: SP variable assignment statement
    DECLARE spvar2 INT;
    SET spvar= f1_with_ps(); -- OK
  END;

- Only assignments to SP variables works for now:
  * SET spvar= func_with_ps(); -- OK
  * SET @uvar= func_with_ps(); -- Error

- Only bare function calls are supported for now. Using a function in
  an expression does not make it PS-safe yet yet:
    SET v= f1()+0;

- The parser now does not reject PS statements in stored functions.
  PS applicability in stored functions is now detected at run time.
  Note, PS statements in triggers are still prohibited by the parser.

- Functions with PS do not acquire MDL locks on tables.
  They work like procedures in terms of table opening.

- Functions with PS are not replicated as a single `SELECT f1()` call.
  They are replicated per-statement, like procedures.
27508f7a9fb3...
27508f7a9fb35e3114b51930654dbdd8528714e0
Rex Johnston
PQ  optional, TODO: simpler cleanup

... not actually simpler, but tidier.
23b74f8bf663...
23b74f8bf6632a1bded3b14b6949227d1e459d40
Rex Johnston
PQ  optional, TODO: simpler cleanup

... not actually simpler, but tidier.
17d22d3c2f2b...
17d22d3c2f2b81515cdd873fa91ab7d4fe3252ed
sjaakola
MDEV-38243 Write binlog row events for changes done by cascading FK operations

3'rd batch of review changes. Now the SE has to call for cascade service only
to report of row before and after image and the end of cascade operation.
There is new SE service interface: include/mysql/service_thd_fk_cascade.h
For server, there are 3 consumers for cascade operationss: binlogging,
firing triggers, client table FK checks, of which only binlogging has
implemeentation
171ad9f0d140...
171ad9f0d14019b000aa2c643dbcbc756107952c
Rex Johnston
PQ  tidy up message queue processing, add in missing tests, fix...

in pwt_worker_base::init_worker_thd()
15fed8d04b1d...
15fed8d04b1dd2fb0baba8d981bf993de0d699aa
Yuchen Pei
MDEV-40168 wip

works:

SET SESSION debug = '+d,test_invisible_index,test_completely_invisible';
create table t1 (c int, j json, key idx ((CAST(j->'$.tags' AS CHAR(6) ARRAY))))engine=innodb;
SHOW CREATE TABLE t1;
Table Create Table
t1 CREATE TABLE `t1` (
  `c` int(11) DEFAULT NULL,
  `j` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL CHECK (json_valid(`j`))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci
show index from t1;
Table Non_unique Key_name Seq_in_index Column_name Collation Cardinality Sub_part Packed Null Index_type Comment Index_comment Ignored
t1 1 invisible1 1 invisible1 A 0 NULL NULL YES BTREE NO
t1 1 idx 1 DB_MVI_1 NULL NULL NULL NULL YES FULLTEXT NO
set @old_innodb_ft_aux_table=@@global.innodb_ft_aux_table;
set global innodb_ft_aux_table='test/t1';
insert into t1 values (1, '{"tags": ["1", "abcde", "34567"]}');
SELECT * FROM INFORMATION_SCHEMA.INNODB_FT_INDEX_CACHE;
WORD FIRST_DOC_ID LAST_DOC_ID DOC_COUNT DOC_ID POSITION
34567 1 1 1 1 2
abcde 1 1 1 1 1
DROP TABLE t1;
set global innodb_ft_aux_table=@old_innodb_ft_aux_table;
select mvi_encode('[1, 42, 3]', int);
mvi_encode('[1, 42, 3]', int)
8000000000000001 800000000000002a 8000000000000003
select mvi_encode('[1, 42, 3]', unsigned);
mvi_encode('[1, 42, 3]', unsigned)
0000000000000001 000000000000002a 0000000000000003
13f67c0b786e...
13f67c0b786e08ce6d5fd7819412eea7470534af
Rex Johnston
PQ  tidy up message queue processing, add in missing tests, fix...

in pwt_worker_base::init_worker_thd()
107b5df017f6...
107b5df017f6eb5a55ab6ec104259b97bc881c3b
Daniel Bartholomew
bump the VERSION
08eb3de223a0...
08eb3de223a0184f9b068cc43fab32105cd16e2b
Vladislav Vaintroub
MDEV-40656 Bypass REVOKE DENY ... FROM PUBLIC privilege check.

DENY ... TO PUBLIC denies everyone, including whoever tries to revoke
it, via the "deny wins" merge at every scope (global, db, table, column,
routine). Allow REVOKE DENY ... FROM PUBLIC when the revoker can UPDATE
mysql.global_priv (same as hand-editing).

Assisted-by: Claude:claude-5-sonnet
BuildBot (0.8.9) working for the MariaDB project.
Page built: Thu 27 Aug 2026 11:40:41 (UTC)