Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
fd3dfb8dcb8c...
fd3dfb8dcb8c6e62803457b058be2fd2e0457e0a
Oleg Smirnov
MDEV-39491 Parallel Query: distribute parallel work more evenly

Lower SPLIT_THRESHOLD from 3 to 2, enabling re-partitioning of chunks
for shallower trees (one root node, one or more levels of internal nodes
and a level of leaf nodes).

Improve the detection of amount of work left: empty run queue does not
always mean the scan is over, it might be that a context pulled
for re-splitting has not enqueued its sub-contexts yet.
f5e9abfebec5...
f5e9abfebec53f6b4080a8d10215b8ba9e9e6823
Kristian Nielsen
MDEV-35691: Invalid access, use-after-free, on rli->description_event_for_exec

This commit rewrites the rpl_master_has_bug() mechanism to solve a problem
with invalid memory access. The rpl_master_has_bug() mechanism detects
certain bugs depending on the master version, and uses that to enable
specific work-arounds on the slave. The problem was that
rpl_master_has_bug() accessed Relay_log_info::description_event_for_exec
that is not valid to access from concurrent parallel replication worker
threads, only from the SQL driver thread. Thus it could use the wrong event
or access invalid/freed memory.

This patch instead computes a bitmask of detected bugs when the SQL driver
thread processes the format description event, and reads that bitmask with
an atomic load from the worker threads. The bitmask of bugs can only change
when the master restarts with a new version, and we do not replicate events
concurrently across a format description event from a master restart. Thus,
the bitmask is safe to read concurrently from the Relay_log_info object
without locking.

This also avoids an expensive match of each entry in the bug list against
the master server version done for every single call to
rpl_master_has_bug(), which could be quite expensive when done eg. per field
in row events as in Field_string::compatible_field_size().

Also remove redundant conditional in table_def::compatible_with().

Thanks to Andrei Elkin for the idea to safely read the bitmask
concurrently from the Relay_log_info.

Reviewed-by: Andrei Elkin <[email protected]>
Signed-off-by: Kristian Nielsen <[email protected]>
f399133965d4...
f399133965d486dff63891cf22904a8d54f0f489
Oleksandr Byelkin
fixed maturity
eb3a216a0175...
eb3a216a01758ffede9eeab6319c7fdd57a0e6d8
Oleg Smirnov
MDEV-39491 Parallel Query: distribute parallel work more evenly

Lower SPLIT_THRESHOLD from 3 to 2, enabling re-partitioning of chunks
for shallower trees (one root node, one or more levels of internal nodes
and a level of leaf nodes).

Improve the detection of amount of work left: empty run queue does not
always mean the scan is over, it might be that a context pulled
for re-splitting has not enqueued its sub-contexts yet.
d557ea8809f3...
d557ea8809f389670c71d8b2769f19d95b80e7ba
ParadoxV5
MDEV-40996 Support `--sync_with_master 0, $variable` in mysqltest

`--sync_with_master` uses `get_string()`,
which has `$variable` support, but it only uses the read buffer,
which is written with the unexpanded string and not the variable value.

Reviewed-by: KhaledR57 <[email protected]>
d515d66c10cb...
d515d66c10cb956cf5adb3865690bcd8289339e9
Arcadiy Ivanov
MDEV-21879 GROUP_CONCAT(DISTINCT ORDER BY) is wrong when Unique spills

`Item_func_group_concat::add()` decided whether a row was a duplicate
by checking whether `Unique::elements_in_tree()` had grown after
`unique_add()`:

    uint count= unique_filter->elements_in_tree();
    unique_filter->unique_add(get_record_pointer());
    if (count == unique_filter->elements_in_tree())
      row_eligible= FALSE;

`Unique` flushes its whole in-memory tree to disk when it runs out of
memory, and `elements_in_tree()` only counts what is still in memory.
After the first flush the test says nothing about the rows that were
already spilled.

**MDEV-11563** made this harmless for `GROUP_CONCAT(DISTINCT x)` by
building the result in `val_str()` from `unique_filter->walk()`, which
merges the spilled parts back in. It left the `ORDER BY` case alone.
There the result comes from the sort tree, which `add()` fills gated by
`row_eligible`, so the defect is still fully live.

Both directions of the failure are reachable, depending on how often
the filter flushes relative to the insert:

1. Duplicates reach the result. 100 rows holding 50 distinct values
  give all 100 values back.
2. Rows are lost. 30 distinct rows of 2000 bytes give one value back.

`JSON_ARRAYAGG(DISTINCT x ORDER BY y)` fails in the same way.

Fixed by not filling the sort tree from `add()` when `DISTINCT` is
used. `val_str()` now walks the merged `unique_filter` into the sort
tree and then walks the sort tree, so the rows are sorted after the
duplicate filtering is complete instead of during it.

`Unique::walk()` merges everything it flushed, so the sort tree can be
handed more rows than fit in memory. `insert_to_order_tree()` repacks
it on the same memory budget `add()` used, and a walk that runs out of
memory sets `result_cut`, so the user gets a cut value warning rather
than a silently short result.

**Behaviour change.** `ORDER BY` does not order rows that tie on the
ordering expression, and which of them comes first changes here. It
used to follow the order the rows were read in; it now follows the
order the duplicate filter keeps them in. Unlike the old order, the new
one depends on neither the memory available nor the physical row order.
`main.gconcat_distinct_spill` checks that, and `main.func_gconcat`
records one such tie.
d46221e5257f...
d46221e5257fc5d09dbec1036391d52e08851919
Rex Johnston
PQ: tidy up and many constraint updates
d3452c70012b...
d3452c70012b6f5324de2a1f96e680d339c8f75e
Yuchen Pei
MDEV-40168 [wip] Add multi valued index over fulltext

TODOs on top of those in the patch diff:

- EXPLAIN output should not say fulltext
- check type match to avoid false negative / positive bugs in mysql
- transcode the value into the index charset in mvi_encode_key
d30cb882a349...
d30cb882a3498b5b6650442f716139160e35f84b
Alexey (Holyfoot) Botchkov
MDEV-40394 XML schema fails on self-referencing type.

Copy XMLSchema_user_type information when the type is used recursively.
Copies that were once created are stored in m_c_free and then reused
later to avoid memory issues.
XMLSchema_item::is_validate_done() replaced with ::end_validation().
XMLSchema_group_def::check_type() now checks for circular groups.
c43d9b0989c4...
c43d9b0989c430d5c6496da8e29e6b6bcf800ecb
Sergei Petrunia
MDEV-40168: JSON-over-fulltext: add estimates.

Add records_in_range-like estimates for fulltext index
c2f630911fc1...
c2f630911fc1944211e96eccaff4e28794ee1a63
Monty
MDEV-40454 UBSAN: maria.aria_pack_mdev invalid-shift-exponent

Shifting with 64 is no-op in the the code (no ill effects)

Added a test to not do anything if shift with 64 would happen.
Tested with ma_test_all that test aria_pack.
c16a29b0da32...
c16a29b0da32404f9b03b385e931f712586bba5d
Alessandro Vetere
MDEV-32286 Reuse remembered clustered leaves in secondary-index scans

Row_sel_get_clust_rec_for_mysql::operator() descends the clustered B-tree
from the root for every row of a non-covering secondary-index scan, although
consecutive rows often land on the same clustered leaf page. ANALYZE
FORMAT=JSON charges each descent its full height, and those descents are
nearly the whole cost: the secondary index is charged its own descent and
one page for each further leaf, and nothing per row, because the position
that its cursor holds between two rows is restored optimistically, which
latches the leaf again without counting an access. The range estimate that
ha_innobase::records_in_range() makes for the optimizer is charged over the
same leaves, once more. So pages_accessed is the row count times the height
of the clustered index, plus that handful. 1000 rows over a 2-level
clustered index cost 2006, of which 3 are the scan of the secondary index
and 3 the range estimate, and 750 rows over a 3-level one cost 2291, of
which 27 and 14, where a full table scan of the same data costs 23 and 110.

Remember, in the new row_prebuilt_t::clust_leaf_hint, the
CLUST_LEAF_HINT_SLOTS (4) clustered leaves that the lookups of this
statement reached, most recently used first. Each slot names one leaf: its
page number, copies of its first and last user record truncated to the key
fields, which bound the key range the leaf held when it was remembered, and
the rec_get_offsets() of both. The copies are needed because the page is
unlatched between two lookups, and the offsets spare a lookup the parsing of
them. Several slots serve the scans that alternate between a few leaves,
which one slot cannot serve at all, and a descent refreshes the slot of a
leaf that is remembered already rather than spend a second one on the same
page. A slot owns its key buffers and grows them only when a longer key
arrives, so a row allocates nothing. The used-slot count and the miss
counter are reset per statement in ha_innobase::reset(), matching
autoinc_last_value.

A lookup first compares its key against the remembered ranges, so an
uncorrelated scan settles its misses in memory, with no buffer pool access
and no pages_accessed. Only a covering range is probed, through the new
btr_cur_t::try_leaf_hint(), which acquires the page with buf_page_try_get():
a hint is never derived from a latched parent page, so by the time it is
tried it may precede the caller's already-latched secondary-index leaf in
the latching order, where a blocking wait can deadlock. A stale range costs
a wasted probe or a needless descent, never a wrong result, because the
checks that try_leaf_hint() makes on the latched page remain the sole
authority, and the ranges therefore need no invalidation protocol.

After CLUST_LEAF_HINT_MAX_MISSES (8) consecutive unanswered lookups, a scan
gives the slots up: row_sel_clust_leaf_hint_armed() stops both the test of
the slots and the copies that refresh them, which are the larger half of
their cost. One lookup in CLUST_LEAF_HINT_RETRY (1024) starts the count
again, so a scan whose order becomes correlated only later recovers, and the
trial that this begins refreshes the slots as it goes.

The run is short because a hit saves little where the pages above the leaf
are resident: one buffer pool access and one page-local search for each
level. Measured against the same tree built without the hints, at 16k with a
resident working set and no adaptive hash index, a wholly correlated scan
runs a quarter faster over half the page accesses, a scan that answers three
lookups in five runs level with it over 30% fewer, one whose locality
appears only half way through runs an eighth faster over a quarter fewer,
and a scan that answers nothing stays within the noise. A run of 8 is what
keeps that last one there. A clustered index small enough to stay in cache
is the exception that the run does not catch: it answers often, so the count
never builds, and it saves nothing, because the descent that a hit replaces
costs almost nothing there. Such a scan pays about a tenth.

Where the adaptive hash index is enabled, the hints are neither used nor
collected: its guess solves the same problem better, landing on the record
with no page-local search and no page access to charge. It is off by
default, so the hints are active in a default configuration.

innodb.non_covering_sec_idx_scan measures pages_accessed over key orders
that differ in how closely the secondary order tracks the clustered one, and
seven further tables check query results over the record formats and key
shapes that a clustered-index lookup has to read, down to the metadata
pseudo-record of instant ALTER TABLE and to leaves that split and merge
while a locking read walks them. non_covering_sec_idx_scan_debug runs the
same body with the hints turned off, through a debug switch that returns
before a lookup tests or refreshes the slots, so a diff of the two .result
files is what the hints save: 2006 to 1028 (2-level clustered index), 2291
to 1007 (3-level), 20020 to 15780 (decorrelated), 4006 to 2012 (two
interleaved key ranges), 20020 to 20006 (shuffled) and 12016 to 9289
(locality in the second half alone). The last of those pins the retry:
without it the count is 11934.
main.rowid_filter_innodb: 90 to 81, and its ahi combination unchanged.
bdd3c3a42df9...
bdd3c3a42df9170cfbce88c71699e0f217899dd2
Sergei Golubchik
windows
bcd0a1a63a9c...
bcd0a1a63a9c506b3edd90929f34c9999b315e4a
Khaled Riyad
MDEV-40551 Copy/Paste friendly output format for MariaDB Command Line Client

Copy/paste friendly output was only reachable by starting the client with
--silent --skip-column-names, which cannot be done from a running
interactive session.

Add \S, a statement terminator which prints the result of one statement in
the tab separated format without column names.

com_silent() sets output_plain, opt_silent and column_names around
com_go(), then restores them, the same way com_ego() handles vertical.
output_plain selects print_tab_data() ahead of the vertical and table
branches, so \S gives the same output whether the session was started
plainly or with --table, --vertical or --silent. --html and --xml still
win, matching \G.
bcbf5d4875d7...
bcbf5d4875d7559cef7f42504dd39a8fc5a03267
Monty
MDEV-39949 Add per-table "SQL_CACHE" option

Add SQL_CACHE=0|1 optioon for tables to disable/enable query caches if
table is part of the query.
SQL_CACHE option can be disable for a table by using SQL_CACHE=DEFAULT

Also provide a workaround for the problem described at:
MDEV-6631 "No cache directive(SQL_CACHE) query still wait for
query cache lock" on query_cache_type=DEMAND mode"
by adding DEMAND_STRICT mode.
The new mode is needed as modifying how DEMAND works would made the
change backward incompatible.

Query_cache_type has now following options (0-2 are same as before)
OFF    (0) = Query cache off
ON    (1) = Query cache on, all queries are cached except queries marked
            with SQL_NO_CACHE
DEMAND (2) = Query cache on, only queries marked with SQL_CACHE are cached
DEMAND_STRICT
      (3) = Query cache on, only queries starting with SELECT SQL_CACHE
            are cached. This avoids a query cache lock+lookup for every
            query.
TABLES (4) = Query cache on, only cache queries marked with SQL_CACHE or
            where all used tables are using the create option SQL_CACHE=1

Note that if any of the tables used in the query has the create option
SQL_CACHE=0 or is a system table, temporary table or other non
cacheable table, then the query will not be cached.

SHOW CREATE TABLE prints SQL_CACHE=0|1 when the option is set. For tables
where the option can never apply (temporary tables, tables in the mysql
database, and engines that do not support the query cache) it is printed
as /* SQL_CACHE=1 */.

co-author: Seunguck Lee
- Took some changes related to MDEV-6631 from his patch.
b8c880ce9932...
b8c880ce9932227a825bc4b0f4b48cf989f02d12
Alessandro Vetere
MDEV-32286 Reuse remembered clustered leaves in secondary-index scans

Row_sel_get_clust_rec_for_mysql::operator() descends the clustered B-tree
from the root for every row whose clustered-index record a secondary-index
scan must read, although consecutive rows often land on the same clustered
leaf page. A non-covering scan needs one for every row, a locking read needs
one whatever the secondary index holds, because an exclusive select lock
type makes ha_innobase::build_template() build its template against the
clustered index, and a covering scan needs one for every row of a secondary
leaf whose PAGE_MAX_TRX_ID its read view cannot see. ANALYZE FORMAT=JSON
charges each descent its full height, and those descents are nearly the
whole cost: the secondary index is charged its own descent and one page for
each further leaf, and nothing per row, because the position that its cursor
holds between two rows is restored optimistically, which latches the leaf
again without counting an access. The range estimate that
ha_innobase::records_in_range() makes for the optimizer is charged over the
same leaves, once more. So pages_accessed is the row count times the height
of the clustered index, plus that handful. 1000 rows over a 2-level
clustered index cost 2006, of which 3 are the scan of the secondary index
and 3 the range estimate, and 750 rows over a 3-level one cost 2291, of
which 27 and 14, where a full table scan of the same data costs 23 and 110.

Remember, in the new row_prebuilt_t::clust_leaf_hint, the
CLUST_LEAF_HINT_SLOTS (4) clustered leaves that the lookups of this
statement reached, most recently used first. Each slot names one leaf: its
page number, copies of its first and last user record truncated to the key
fields, which bound the key range the leaf held when it was remembered, and
the rec_get_offsets() of both. The copies are needed because the page is
unlatched between two lookups, and the offsets spare a lookup the parsing of
them. Several slots serve the scans that alternate between a few leaves,
which one slot cannot serve at all, and a descent refreshes the slot of a
leaf that is remembered already rather than spend a second one on the same
page. A slot owns its key buffers and grows them only when a longer key
arrives, so a row allocates nothing. The used-slot count and the miss
counter are reset per statement in ha_innobase::reset(), matching
autoinc_last_value.

A lookup first compares its key against the remembered ranges, so an
uncorrelated scan settles its misses in memory, with no buffer pool access
and no pages_accessed. Only a covering range is probed, through the new
btr_cur_t::try_leaf_hint(), which acquires the page with buf_page_try_get():
a hint is never derived from a latched parent page, so by the time it is
tried it may precede the caller's already-latched secondary-index leaf in
the latching order, where a blocking wait can deadlock. A stale range costs
a wasted probe or a needless descent, never a wrong result, because the
checks that try_leaf_hint() makes on the latched page remain the sole
authority, and the ranges therefore need no invalidation protocol.

After CLUST_LEAF_HINT_MAX_MISSES (8) consecutive unanswered lookups, a scan
gives the slots up: row_sel_clust_leaf_hint_armed() stops both the test of
the slots and the copies that refresh them, which are the larger half of
their cost. One lookup in CLUST_LEAF_HINT_RETRY (1024) starts the count
again, so a scan whose order becomes correlated only later recovers, and the
trial that this begins refreshes the slots as it goes.

The run is short because a hit saves little where the pages above the leaf
are resident: one buffer pool access and one page-local search for each
level. Measured against the same tree built without the hints, at 16k with a
resident working set and no adaptive hash index, a wholly correlated scan
runs a quarter faster over half the page accesses, a scan that answers three
lookups in five runs level with it over 30% fewer, one whose locality
appears only half way through runs an eighth faster over a quarter fewer,
and a scan that answers nothing stays within the noise. A run of 8 is what
keeps that last one there. A clustered index small enough to stay in cache
is the exception that the run does not catch: it answers often, so the count
never builds, and it saves nothing, because the descent that a hit replaces
costs almost nothing there. Such a scan pays about a tenth.

Where the adaptive hash index is enabled, the hints are neither used nor
collected: its guess solves the same problem better, landing on the record
with no page-local search and no page access to charge. It is off by
default, so the hints are active in a default configuration.

innodb.non_covering_sec_idx_scan measures pages_accessed over key orders
that differ in how closely the secondary order tracks the clustered one, and
eight further tables check query results over the record formats and key
shapes that a clustered-index lookup has to read, down to the metadata
pseudo-record of instant ALTER TABLE, to leaves that split and merge while a
locking read walks them, and to a record that a remembered leaf supplies for
a scan that must then rebuild an older version of it.
non_covering_sec_idx_scan_debug runs the same body with the hints turned
off, through a debug switch that returns before a lookup tests or refreshes
the slots, so a diff of the two .result files is what the hints save: 2006
to 1028 (2-level clustered index), 2291 to 1007 (3-level), 20020 to 15780
(decorrelated), 4006 to 2012 (two interleaved key ranges), 20020 to 20006
(shuffled), 12016 to 9289 (locality in the second half alone) and 20020 to
10042 for a covering scan that FOR UPDATE makes non-covering, where the same
scan without FOR UPDATE costs 20 in both files. The locality count pins the
retry: without it the scan would keep only the 82 hits it makes before it
gives up, and the count would be 11934.
main.rowid_filter_innodb: 90 to 81, and its ahi combination unchanged.
b655aec30937...
b655aec30937a6d62f070503b180f51365830a00
Kristian Nielsen
MDEV-22848: SET GLOBAL gtid_slave_pos leaves dangling partial transaction

When AUTOCOMMIT=0, SET GLOBAL gtid_slave_pos did not properly commit the
(full) transaction, leaving the InnoDB hton registrered in the ha_list. This
could then later assert when InnoDB was called upon to eg. prepare() a
transaction that it does not participate in.

This patch makes rpl_slave_state::load() properly commit the (full)
transaction to solve the issue.

Reviewed-by: Brandon Nesterenko <[email protected]>
Reviewed-by: Monty <[email protected]>
Signed-off-by: Kristian Nielsen <[email protected]>
a9c2f6f1dcae...
a9c2f6f1dcaee85a584da1db68fa68ca701d3d2d
Monty
MDEV-40454 UBSAN: maria.aria_pack_mdev invalid-shift-exponent

Shifting with 64 is no-op in the the code (no ill effects).

Added a test to not do anything if shift with 64 would happen.
Tested with ma_test_all that test aria_pack.
a6118c48d175...
a6118c48d1758e2face019d7d5f240982f575b78
Rex Johnston
MDEV-39492 Parallel Query: the manager applies the plan's ORDER BY

A plain ORDER BY was refused, so most sorted queries could not run in the
workers at all. The refusal was right at the time: make_aggr_tables_info()
sorts the driving table's own read for a plan that needs no temporary table,
and the workers take that read over, so the sort lost the scan it was attached
to. This shape has no aggregation table to move it to either -- the terminal
after the driving tab sends straight to the client.

The manager does it instead, over the rows it drains. A sort is the one
post-join step indifferent to the order its input arrives in, which is what
lets a plan that ends in one be handed out in chunks.

It sorts a container of the transport's own layout, not the base table, so the
plan's Filesort cannot be reused: its order names the manager's base-table
fields and the rows are in the container. pwt_row_layout remembers which
shipped column each ORDER element sorts on and builds the equivalent order over
a container's own fields, the same way the group key is rebuilt for the
pre-aggregation containers. The drain collects instead of sending, filesort()
runs, and each row read back takes the path a drained row would have taken:
copy_back_row() into the manager's base-table records, then out. The container
is rebuilt on disk if it fills, which needs none of the cross-thread accounting
a worker's does, this one being written, sorted and read in one thread.

pwt_manager_sort_order() decides whether a plan's sort is ours, and the gate and
the setup ask the same function. Every ORDER BY element has to name a column the
container holds -- a field of a scanned table that the query reads, and so ships.
An expression has no column there, and a sort that returns row ids, unpacks into
other fields or stops early is doing something for the plan beyond ordering; both
are left serial. A plan with an aggregation table is a different shape whose sort
AGGR_OP::end_send() already performs.

pwt_table_conds() had to change with it and the order of its three sources is
load-bearing. add_sorting_to_table() hands tab->select to the Filesort and nulls
tab->select_cond, so a sorted tab keeps its condition only in the filesort -- but
push_index_cond() ran long before and left that SQL_SELECT holding just the
remainder. pre_idx_push_select_cond therefore still wins; the filesort is asked
only when there was no pushdown. Taking them the other way round silently drops
the pushed half, which main.innodb_icp catches as rows the serial plan rejects.

main.parallel_query_sort reads named rows back with query_get_value, which takes
the Nth row as the client received it and so asserts the order rather than the
multiset, ascending and descending, and again once the container has spilled to
disk. main.parallel_query_trace used ORDER BY as its example of a declined query
shape and now uses LIMIT, which is still declined.

This commit was prepared with Claude Code: it built the manager-side sort stage
on the existing container and layout machinery, and found the condition-order
defect above by running the whole main suite with parallel_worker_threads forced
on and separating the row differences from the EXPLAIN ones.
a071d8a2b927...
a071d8a2b927f61c66a88fc3a7f4d0c5ae586399
Rex Johnston
MDEV-40012 Parallel Query: the manager applies the plan's ORDER BY

A plain ORDER BY was refused, so most sorted queries could not run in the
workers at all. The refusal was right at the time: make_aggr_tables_info()
sorts the driving table's own read for a plan that needs no temporary table,
and the workers take that read over, so the sort lost the scan it was attached
to. This shape has no aggregation table to move it to either -- the terminal
after the driving tab sends straight to the client.

The manager does it instead, over the rows it drains. A sort is the one
post-join step indifferent to the order its input arrives in, which is what
lets a plan that ends in one be handed out in chunks.

It sorts a container of the transport's own layout, not the base table, so the
plan's Filesort cannot be reused: its order names the manager's base-table
fields and the rows are in the container. pwt_row_layout remembers which
shipped column each ORDER element sorts on and builds the equivalent order over
a container's own fields, the same way the group key is rebuilt for the
pre-aggregation containers. The drain collects instead of sending, filesort()
runs, and each row read back takes the path a drained row would have taken:
copy_back_row() into the manager's base-table records, then out. The container
is rebuilt on disk if it fills, which needs none of the cross-thread accounting
a worker's does, this one being written, sorted and read in one thread.

pwt_manager_sort_order() decides whether a plan's sort is ours, and the gate and
the setup ask the same function. Every ORDER BY element has to name a column the
container holds -- a field of a scanned table that the query reads, and so ships.
An expression has no column there, and a sort that returns row ids, unpacks into
other fields or stops early is doing something for the plan beyond ordering; both
are left serial. A plan with an aggregation table is a different shape whose sort
AGGR_OP::end_send() already performs.

pwt_table_conds() had to change with it and the order of its three sources is
load-bearing. add_sorting_to_table() hands tab->select to the Filesort and nulls
tab->select_cond, so a sorted tab keeps its condition only in the filesort -- but
push_index_cond() ran long before and left that SQL_SELECT holding just the
remainder. pre_idx_push_select_cond therefore still wins; the filesort is asked
only when there was no pushdown. Taking them the other way round silently drops
the pushed half, which main.innodb_icp catches as rows the serial plan rejects.

main.parallel_query_sort reads named rows back with query_get_value, which takes
the Nth row as the client received it and so asserts the order rather than the
multiset, ascending and descending, and again once the container has spilled to
disk. main.parallel_query_trace used ORDER BY as its example of a declined query
shape and now uses LIMIT, which is still declined.

A capped SELECT must not run in the workers

With SET sql_select_limit=3, a query the gate accepted returned every row --
500 where the serial plan sends 3. The serial executor enforces a row cap in
end_send(), against unit->lim; the manager's drain sends every row a worker
ships and never consults it. So the gate must refuse any select whose unit
carries a cap, and it was testing the syntax instead of the cap:
limit_params.explicit_limit is only set by a LIMIT clause, while
sql_select_limit is installed by mysql_execute_command() as the default limit
of a top-level SELECT, with explicit_limit still unset.

The gate now tests join->unit->lim.is_unlimited(), which is the very value
end_send() would have enforced. This subsumes the explicit-LIMIT case, and it
is per-unit, so the session cap -- which applies only to the top-level select
-- does not cost a derived table's select its parallel scan.

Found reviewing the ORDER BY commit: its fs->limit != HA_ROWS_MAX check turned
out to be load-bearing for the same reason (the implicit cap becomes a filesort
limit), which raised the question of what protected the unsorted path. Nothing
did.

setup_worker_jointabs() builds each worker tab as a struct copy of the
manager's, and since the manager grew a sort stage the driving tab's copy has
carried the plan's Filesort pointer. Nothing reads it in a worker -- the
driving tab reads through the chunk reader, not join_init_read_record() -- and
worker tabs never run JOIN_TAB::cleanup(), so it is inert today. But
JOIN_TAB::cleanup() deletes the filesort and, through it, the SQL_SELECT
holding the plan's condition, so the copy was one future teardown call away
from a double free. The function's own policy is that manager-owned pointers
are cleared from the copy (select, cache_select, pre_idx_push_select_cond);
filesort and filesort_result now join the list, and pwt_assert_tab_inert()
pins the shape: a filesort is only ever the driving table's, and no sort has
run when workers are set up.

No test: the hazard is latent, reachable only by a teardown path that does not
exist yet. The asserts held over the whole main suite run with
parallel_worker_threads forced on.

This commit was prepared with Claude Code: it built the manager-side sort stage
on the existing container and layout machinery, and found the condition-order
defect above by running the whole main suite with parallel_worker_threads forced
on and separating the row differences from the EXPLAIN ones.

This commit was prepared with Claude Code: it probed the drain path with an
implicit sql_select_limit after noticing that only the sorted shape declined,
and confirmed the serial and parallel row counts diverged.

This commit was prepared with Claude Code, as a hardening it proposed while
reviewing the manager-side ORDER BY commit.
7fd0ed6419c9...
7fd0ed6419c9219fb8de3029f8110b5c4ff9dd4d
Georg Richter
CONC-850: Fix out-of-bounds reads in parse_server_packet (GSSAPI)

Replace strncpy/strnlen parsing in plugins/auth/auth_gssapi_client.c
with bounded memchr/memcpy calls to prevent two security vulnerabilities:

1. Stack Buffer Over-Read: Oversized SPN inputs without a NUL byte caused
  strncpy to omit NUL-termination, leading to OOB reads in strlen().
2. Packet Over-Read: Malformed packets missing a NUL terminator caused
  mechanism parsing to read past the end of the packet buffer.

Thanks to Aisle Research for reporting this issue.
7cdbbb44a7ed...
7cdbbb44a7ed7834e9faae1bf10d1bb15464b0a0
Kristian Nielsen
MDEV-22848: SET GLOBAL gtid_slave_pos leaves dangling partial transaction

When AUTOCOMMIT=0, SET GLOBAL gtid_slave_pos did not properly commit the
(full) transaction, leaving the InnoDB hton registrered in the ha_list. This
could then later assert when InnoDB was called upon to eg. prepare() a
transaction that it does not participate in.

This patch makes rpl_slave_state::load() properly commit the (full)
transaction to solve the issue.

Reviewed-by: Brandon Nesterenko <[email protected]>
Reviewed-by: Monty <[email protected]>
Signed-off-by: Kristian Nielsen <[email protected]>
77afbfd57812...
77afbfd57812422ef25b5e634dc0060e4652824b
Monty
Speed up the query_cache interface

- Do checks inline before calling query_cache.send_result_to_client()

Things tried that failed:
- I tried removing calling lex_start() and reset_for_next_command() if
  query is cached, but this is would cause duplicated, hard to maintain
  code and would require double resets of things if query cache was
  not used. I have documented in sql_parse.cc what would need to be
  reset for this approach to work.
6d7c9d455ad6...
6d7c9d455ad626770b709372cf0c8a6c6ba17fd9
Oleg Smirnov
MDEV-39491 Parallel Query: distribute parallel work more evenly

Lower SPLIT_THRESHOLD from 3 to 2, enabling re-partitioning of chunks
for shallower trees (one root node, one or more levels of internal nodes
and a level of leaf nodes).

Improve the detection of amount of work left: empty run queue does not
always mean the scan is over, it might be that a context pulled
for re-splitting has not enqueued its sub-contexts yet.
66951c20a5ac...
66951c20a5ac169beb123710c902bb2745a9eaf0
Georg Richter
CONC-842: Fix incomplete source bounds check and state desync in mthd_my_read_rows

Commit 85c322f70e47 introduced source buffer bounds validation against end_cp,
but contained edge-case flaws allowing heap buffer over-reads and data leaks:

1. Pointer Underflow: net_field_length() advances 'cp'. If 'cp' advances past
  'end_cp', (end_cp - cp) underflows into a large positive unsigned integer,
  causing 'len > (ulong)(end_cp - cp)' to evaluate to false and bypass
  memcpy bounds enforcement. Fix this by explicitly checking 'cp > end_cp'.

2. State Desynchronization: When 'cp >= end_cp', the previous fix set
  remaining field pointers to NULL and continued loop execution. On truncated
  or malformed packets, completing the row allows corrupted or stale heap
  data from previous queries to be returned. Fix this by failing immediately
  with CR_MALFORMED_PACKET when 'cp >= end_cp'.

3. EOF Status Read Bounds: Added validation before reading warning_count
  and server_status from EOF status packets to prevent out-of-bounds reads
  on short EOF frames.

Thanks to fg0x0 for analyzing and reporting the guard bypass.
5eeea51e83be...
5eeea51e83bebe498427a8987d548cc314070484
Oleg Smirnov
MDEV-39491 Parallel Query: distribute parallel work more evenly

Lower SPLIT_THRESHOLD from 3 to 2, enabling re-partitioning of chunks
for shallower trees (one root node, one or more levels of internal nodes
and a level of leaf nodes).

Improve the detection of amount of work left: empty run queue does not
always mean the scan is over, it might be that a context pulled
for re-splitting has not enqueued its sub-contexts yet.
5dd6b9252bff...
5dd6b9252bffbb246cd23a1f60b2da89b521001a
Kristian Nielsen
MDEV-35691: Invalid access, use-after-free, on rli->description_event_for_exec

This commit rewrites the rpl_master_has_bug() mechanism to solve a problem
with invalid memory access. The rpl_master_has_bug() mechanism detects
certain bugs depending on the master version, and uses that to enable
specific work-arounds on the slave. The problem was that
rpl_master_has_bug() accessed Relay_log_info::description_event_for_exec
that is not valid to access from concurrent parallel replication worker
threads, only from the SQL driver thread. Thus it could use the wrong event
or access invalid/freed memory.

This patch instead computes a bitmask of detected bugs when the SQL driver
thread processes the format description event, and reads that bitmask with
an atomic load from the worker threads. The bitmask of bugs can only change
when the master restarts with a new version, and we do not replicate events
concurrently across a format description event from a master restart. Thus,
the bitmask is safe to read concurrently from the Relay_log_info object
without locking.

This also avoids an expensive match of each entry in the bug list against
the master server version done for every single call to
rpl_master_has_bug(), which could be quite expensive when done eg. per field
in row events as in Field_string::compatible_field_size().

Also remove redundant conditional in table_def::compatible_with().

Thanks to Andrei Elkin for the idea to safely read the bitmask
concurrently from the Relay_log_info.

Reviewed-by: Andrei Elkin <[email protected]>
Signed-off-by: Kristian Nielsen <[email protected]>
5ae99c3831fa...
5ae99c3831fae0efdf7f716a049b872d1941d70b
ParadoxV5
MDEV-39880: Reïmplement MDEV-37146 to include MDEV-39519

MDEV-39519 added MySQL 8 compatibility to `mariadb-dump --dump-slave`
by attempting `SHOW REPLICA STATUS` first and then `SHOW SLAVE STATUS`.
This conflicted with MDEV-37146, where `mariadb-dump --dump-slave`
queries either `SELECT … FROM information_schema.SLAVE_STATUS`
or `SHOW ALL SLAVES STATUS` depending on the server version.

This commit merges MDEV-37146 and MDEV-39519:
* Use MDEV-39519’s strategy based on syntax error handling.
* Use MDEV-37146’s preference order:
  1. `SELECT … FROM information_schema.SLAVE_STATUS`
  2. `SHOW ALL SLAVES STATUS`
  3. `SHOW REPLICA STATUS` (for MySQL compatibility _only_)
* Send `STOP`/`START REPLICA SQL_THREAD FOR CHANNEL '…'`
  commands for both MariaDB 10.7+ and MySQL.
* Refactor column indices to variables set when a query succeeds.
* Partially revert MDEV-37146’s removal of `--dump-slave`’s support for
  pre-GTID & pre-multi-source, but tailored for MySQL compatibility;
  coverage for MariaDB pre-10.0 is not fully restored.

Reviewed-by: Brandon Nesterenko <[email protected]>
5a0d2432efb7...
5a0d2432efb7e44747d11f58170e906663b88679
Yuchen Pei
MDEV-40168 [wip] Add multi valued index over fulltext

TODOs on top of those in the patch diff:

- EXPLAIN output should not say fulltext
- check type match to avoid false negative / positive bugs in mysql
- transcode the value into the index charset in mvi_encode_key
59e04c3aa079...
59e04c3aa079b61e86ebf8021279ea53f1d6bd0c
ParadoxV5
MDEV-39880: Reïmplement MDEV-37146 to include MDEV-39519

MDEV-39519 added MySQL 8 compatibility to `mariadb-dump --dump-slave`
by attempting `SHOW REPLICA STATUS` first and then `SHOW SLAVE STATUS`.
This conflicted with MDEV-37146, where `mariadb-dump --dump-slave`
queries either `SELECT … FROM information_schema.SLAVE_STATUS`
or `SHOW ALL SLAVES STATUS` depending on the server version.

This commit merges MDEV-37146 and MDEV-39519:
* Use MDEV-39519’s strategy based on syntax error handling.
* Use MDEV-37146’s preference order:
  1. `SELECT … FROM information_schema.SLAVE_STATUS`
  2. `SHOW ALL SLAVES STATUS`
  3. `SHOW REPLICA STATUS` (for MySQL compatibility _only_)
* Send `STOP`/`START REPLICA SQL_THREAD FOR CHANNEL '…'`
  commands for both MariaDB 10.7+ and MySQL.
* Refactor column indices to variables set when a query succeeds.
* Partially revert MDEV-37146’s removal of `--dump-slave`’s support for
  pre-GTID & pre-multi-source, but tailored for MySQL compatibility;
  coverage for MariaDB pre-10.0 is not fully restored.

Reviewed-by: Brandon Nesterenko <[email protected]>
454881b41ecc...
454881b41ecc7a4689097d0bf07908290194f9bc
Marko Mäkelä
MDEV-40932 SET GLOBAL innodb_log_archive=OFF may still break recovery

log_t::set_archive(archive=false): Ensure that both the latest checkpoint
and the latest log record (which has possibly not been written out yet)
will carry the log_sys.get_sequence_bit(lsn)==1, to guarantee a successful
recovery after the switch to the innodb_log_archive=OFF format.

Tested by: Matthias Leich
Reviewed by: Thirunarayanan Balathandayuthapani
344ad212a1b5...
344ad212a1b58ad22ee84a624e34493f19f266bc
Yuchen Pei
MDEV-40168 [wip] Add multi valued index over fulltext

TODOs on top of those in the patch diff:

- EXPLAIN output should not say fulltext
- check type match to avoid false negative / positive bugs in mysql
- transcode the value into the index charset in mvi_encode_key
31914fb74029...
31914fb7402925bb95f13394bcb529aae4c9329c
Sergei Petrunia
Tmp: added comments
2362eb264048...
2362eb264048e930dc9eb42da444086d8fce464b
Arcadiy Ivanov
MDEV-41007 Warn when GROUP_CONCAT(DISTINCT) loses rows silently

Give a warning when `GROUP_CONCAT(DISTINCT x)` or
`JSON_ARRAYAGG(DISTINCT x)` returns only part of a group, or nothing at
all, because the walk of the duplicate filter failed. The result is
wrong rather than deliberately cut, and nothing was said about it.

Both build their result in `val_str()` by walking `unique_filter`, and
threw the walk's return value away. `Unique::walk()` reports its own
failures through it, from allocating the merge buffer to reading back
the chunks it merged, so a failure gave a short result, or an empty
one, in silence.

The return value cannot be used on its own. `dump_leaf_key()` also
stops the walk, for two reasons that are not failures: it cuts the
result at `group_concat_max_len`, which it already reports by setting
`result_cut`, and it stops without losing anything once the `LIMIT` is
used up. Reporting every non-zero return as a cut warns about
`GROUP_CONCAT(DISTINCT a LIMIT 5)` returning exactly the five rows
that were asked for.

`dump_leaf_key()` now records that it was the one that stopped the
walk, so `val_str()` asks for the cut value warning only when the walk
itself failed.

The debug keyword `unique_walk_merge_fail` fails the merging walk, and
`main.gconcat_distinct_walk_fail` uses it. The `LIMIT` case needs no
debug build and is checked in `main.gconcat_distinct_spill`.
18f34b2d00e8...
18f34b2d00e8d36b9f31f56d1d62c192cfa6d9e1
ParadoxV5
MDEV-39880: Reïmplement MDEV-37146 to include MDEV-39519

MDEV-39519 added MySQL 8 compatibility to `mariadb-dump --dump-slave`
by attempting `SHOW REPLICA STATUS` first and then `SHOW SLAVE STATUS`.
This conflicted with MDEV-37146, where `mariadb-dump --dump-slave`
queries either `SELECT … FROM information_schema.SLAVE_STATUS`
or `SHOW ALL SLAVES STATUS` depending on the server version.

This commit merges MDEV-37146 and MDEV-39519:
* Use MDEV-39519’s strategy based on syntax error handling.
* Use MDEV-37146’s preference order:
  1. `SELECT … FROM information_schema.SLAVE_STATUS`
  2. `SHOW ALL SLAVES STATUS`
  3. `SHOW REPLICA STATUS` (for MySQL compatibility _only_)
* Send `STOP`/`START REPLICA SQL_THREAD FOR CHANNEL '…'`
  commands for both MariaDB 10.7+ and MySQL.
* Refactor column indices to variables set when a query succeeds.
* Partially revert MDEV-37146’s removal of `--dump-slave`’s support for
  pre-GTID & pre-multi-source, but tailored for MySQL compatibility;
  coverage for MariaDB pre-10.0 is not fully restored.

Reviewed-by: Brandon Nesterenko <[email protected]>
12c540ddd4bb...
12c540ddd4bb6e724d02abf69ddbb6f2e07a4c7e
Thirunarayanan Balathandayuthapani
MDEV-19574: innodb_stats_method is not honored when innodb_stats_persistent=ON

Problem:
=======
When persistent statistics are enabled (innodb_stats_persistent=ON),
the innodb_stats_method setting is not properly utilized during
statistics calculation.

The statistics collection functions always use a hardcoded default
behavior for NULL value comparison instead of respecting the
configured stats method. This affects the accuracy of
n_diff_key_vals (distinct key count), particularly for
indexes with nullable columns containing NULL values.

Moreover, stat_n_non_null_key_vals[] was never computed for
persistent statistics; it stayed at the 0 that
dict_stats_empty_index() assigns.

With innodb_stats_method=nulls_ignored, innodb_rec_per_key()
therefore always found n_diff <= n_null and reported one record
per key for every index. This impacts the query optimizer,
which makes decisions based on inaccurate cardinality estimates.

Solution:
========
Introduced IndexLevelStats to collect statistics at a specific
B-tree level during index analysis.

Introduced PageStats to collect statistics for leaf page analysis.

Refactored the following functions:
dict_stats_analyze_index_level() to IndexLevelStats::analyze_level()
dict_stats_analyze_index_for_n_prefix() to IndexLevelStats::sample_leaf_pages()
dict_stats_analyze_index_below_cur() to PageStats::scan_below()
dict_stats_scan_page() to PageStats::scan()

The innodb_stats_method value is read once per table in
dict_stats_update_persistent() and passed down, so that all
indexes of a table are analyzed with the same method.

Add the stats method name to stat_description when
innodb_stats_method has a non-default value. The suffix is
dropped when the description is already full.

Added the new stat name n_nonnull_fld01, n_nonnull_fld02, etc.
with a stats description, to indicate how many non-null values
exist for the nth field of the index. This value is retrieved
and stored in the index statistics
in dict_stats_fetch_index_stats_step(). The counts are per
column, not per n-column prefix.

rec_get_n_blob_pages(): Calculate the number of
externally stored pages for a record, using ceiling division
by the usable BLOB page payload (blob_part_size), which differs
between ROW_FORMAT=COMPRESSED (zip_size minus FIL_PAGE_DATA) and
the other formats (srv_page_size minus the BLOB header and the
page trailer). For ROW_FORMAT=COMPRESSED the length in
the field reference is the uncompressed length, so the result is an
upper bound.

When the leaf level is scanned in full, the number of leaf pages that
were scanned is reported as n_leaf_pages for a multi level index.
Before, result.n_leaf_pages was overwritten with
index->stat_n_leaf_pages, which dict_stats_empty_index() had
just set to 1, so every index that took the full scan path
reported n_leaf_pages=1. Single page indexes report 1.
This changes cardinality estimates and
therefore leads to multiple changes in existing test cases.

Non-null values are counted only at the leaf level, since only leaf
pages hold actual records. A full scan of the leaf level counts them
exactly. When the level is sampled, the per column count is derived
from the sampled leaves with the same formula as n_diff:

  n_ordinary_leaf_pages * n_non_null_all_analyzed_pages
                        / n_leaf_pages_to_analyze

This is an estimate for NOT NULL columns as well: the sampled leaves
may hold fewer or more records than the average, and a dive that
stops at a boring page contributes nothing to the sum while still
counting in the divisor.

innodb_rec_per_key(): stat_n_non_null_key_vals[i] holds the
number of records in which the i-th indexed column alone is
not NULL, while what has to be excluded here is the number
of records whose first i+1 columns are all not NULL,
because that is the population which the n-column
prefix statistic stat_n_diff_key_vals[i] has to be
corrected against when innodb_stats_method=nulls_ignored:
with NULLs compared as unequal, every record carrying a NULL
anywhere in the prefix adds a distinct value of its own to n_diff.

PageStats::scan(): n_non_null is accumulated and assigned only
for leaf pages, so that a non-leaf scan cannot leave a node
pointer count behind when scan_below() stops at a boring page
without reaching a leaf.

IndexLevelStats::reset_for_level() also clears n_diff[], and
dict_stats_analyze_index() zero initializes the buffer backing it, so
that a level scan which finds no records (a failed
btr_pcur_open_level(), or a non-leaf page whose first record is not
marked as the leftmost one on the level) leaves n_diff[] at 0 instead
of stale values.

IndexLevelStats::sample_leaf_pages() returns early when the group
boundaries for the prefix are empty, which is the same condition.

IndexLevelStats::analyze_level(): Instead of copying the last record
of the page, retain the latch on the page until the record has been
compared with the first record of the next page

dict_stats_fetch_index_stats_step() no longer resets
stat_n_non_null_key_vals[] while processing an n_diff_pfxNN row:
dict_stats_empty_table() has already cleared the array before the
fetch, and with n_nonnull_fldNN rows now being read too,
that reset would make the result depend on the order in which the
rows arrive.

dict_stats_save(): now static function in dict0stats.cc that
takes the innodb_stats_method value, and is removed from dict0stats.h.
dict_stats_update_persistent() saves the statistics itself, so its
callers no longer have to.

Replaced btr_rec_get_externally_stored_len() with
rec_get_n_blob_pages() in dict0stats.cc.

btr_rec_get_field_ref_offs() and btr_rec_get_field_ref(),
together with the BTR_BLOB_HDR_* macros, were moved from btr0cur.cc
to btr0cur.h so that rec_get_n_blob_pages() can reuse them;

btr_rec_get_field_ref_offs() is now a noexcept function
returning size_t.

Changed stat_n_diff_key_vals and stat_n_non_null_key_vals from
ib_uint64_t* to uint64_t*

len_is_stored(): simplified to a single comparison, which is
equivalent for the unsigned lengths that it is used with.

Removed the unused UNIV_STATS_DEBUG build macro (univ.i) and
turned the DEBUG_PRINTF() helper in dict0stats.cc into an
unconditional no-op
0d9923af3193...
0d9923af3193d81b54252e5c7caa36c65aec82e3
Yuchen Pei
MDEV-40168 [wip] Add multi valued index over fulltext

TODOs on top of those in the patch diff:

- EXPLAIN output should not say fulltext
- check type match to avoid false negative / positive bugs in mysql
- transcode the value into the index charset in mvi_encode_key
02da3aca98b7...
02da3aca98b7118d58c06b29209de4e2756998b4
Alessandro Vetere
MDEV-32286 Reuse remembered clustered leaves in secondary-index scans

Row_sel_get_clust_rec_for_mysql::operator() descends the clustered B-tree
from the root for every row whose clustered-index record a secondary-index
scan must read, although consecutive rows often land on the same clustered
leaf page. A non-covering scan needs one for every row, a locking read needs
one whatever the secondary index holds, because an exclusive select lock
type makes ha_innobase::build_template() build its template against the
clustered index, and a covering scan needs one for every row of a secondary
leaf whose PAGE_MAX_TRX_ID its read view cannot see. ANALYZE FORMAT=JSON
charges each descent its full height, and those descents are nearly the
whole cost: the secondary index is charged its own descent and one page for
each further leaf, and nothing per row, because the position that its cursor
holds between two rows is restored optimistically, which latches the leaf
again without counting an access. The range estimate that
ha_innobase::records_in_range() makes for the optimizer is charged over the
same leaves, once more. So pages_accessed is the row count times the height
of the clustered index, plus that handful. 1000 rows over a 2-level
clustered index cost 2006, of which 3 are the scan of the secondary index
and 3 the range estimate, and 750 rows over a 3-level one cost 2291, of
which 27 and 14, where a full table scan of the same data costs 23 and 110.

Remember, in the new row_prebuilt_t::clust_leaf_hint, the
CLUST_LEAF_HINT_SLOTS (4) clustered leaves that the lookups of this
statement reached, most recently used first. Each slot names one leaf: its
page number, copies of its first and last user record truncated to the key
fields, which bound the key range the leaf held when it was remembered, and
the rec_get_offsets() of both. The copies are needed because the page is
unlatched between two lookups, and the offsets spare a lookup the parsing of
them. Several slots serve the scans that alternate between a few leaves,
which one slot cannot serve at all, and a descent refreshes the slot of a
leaf that is remembered already rather than spend a second one on the same
page. A slot owns its key buffers and grows them only when a longer key
arrives, so a row allocates nothing. The used-slot count and the miss
counter are reset per statement in ha_innobase::reset(), matching
autoinc_last_value.

A lookup first compares its key against the remembered ranges, so an
uncorrelated scan settles its misses in memory, with no buffer pool access
and no pages_accessed. Only a covering range is probed, through the new
btr_cur_t::try_leaf_hint(), which acquires the page with buf_page_try_get():
a hint is never derived from a latched parent page, so by the time it is
tried it may precede the caller's already-latched secondary-index leaf in
the latching order, where a blocking wait can deadlock. A stale range costs
a wasted probe or a needless descent, never a wrong result, because the
checks that try_leaf_hint() makes on the latched page remain the sole
authority, and the ranges therefore need no invalidation protocol.

After CLUST_LEAF_HINT_MAX_MISSES (8) consecutive unanswered lookups, a scan
gives the slots up: row_sel_clust_leaf_hint_armed() stops both the test of
the slots and the copies that refresh them, which are the larger half of
their cost. One lookup in CLUST_LEAF_HINT_RETRY (1024) starts the count
again, so a scan whose order becomes correlated only later recovers, and the
trial that this begins refreshes the slots as it goes.

The run is short because a hit saves little where the pages above the leaf
are resident: one buffer pool access and one page-local search for each
level. Measured against the same tree built without the hints, at 16k with a
resident working set and no adaptive hash index, a wholly correlated scan
runs a quarter faster over half the page accesses, a scan that answers three
lookups in five runs level with it over 30% fewer, one whose locality
appears only half way through runs an eighth faster over a quarter fewer,
and a scan that answers nothing stays within the noise. A run of 8 is what
keeps that last one there. A clustered index small enough to stay in cache
is the exception that the run does not catch: it answers often, so the count
never builds, and it saves nothing, because the descent that a hit replaces
costs almost nothing there. Such a scan pays about a tenth.

Where the adaptive hash index is enabled, the hints are neither used nor
collected: its guess solves the same problem better, landing on the record
with no page-local search and no page access to charge. It is off by
default, so the hints are active in a default configuration.

innodb.non_covering_sec_idx_scan measures pages_accessed over key orders
that differ in how closely the secondary order tracks the clustered one, and
eight further tables check query results over the record formats and key
shapes that a clustered-index lookup has to read, down to the metadata
pseudo-record of instant ALTER TABLE, to leaves that split and merge while a
locking read walks them, and to a record that a remembered leaf supplies for
a scan that must then rebuild an older version of it.
non_covering_sec_idx_scan_debug runs the same body with the hints turned
off, through a debug switch that returns before a lookup tests or refreshes
the slots, so a diff of the two .result files is what the hints save: 2006
to 1028 (2-level clustered index), 2291 to 1007 (3-level), 20020 to 15780
(decorrelated), 4006 to 2012 (two interleaved key ranges), 20020 to 20006
(shuffled), 12016 to 9289 (locality in the second half alone) and 20020 to
10042 for a covering scan that FOR UPDATE makes non-covering, where the same
scan without FOR UPDATE costs 20 in both files. The locality count pins the
retry: without it the scan would keep only the 82 hits it makes before it
gives up, and the count would be 11934.
main.rowid_filter_innodb: 90 to 81, and its ahi combination unchanged.
BuildBot (0.8.9) working for the MariaDB project.
Page built: Thu 03 Sep 2026 06:31:34 (UTC)