Console View
|
Categories: connectors experimental galera main |
|
| connectors | experimental | galera | main | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thirunarayanan Balathandayuthapani
thiru@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40922 Compilation warning "'<anonymous>' may be used uninitialized" in RecordCallback constructor with GCC Move the std::function callbacks into the members instead of copying them, and pass the captureless FTS comparators as plain functions instead of lambdas. This is to avoid GCC -Wmaybe-uninitialized false positive in RecordCallback |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Georgi (Joro) Kodinov
joro@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40815: resolveip is not built for the minbuild cmake target Added the resolveip target to the minbuild target. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| use export target | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thirunarayanan Balathandayuthapani
thiru@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40621 InnoDB: Failing assertion: doc_id == src_node->last_doc_id AuxRecordReader::default_word_processor(): InnoDB fails to consider the ilist data can be stored externally while decoding the auxiliary table record. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Georgi (Joro) Kodinov
joro@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39718: Produce Markdown plugin API documentation Generated the plugin API headers using a shell script. Fixed some doxygen comment mistakes in the headers. Added a cmake conveninence target to generate the docs into $BUILD_DIR/docs Added a main page for the API docs. Included all of the existing group .md files into the CMake target Leveraged moxygen 2.1.11's fixes to produce the full API docs in a single go Removed the list of output .md files from the CMake target and switched to a stamp file to avoid unnecessary rebuilds of the docs when the list of .md files changes. Addressed various review comments. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Removed some not needed checks and add a DBUG_ASSERT() for not covered code - In ha_partition.cc:check_parallel_search(), remove check if item_field->field is null. This is not needed as the function is run after fix_field() which guarnatees that the field is always set. - Added DBUG_ASSERT(new_field) to Item_field::fix_fields() to check if a select-list item, found by name or alias when resolving ORDER BY/GROUP BY/HAVING, can have field == 0. This error path is not covered by any mtr test. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
squash! e61cabc0a2ae1a58c401fc24bb9f25caab6dc9b5 Remove traces of HTON_CHECK_NEEDED_FOR_CREATE_OR_REPLACE and handler::can_be_renamed_to_backup() that were made redundant by MDEV-28933 (commit cffbb17480a6fba6bf8cb42d943833cca214b34a) |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Georgi (Joro) Kodinov
joro@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Addendum to MDEV-20749's fix: addressed Kristian's comments on indenting and #ifdef-ing. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| fix include paths again | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40454 UBSAN: maria.aria_pack_mdev invalid-shift-exponent Shifting with 64 is no-op in the the code (no ill effects) Added a test to not do anything if shift with 64 would happen. Tested with ma_test_all that test aria_pack. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
cleanup: mariadb-dump remove dead code, handling of MariaDB/MySQL servers that didn't have SQL_QUOTE_SHOW_CREATE. SQL_QUOTE_SHOW_CREATE was added in 2000 (commit 5762523b788, MySQL 3.23.25-beta) |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alessandro Vetere
iminelink@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-32286 Reuse remembered clustered leaves in secondary-index scans Row_sel_get_clust_rec_for_mysql::operator() descends the clustered B-tree from the root for every row of a non-covering secondary-index scan, although consecutive rows often land on the same clustered leaf page. ANALYZE FORMAT=JSON charges each descent its full height, and those descents are nearly the whole cost: the secondary index is charged its own descent and one page for each further leaf, and nothing per row, because the position that its cursor holds between two rows is restored optimistically, which latches the leaf again without counting an access. The range estimate that ha_innobase::records_in_range() makes for the optimizer is charged over the same leaves, once more. So pages_accessed is the row count times the height of the clustered index, plus that handful. 1000 rows over a 2-level clustered index cost 2006, of which 3 are the scan of the secondary index and 3 the range estimate, and 750 rows over a 3-level one cost 2291, of which 27 and 14, where a full table scan of the same data costs 23 and 110. Remember, in the new row_prebuilt_t::clust_leaf_hint, the CLUST_LEAF_HINT_SLOTS (4) clustered leaves that the lookups of this statement reached, most recently used first. Each slot names one leaf: its page number, copies of its first and last user record truncated to the key fields, which bound the key range the leaf held when it was remembered, and the rec_get_offsets() of both. The copies are needed because the page is unlatched between two lookups, and the offsets spare a lookup the parsing of them. Several slots serve the scans that alternate between a few leaves, which one slot cannot serve at all, and a descent refreshes the slot of a leaf that is remembered already rather than spend a second one on the same page. A slot owns its key buffers and grows them only when a longer key arrives, so a row allocates nothing. The used-slot count and the miss counter are reset per statement in ha_innobase::reset(), matching autoinc_last_value. A lookup first compares its key against the remembered ranges, so an uncorrelated scan settles its misses in memory, with no buffer pool access and no pages_accessed. Only a covering range is probed, through the new btr_cur_t::try_leaf_hint(), which acquires the page with buf_page_try_get(): a hint is never derived from a latched parent page, so by the time it is tried it may precede the caller's already-latched secondary-index leaf in the latching order, where a blocking wait can deadlock. A stale range costs a wasted probe or a needless descent, never a wrong result, because the checks that try_leaf_hint() makes on the latched page remain the sole authority, and the ranges therefore need no invalidation protocol. After CLUST_LEAF_HINT_MAX_MISSES (8) consecutive unanswered lookups, a scan gives the slots up: row_sel_clust_leaf_hint_armed() stops both the test of the slots and the copies that refresh them, which are the larger half of their cost. One lookup in CLUST_LEAF_HINT_RETRY (1024) starts the count again, so a scan whose order becomes correlated only later recovers, and the trial that this begins refreshes the slots as it goes. The run is short because a hit saves little where the pages above the leaf are resident: one buffer pool access and one page-local search for each level. Measured against the same tree built without the hints, at 16k with a resident working set and no adaptive hash index, a wholly correlated scan runs a quarter faster over half the page accesses, a scan that answers three lookups in five runs level with it over 30% fewer, one whose locality appears only half way through runs an eighth faster over a quarter fewer, and a scan that answers nothing stays within the noise. A run of 8 is what keeps that last one there. A clustered index small enough to stay in cache is the exception that the run does not catch: it answers often, so the count never builds, and it saves nothing, because the descent that a hit replaces costs almost nothing there. Such a scan pays about a tenth. Where the adaptive hash index is enabled, the hints are neither used nor collected: its guess solves the same problem better, landing on the record with no page-local search and no page access to charge. It is off by default, so the hints are active in a default configuration. innodb.non_covering_sec_idx_scan measures pages_accessed over key orders that differ in how closely the secondary order tracks the clustered one, and seven further tables check query results over the record formats and key shapes that a clustered-index lookup has to read, down to the metadata pseudo-record of instant ALTER TABLE and to leaves that split and merge while a locking read walks them. non_covering_sec_idx_scan_debug runs the same body with the hints turned off, through a debug switch that returns before a lookup tests or refreshes the slots, so a diff of the two .result files is what the hints save: 2006 to 1028 (2-level clustered index), 2291 to 1007 (3-level), 20020 to 15780 (decorrelated), 4006 to 2012 (two interleaved key ranges), 20020 to 20006 (shuffled) and 12016 to 9289 (locality in the second half alone). The last of those pins the retry: without it the count is 11934. main.rowid_filter_innodb: 90 to 81, and its ahi combination unchanged. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| windows | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Khaled Riyad
khaled57.dev@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40551 Copy/Paste friendly output format for MariaDB Command Line Client Copy/paste friendly output was only reachable by starting the client with --silent --skip-column-names, which cannot be done from a running interactive session. Add \S, a statement terminator which prints the result of one statement in the tab separated format without column names. com_silent() sets output_plain, opt_silent and column_names around com_go(), then restores them, the same way com_ego() handles vertical. output_plain selects print_tab_data() ahead of the vertical and table branches, so \S gives the same output whether the session was started plainly or with --table, --vertical or --silent. --html and --xml still win, matching \G. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39949 Add per-table "SQL_CACHE" option Add SQL_CACHE=0|1 optioon for tables to disable/enable query caches if table is part of the query. SQL_CACHE option can be disable for a table by using SQL_CACHE=DEFAULT Also provide a workaround for the problem described at: MDEV-6631 "No cache directive(SQL_CACHE) query still wait for query cache lock" on query_cache_type=DEMAND mode" by adding DEMAND_STRICT mode. The new mode is needed as modifying how DEMAND works would made the change backward incompatible. Query_cache_type has now following options (0-2 are same as before) OFF (0) = Query cache off ON (1) = Query cache on, all queries are cached except queries marked with SQL_NO_CACHE DEMAND (2) = Query cache on, only queries marked with SQL_CACHE are cached DEMAND_STRICT (3) = Query cache on, only queries starting with SELECT SQL_CACHE are cached. This avoids a query cache lock+lookup for every query. TABLES (4) = Query cache on, only cache queries marked with SQL_CACHE or where all used tables are using the create option SQL_CACHE=1 Note that if any of the tables used in the query has the create option SQL_CACHE=0 or is a system table, temporary table or other non cacheable table, then the query will not be cached. SHOW CREATE TABLE prints SQL_CACHE=0|1 when the option is set. For tables where the option can never apply (temporary tables, tables in the mysql database, and engines that do not support the query cache) it is printed as /* SQL_CACHE=1 */. co-author: Seunguck Lee - Took some changes related to MDEV-6631 from his patch. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| bintar plugin names | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Added proper cleanup of main.cte_update_delete.test | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alessandro Vetere
iminelink@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-32286 Reuse remembered clustered leaves in secondary-index scans Row_sel_get_clust_rec_for_mysql::operator() descends the clustered B-tree from the root for every row whose clustered-index record a secondary-index scan must read, although consecutive rows often land on the same clustered leaf page. A non-covering scan needs one for every row, a locking read needs one whatever the secondary index holds, because an exclusive select lock type makes ha_innobase::build_template() build its template against the clustered index, and a covering scan needs one for every row of a secondary leaf whose PAGE_MAX_TRX_ID its read view cannot see. ANALYZE FORMAT=JSON charges each descent its full height, and those descents are nearly the whole cost: the secondary index is charged its own descent and one page for each further leaf, and nothing per row, because the position that its cursor holds between two rows is restored optimistically, which latches the leaf again without counting an access. The range estimate that ha_innobase::records_in_range() makes for the optimizer is charged over the same leaves, once more. So pages_accessed is the row count times the height of the clustered index, plus that handful. 1000 rows over a 2-level clustered index cost 2006, of which 3 are the scan of the secondary index and 3 the range estimate, and 750 rows over a 3-level one cost 2291, of which 27 and 14, where a full table scan of the same data costs 23 and 110. Remember, in the new row_prebuilt_t::clust_leaf_hint, the CLUST_LEAF_HINT_SLOTS (4) clustered leaves that the lookups of this statement reached, most recently used first. Each slot names one leaf: its page number, copies of its first and last user record truncated to the key fields, which bound the key range the leaf held when it was remembered, and the rec_get_offsets() of both. The copies are needed because the page is unlatched between two lookups, and the offsets spare a lookup the parsing of them. Several slots serve the scans that alternate between a few leaves, which one slot cannot serve at all, and a descent refreshes the slot of a leaf that is remembered already rather than spend a second one on the same page. A slot owns its key buffers and grows them only when a longer key arrives, so a row allocates nothing. The used-slot count and the miss counter are reset per statement in ha_innobase::reset(), matching autoinc_last_value. A lookup first compares its key against the remembered ranges, so an uncorrelated scan settles its misses in memory, with no buffer pool access and no pages_accessed. Only a covering range is probed, through the new btr_cur_t::try_leaf_hint(), which acquires the page with buf_page_try_get(): a hint is never derived from a latched parent page, so by the time it is tried it may precede the caller's already-latched secondary-index leaf in the latching order, where a blocking wait can deadlock. A stale range costs a wasted probe or a needless descent, never a wrong result, because the checks that try_leaf_hint() makes on the latched page remain the sole authority, and the ranges therefore need no invalidation protocol. After CLUST_LEAF_HINT_MAX_MISSES (8) consecutive unanswered lookups, a scan gives the slots up: row_sel_clust_leaf_hint_armed() stops both the test of the slots and the copies that refresh them, which are the larger half of their cost. One lookup in CLUST_LEAF_HINT_RETRY (1024) starts the count again, so a scan whose order becomes correlated only later recovers, and the trial that this begins refreshes the slots as it goes. The run is short because a hit saves little where the pages above the leaf are resident: one buffer pool access and one page-local search for each level. Measured against the same tree built without the hints, at 16k with a resident working set and no adaptive hash index, a wholly correlated scan runs a quarter faster over half the page accesses, a scan that answers three lookups in five runs level with it over 30% fewer, one whose locality appears only half way through runs an eighth faster over a quarter fewer, and a scan that answers nothing stays within the noise. A run of 8 is what keeps that last one there. A clustered index small enough to stay in cache is the exception that the run does not catch: it answers often, so the count never builds, and it saves nothing, because the descent that a hit replaces costs almost nothing there. Such a scan pays about a tenth. Where the adaptive hash index is enabled, the hints are neither used nor collected: its guess solves the same problem better, landing on the record with no page-local search and no page access to charge. It is off by default, so the hints are active in a default configuration. innodb.non_covering_sec_idx_scan measures pages_accessed over key orders that differ in how closely the secondary order tracks the clustered one, and eight further tables check query results over the record formats and key shapes that a clustered-index lookup has to read, down to the metadata pseudo-record of instant ALTER TABLE, to leaves that split and merge while a locking read walks them, and to a record that a remembered leaf supplies for a scan that must then rebuild an older version of it. non_covering_sec_idx_scan_debug runs the same body with the hints turned off, through a debug switch that returns before a lookup tests or refreshes the slots, so a diff of the two .result files is what the hints save: 2006 to 1028 (2-level clustered index), 2291 to 1007 (3-level), 20020 to 15780 (decorrelated), 4006 to 2012 (two interleaved key ranges), 20020 to 20006 (shuffled), 12016 to 9289 (locality in the second half alone) and 20020 to 10042 for a covering scan that FOR UPDATE makes non-covering, where the same scan without FOR UPDATE costs 20 in both files. The locality count pins the retry: without it the scan would keep only the 82 hits it makes before it gives up, and the count would be 11934. main.rowid_filter_innodb: 90 to 81, and its ahi combination unchanged. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-34805 post-review fixes * keep `vec_len >= subdist_part * 2` logic in one place only * keep "distance-greater-than" mode logic in one place only * simplify VECTOR_DIMENSIONS (no need to have a special ctx->vec_len path if the other one always works) * new plugin = maturity beta * remove redundant casts, etc * moved vector_indexes_fields_enum to the global scope to use it for setting schema->idx_field1/schema->idx_field2 * open the hlindex graph table, if needed, otherwise most values are unknown unless a user did vector search before * added TABLE_CATALOG column * remove CACHE_OVERFLOWS column, doesn't work as implemented, the fix is complex and isn't worth it * add privilege checks (MDEV-40793) in the test: * prefer query_vertical for readability * select all columns at least once * select INDEX_SIZE even if engine-dependent, use rdiff files * test how get_all_tables only open one specific table, and even only .frm file, if possible |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-33463 Add an option to truncate excessively long queries in the slow log | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
InnoDB review changes table_name_t::is_create_or_replace(): A new predicate to check for CREATE OR REPLACE TABLE will rename an old table to and eventually drop after creating the replacement. dict_table_t::parse_name(): Do acquire MDL on #sql-create- names for partitioned tables. dict_table_rename_in_cache(): On CREATE OR REPLACE TABLE ... SELECT, forget the original dict_table_t::mdl_name so that purge will acquire MDL on the #sql-create- name instead. In this way, the MDL_EXCLUSIVE that the CREATE OR REPLACE TABLE holds on the user-visible name will not unnecessarily block any purge of old history until the very end when the #sql-create- table will be dropped. ha_innobase::delete_table(): Do not check FOREIGN KEY consistency when dropping an #sql-create- table. row_rename_table_for_mysql(): Update SYS_FOREIGN.ID also when renaming to #sql-create- in order to avoid any duplicate key error when CREATE OR REPLACE TABLE is creating some FOREIGN KEY constraints by names that existed in the old table. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Georg Richter
georg@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
CONC-850: Fix out-of-bounds reads in parse_server_packet (GSSAPI) Replace strncpy/strnlen parsing in plugins/auth/auth_gssapi_client.c with bounded memchr/memcpy calls to prevent two security vulnerabilities: 1. Stack Buffer Over-Read: Oversized SPN inputs without a NUL byte caused strncpy to omit NUL-termination, leading to OOB reads in strlen(). 2. Packet Over-Read: Malformed packets missing a NUL terminator caused mechanism parsing to read past the end of the packet buffer. Thanks to Aisle Research for reporting this issue. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
squash! e61cabc0a2ae1a58c401fc24bb9f25caab6dc9b5 Table_specification_st::end_create_table(): If or_replace(), acquire MDL_EXCLUSIVE on the backup table name to keep the InnoDB purge out while we are dropping the table. (To prevent MDEV-36493 we must defer the acquisition until it is really necessary.) |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Speed up the query_cache interface - Do checks inline before calling query_cache.send_result_to_client() Things tried that failed: - I tried removing calling lex_start() and reset_for_next_command() if query is cached, but this is would cause duplicated, hard to maintain code and would require double resets of things if query cache was not used. I have documented in sql_parse.cc what would need to be reset for this approach to work. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Revert all InnoDB changes | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39491 Parallel Query: distribute parallel work more evenly Lower SPLIT_THRESHOLD from 3 to 2, enabling re-partitioning of chunks for shallower trees (one root node, one or more levels of internal nodes and a level of leaf nodes). Improve the detection of amount of work left: empty run queue does not always mean the scan is over, it might be that a context pulled for re-splitting has not enqueued its sub-contexts yet. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Georg Richter
georg@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
CONC-842: Fix incomplete source bounds check and state desync in mthd_my_read_rows Commit 85c322f70e47 introduced source buffer bounds validation against end_cp, but contained edge-case flaws allowing heap buffer over-reads and data leaks: 1. Pointer Underflow: net_field_length() advances 'cp'. If 'cp' advances past 'end_cp', (end_cp - cp) underflows into a large positive unsigned integer, causing 'len > (ulong)(end_cp - cp)' to evaluate to false and bypass memcpy bounds enforcement. Fix this by explicitly checking 'cp > end_cp'. 2. State Desynchronization: When 'cp >= end_cp', the previous fix set remaining field pointers to NULL and continued loop execution. On truncated or malformed packets, completing the row allows corrupted or stale heap data from previous queries to be returned. Fix this by failing immediately with CR_MALFORMED_PACKET when 'cp >= end_cp'. 3. EOF Status Read Bounds: Added validation before reading warning_count and server_status from EOF status packets to prevent out-of-bounds reads on short EOF frames. Thanks to fg0x0 for analyzing and reporting the guard bypass. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39491 Parallel Query: distribute parallel work more evenly Lower SPLIT_THRESHOLD from 3 to 2, enabling re-partitioning of chunks for shallower trees (one root node, one or more levels of internal nodes and a level of leaf nodes). Improve the detection of amount of work left: empty run queue does not always mean the scan is over, it might be that a context pulled for re-splitting has not enqueued its sub-contexts yet. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40656 Bypass REVOKE DENY ... FROM PUBLIC privilege check. DENY ... TO PUBLIC denies everyone, including whoever tries to revoke it, via the "deny wins" merge at every scope (global, db, table, column, routine). Allow REVOKE DENY ... FROM PUBLIC when the revoker can UPDATE mysql.global_priv (same as hand-editing). Assisted-by: Claude:claude-5-sonnet |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Georgi Kodinov
joro@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39806: Add a per-push/per-pull-request checker for Markdown API docs Create a GitHub action into the MariaDB server tree to build the Markdown documentation using the cmake target provided by MDEV-39718. This workflow generates API documentation for the plugin using a Docker container packed with all the extra doxygen/moxygen tools. Fixed some more doxygen problems. Moved the repo to ghcr.io/mariadb/mariadb-doc-gen:latest. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexander Barkov
bar@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39518 Allow prepared statements in stored functions in assignment right hand Allowing prepared statements in stored functions when a stored function is used in an assignment right hand. Both DEFAULT clause of a variable initialization and the right side of the SET statement are supported: CREATE PROCEDURE p1() BEGIN -- case 1: DEFAULT clause DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK -- case 2: SP variable assignment statement DECLARE spvar2 INT; SET spvar2= f1_with_ps(); -- OK END; - Only assignments to SP variables works for now: * SET spvar= func_with_ps(); -- OK * SET @uvar= func_with_ps(); -- Error - Only bare function calls are supported for now. Using a function in an expression does not make it PS-safe yet: SET v= f1()+0; - The parser now does not reject PS statements in stored functions. PS applicability in stored functions is now detected at run time. Note, PS statements in triggers are still prohibited by the parser. - Functions with PS do not acquire MDL locks on tables, and no MDL is taken on the routines themselves either. They work like procedures in terms of table opening and routine locking: a concurrent DROP FUNCTION can complete while such a function is executing. - Functions with PS are not replicated as a single `SELECT f1()` call. They are replicated per-statement, like procedures. Helper changes: - Changing the return result for LEX::sp_variable_declarations_init() from void to bool to catch errors in the caller properly. Misc: - This patch incorporates fixes for the following bugs found during debugging: MDEV-39518,MDEV-40224,MDEV-40225,MDEV-40226,MDEV-40227,MDEV-40240, MDEV-40285,MDEV-40288,MDEV-40315,MDEV-40318,MDEV-40890,MDEV-40900, MDEV-40901,MDEV-40913,MDEV-40914 Assisted-by: Claude - reviews and minor clean-ups |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Ahmad
ahmedshapan913@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-34805 provide various information about vector indexes Adds INFORMATION_SCHEMA.VECTOR_INDEXES table exposing statistics for MHNSW vector indexes via a MYSQL_INFORMATION_SCHEMA_PLUGIN registered alongside the existing mhnsw daemon plugin. columns covered: (TABLE_SCHEMA, TABLE_NAME, INDEX_NAME, VECTOR_DIMENSIONS, SUBDIST_ENABLED, INDEX_SIZE, TOTAL_NODES, DELETED_ROWS, MEMORY_SIZE, CACHE_OVERFLOWS). |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
monthdev
monthdevops@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20749 Improve mysqlbinlog --flashback error reporting Handle corrupted row events by reporting whether the field length could not be determined from metadata or whether the field extends past the row buffer instead of falling out through debug-only assertion paths or ad hoc exits during flashback row conversion. Propagate flashback conversion failures through the normal mysqlbinlog error path so debug builds produce the intended diagnostics cleanly. Add replication tests covering corrupted BLOB metadata and corrupted BLOB length prefixes using server-side debug injection to write broken binlog contents and mysqltest-friendly mysqlbinlog invocation patterns. Reviewed-by: Brandon Nesterenko <[email protected]> Reviewed-by: Georgi Kodinov <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge main | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thirunarayanan Balathandayuthapani
thiru@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-19574: innodb_stats_method is not honored when innodb_stats_persistent=ON Problem: ======= When persistent statistics are enabled (innodb_stats_persistent=ON), the innodb_stats_method setting is not properly utilized during statistics calculation. The statistics collection functions always use a hardcoded default behavior for NULL value comparison instead of respecting the configured stats method. This affects the accuracy of n_diff_key_vals (distinct key count), particularly for indexes with nullable columns containing NULL values. Moreover, stat_n_non_null_key_vals[] was never computed for persistent statistics; it stayed at the 0 that dict_stats_empty_index() assigns. With innodb_stats_method=nulls_ignored, innodb_rec_per_key() therefore always found n_diff <= n_null and reported one record per key for every index. This impacts the query optimizer, which makes decisions based on inaccurate cardinality estimates. Solution: ======== Introduced IndexLevelStats to collect statistics at a specific B-tree level during index analysis. Introduced PageStats to collect statistics for leaf page analysis. Refactored the following functions: dict_stats_analyze_index_level() to IndexLevelStats::analyze_level() dict_stats_analyze_index_for_n_prefix() to IndexLevelStats::sample_leaf_pages() dict_stats_analyze_index_below_cur() to PageStats::scan_below() dict_stats_scan_page() to PageStats::scan() The innodb_stats_method value is read once per table in dict_stats_update_persistent() and passed down, so that all indexes of a table are analyzed with the same method. Add the stats method name to stat_description when innodb_stats_method has a non-default value. The suffix is dropped when the description is already full. Added the new stat name n_nonnull_fld01, n_nonnull_fld02, etc. with a stats description, to indicate how many non-null values exist for the nth field of the index. This value is retrieved and stored in the index statistics in dict_stats_fetch_index_stats_step(). The counts are per column, not per n-column prefix. rec_get_n_blob_pages(): Calculate the number of externally stored pages for a record, using ceiling division by the usable BLOB page payload (blob_part_size), which differs between ROW_FORMAT=COMPRESSED (zip_size minus FIL_PAGE_DATA) and the other formats (srv_page_size minus the BLOB header and the page trailer). For ROW_FORMAT=COMPRESSED the length in the field reference is the uncompressed length, so the result is an upper bound. When the leaf level is scanned in full, the number of leaf pages that were scanned is reported as n_leaf_pages for a multi level index. Before, result.n_leaf_pages was overwritten with index->stat_n_leaf_pages, which dict_stats_empty_index() had just set to 1, so every index that took the full scan path reported n_leaf_pages=1. Single page indexes report 1. This changes cardinality estimates and therefore leads to multiple changes in existing test cases. Non-null values are counted only at the leaf level, since only leaf pages hold actual records. A full scan of the leaf level counts them exactly. When the level is sampled, the per column count is derived from the sampled leaves with the same formula as n_diff: n_ordinary_leaf_pages * n_non_null_all_analyzed_pages / n_leaf_pages_to_analyze This is an estimate for NOT NULL columns as well: the sampled leaves may hold fewer or more records than the average, and a dive that stops at a boring page contributes nothing to the sum while still counting in the divisor. innodb_rec_per_key(): stat_n_non_null_key_vals[i] holds the number of records in which the i-th indexed column alone is not NULL, while what has to be excluded here is the number of records whose first i+1 columns are all not NULL, because that is the population which the n-column prefix statistic stat_n_diff_key_vals[i] has to be corrected against when innodb_stats_method=nulls_ignored: with NULLs compared as unequal, every record carrying a NULL anywhere in the prefix adds a distinct value of its own to n_diff. PageStats::scan(): n_non_null is accumulated and assigned only for leaf pages, so that a non-leaf scan cannot leave a node pointer count behind when scan_below() stops at a boring page without reaching a leaf. IndexLevelStats::reset_for_level() also clears n_diff[], and dict_stats_analyze_index() zero initializes the buffer backing it, so that a level scan which finds no records (a failed btr_pcur_open_level(), or a non-leaf page whose first record is not marked as the leftmost one on the level) leaves n_diff[] at 0 instead of stale values. IndexLevelStats::sample_leaf_pages() returns early when the group boundaries for the prefix are empty, which is the same condition. IndexLevelStats::analyze_level(): Instead of copying the last record of the page, retain the latch on the page until the record has been compared with the first record of the next page dict_stats_fetch_index_stats_step() no longer resets stat_n_non_null_key_vals[] while processing an n_diff_pfxNN row: dict_stats_empty_table() has already cleared the array before the fetch, and with n_nonnull_fldNN rows now being read too, that reset would make the result depend on the order in which the rows arrive. dict_stats_save(): now static function in dict0stats.cc that takes the innodb_stats_method value, and is removed from dict0stats.h. dict_stats_update_persistent() saves the statistics itself, so its callers no longer have to. Replaced btr_rec_get_externally_stored_len() with rec_get_n_blob_pages() in dict0stats.cc. btr_rec_get_field_ref_offs() and btr_rec_get_field_ref(), together with the BTR_BLOB_HDR_* macros, were moved from btr0cur.cc to btr0cur.h so that rec_get_n_blob_pages() can reuse them; btr_rec_get_field_ref_offs() is now a noexcept function returning size_t. Changed stat_n_diff_key_vals and stat_n_non_null_key_vals from ib_uint64_t* to uint64_t* len_is_stored(): simplified to a single comparison, which is equivalent for the unsigned lengths that it is used with. Removed the unused UNIV_STATS_DEBUG build macro (univ.i) and turned the DEBUG_PRINTF() helper in dict0stats.cc into an unconditional no-op |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
cleanup: remove buggy str2int, replace with a template remove one str->int implementation (we still have 10+ more), replace with a convenience template that calls my_strntoll_8bit() |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| remove checks for strto[u][l]l family | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alessandro Vetere
iminelink@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-32286 Reuse remembered clustered leaves in secondary-index scans Row_sel_get_clust_rec_for_mysql::operator() descends the clustered B-tree from the root for every row whose clustered-index record a secondary-index scan must read, although consecutive rows often land on the same clustered leaf page. A non-covering scan needs one for every row, a locking read needs one whatever the secondary index holds, because an exclusive select lock type makes ha_innobase::build_template() build its template against the clustered index, and a covering scan needs one for every row of a secondary leaf whose PAGE_MAX_TRX_ID its read view cannot see. ANALYZE FORMAT=JSON charges each descent its full height, and those descents are nearly the whole cost: the secondary index is charged its own descent and one page for each further leaf, and nothing per row, because the position that its cursor holds between two rows is restored optimistically, which latches the leaf again without counting an access. The range estimate that ha_innobase::records_in_range() makes for the optimizer is charged over the same leaves, once more. So pages_accessed is the row count times the height of the clustered index, plus that handful. 1000 rows over a 2-level clustered index cost 2006, of which 3 are the scan of the secondary index and 3 the range estimate, and 750 rows over a 3-level one cost 2291, of which 27 and 14, where a full table scan of the same data costs 23 and 110. Remember, in the new row_prebuilt_t::clust_leaf_hint, the CLUST_LEAF_HINT_SLOTS (4) clustered leaves that the lookups of this statement reached, most recently used first. Each slot names one leaf: its page number, copies of its first and last user record truncated to the key fields, which bound the key range the leaf held when it was remembered, and the rec_get_offsets() of both. The copies are needed because the page is unlatched between two lookups, and the offsets spare a lookup the parsing of them. Several slots serve the scans that alternate between a few leaves, which one slot cannot serve at all, and a descent refreshes the slot of a leaf that is remembered already rather than spend a second one on the same page. A slot owns its key buffers and grows them only when a longer key arrives, so a row allocates nothing. The used-slot count and the miss counter are reset per statement in ha_innobase::reset(), matching autoinc_last_value. A lookup first compares its key against the remembered ranges, so an uncorrelated scan settles its misses in memory, with no buffer pool access and no pages_accessed. Only a covering range is probed, through the new btr_cur_t::try_leaf_hint(), which acquires the page with buf_page_try_get(): a hint is never derived from a latched parent page, so by the time it is tried it may precede the caller's already-latched secondary-index leaf in the latching order, where a blocking wait can deadlock. A stale range costs a wasted probe or a needless descent, never a wrong result, because the checks that try_leaf_hint() makes on the latched page remain the sole authority, and the ranges therefore need no invalidation protocol. After CLUST_LEAF_HINT_MAX_MISSES (8) consecutive unanswered lookups, a scan gives the slots up: row_sel_clust_leaf_hint_armed() stops both the test of the slots and the copies that refresh them, which are the larger half of their cost. One lookup in CLUST_LEAF_HINT_RETRY (1024) starts the count again, so a scan whose order becomes correlated only later recovers, and the trial that this begins refreshes the slots as it goes. The run is short because a hit saves little where the pages above the leaf are resident: one buffer pool access and one page-local search for each level. Measured against the same tree built without the hints, at 16k with a resident working set and no adaptive hash index, a wholly correlated scan runs a quarter faster over half the page accesses, a scan that answers three lookups in five runs level with it over 30% fewer, one whose locality appears only half way through runs an eighth faster over a quarter fewer, and a scan that answers nothing stays within the noise. A run of 8 is what keeps that last one there. A clustered index small enough to stay in cache is the exception that the run does not catch: it answers often, so the count never builds, and it saves nothing, because the descent that a hit replaces costs almost nothing there. Such a scan pays about a tenth. Where the adaptive hash index is enabled, the hints are neither used nor collected: its guess solves the same problem better, landing on the record with no page-local search and no page access to charge. It is off by default, so the hints are active in a default configuration. innodb.non_covering_sec_idx_scan measures pages_accessed over key orders that differ in how closely the secondary order tracks the clustered one, and eight further tables check query results over the record formats and key shapes that a clustered-index lookup has to read, down to the metadata pseudo-record of instant ALTER TABLE, to leaves that split and merge while a locking read walks them, and to a record that a remembered leaf supplies for a scan that must then rebuild an older version of it. non_covering_sec_idx_scan_debug runs the same body with the hints turned off, through a debug switch that returns before a lookup tests or refreshes the slots, so a diff of the two .result files is what the hints save: 2006 to 1028 (2-level clustered index), 2291 to 1007 (3-level), 20020 to 15780 (decorrelated), 4006 to 2012 (two interleaved key ranges), 20020 to 20006 (shuffled), 12016 to 9289 (locality in the second half alone) and 20020 to 10042 for a covering scan that FOR UPDATE makes non-covering, where the same scan without FOR UPDATE costs 20 in both files. The locality count pins the retry: without it the scan would keep only the 82 hits it makes before it gives up, and the count would be 11934. main.rowid_filter_innodb: 90 to 81, and its ahi combination unchanged. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||