Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Dave Gosselin
MDEV-39870: macOSB Build Warning on Typecast

On THD::reset, typecast the result of query_start_sec_part() to
tv_usec in a more portable way.  Previously, we assumed that the
left hand side was always long-compatible but that isn't the case
on mac.
Dave Gosselin
MDEV-39856: innochecksum help never stops printing whitespace

The help printer wraps each option description at spaces so it fits a
fixed column.  When a single word is wider than that column there is
no space to break on, so it stops advancing and prints blank,
indented lines without end.  Running innochecksum with no arguments
showed this after a long documentation link was placed in an option
description, but any tool with a similar description is affected.

When a word does not fit the column, print it whole on its own line
instead of looping.  If it is the last word in the description, leave
it for the final write so the help does not end on a blank line.
Brandon Nesterenko
MDEV-38796 Remove silent option from mysql_create_db_internal

TODO addressed
    - Remove 'silent' option from mysql_create_db_internal and instead use
      tmp_disable_binlog() / reenable_binlog()
Bill Jin
MDEV-39548 Further cleanup of MDL_request boilerplate

- Change thd->backup_commit_lock from MDL_request* to MDL_ticket*,
  and convert related call sites in handler.cc, log.cc, sql_class.cc,
  and xa.cc to use MDL_ACQUIRE_LOCK.

- Convert reload_acl_and_cache() in sql_reload.cc, purge_master_logs()
  and reset_master() in sql_repl.cc to MDL_ACQUIRE_LOCK, holding the
  ticket in a local MDL_ticket* (these functions used a local
  MDL_request originally and should not touch thd->backup_commit_lock).

- Convert acquire_lock() in partition_info.cc to MDL_ACQUIRE_LOCK.

- Add MDL_REQUEST_LIST_ADD() helper for enqueuing lock requests into
  MDL_request_list, and convert call sites in sql_base.cc and sp.cc.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
bsrikanth-mariadb
Fix opt_context_replay_basic failure due to sql_buffer_result addition
Yuchen Pei
MDEV-15621 Auto add RANGE COLUMNS partitions by interval

Allow auto partitioning by interval in PARTITION BY RANGE COLUMNS

PARTITION BY RANGE COLUMNS (col_name)
INTERVAL interval [AUTO]
(
  PARTITION partition_name VALUES LESS THAN (value)
  [, PARTITION partition_name VALUES LESS THAN (value) ... ]
)

where

- col_name is the name of one column of type DATE or DATETIME or
  TIMESTAMP

- at least one partition is supplied, and the highest partition cannot
  have MAXVALUE range

- INTERVAL interval is a positive time interval. it can be mariadb
  format or oracle NUMTODSINTERVAL/NUMTOYMINTERVAL format. Like
  versioning, the smallest unit is second, i.e. no subsecond like
  microsecond.

- DATE column cannot have interval with values less than a day

- Subpartitions are allowed, but restricted to existing subpartition
  types, i.e. [LINEAR] (KEY|HASH)

When performing one of the following DML statements on such a table,
it will first add partitions by the specified interval until the
partition covers the current time:

- INSERT
- INSERT ... SELECT
- LOAD
- UPDATE
- REPLACE
- REPLACE ... SELECT

Partition addition will not cause an implicit commit like DDL normally
does.

The partitions are named pN.

Otherwise the table behaves exactly the same as a normal RANGE COLUMNS
partitioned table.

Note that TIMESTAMP is not allowed as a type for PARTITION BY RANGE
COLUMNS otherwise.
Yuchen Pei
MDEV-15621 Auto add RANGE COLUMNS partitions by interval

Allow auto partitioning by interval in PARTITION BY RANGE COLUMNS

PARTITION BY RANGE COLUMNS (col_name)
INTERVAL interval [AUTO]
(
  PARTITION partition_name VALUES LESS THAN (value)
  [, PARTITION partition_name VALUES LESS THAN (value) ... ]
)

where

- col_name is the name of one column of type DATE or DATETIME or
  TIMESTAMP

- at least one partition is supplied, and the highest partition cannot
  have MAXVALUE range

- INTERVAL interval is a positive time interval. it can be mariadb
  format or oracle NUMTODSINTERVAL/NUMTOYMINTERVAL format. Like
  versioning, the smallest unit is second, i.e. no subsecond like
  microsecond.

- DATE column cannot have interval with values less than a day

- Subpartitions are allowed, but restricted to existing subpartition
  types, i.e. [LINEAR] (KEY|HASH)

When performing one of the following DML statements on such a table,
it will first add partitions by the specified interval until the
partition covers the current time:

- INSERT
- INSERT ... SELECT
- LOAD
- UPDATE
- REPLACE
- REPLACE ... SELECT

Partition addition will not cause an implicit commit like DDL normally
does.

The partitions are named pN.

Otherwise the table behaves exactly the same as a normal RANGE COLUMNS
partitioned table.

Note that TIMESTAMP is not allowed as a type for PARTITION BY RANGE
COLUMNS otherwise.
Alexey Botchkov
MDEV-37262 XMLISVALID() schema validation function.

XMLISVALID function added to the XMLTYPE plugin.
Brandon Nesterenko
MDEV-38796 Replace mysql_bin_log.is_open with binlog_ready()

Addresses TODO

    - Remove all testing of mysql_bin_log.is_open(). Instead test for
      binlog_ready() in main code and add testing of is_open() when
      trying to commit the binary log.  This is needed as currently
      mysql_bin_log.is_open() is tested without a mutex which makes
      it unreliable.

And also adds tests for the binlog closing for various types of
statements in-between a thd->binlog_ready() check and the commit-time
acquiring of mysql_bin_log.LOCK_log.

Still TODO from the above request:
* WSREP related mysql_bin_log.is_open() changes as a part of MDEV-38865
Marko Mäkelä
Fix the Windows glitch
Alexey Botchkov
MDEV-37262 XMLISVALID() schema validation function.

XMLISVALID function added to the XMLTYPE plugin.
Thirunarayanan Balathandayuthapani
MDEV-34358  Encryption threads consume CPU and deadlock with DROP TABLE/purge

Problem:
========
1. Encryption threads busy-wait when no work is available:

When reaching fil_system.space_list.end(), fil_crypt_return_iops() is called
with wake=true, causing pthread_cond_broadcast() to wake all threads
unnecessarily, leading to CPU waste.

2. Tablespaces with CLOSING/STOPPING flags skipped during iteration:

Since DDL completion doesn't wake encryption threads, these spaces may never
be encrypted if threads sleep indefinitely.

3. For default_encrypt_list iteration, when spaces exist but none are
acquirable, threads need to wake others for cooperative retry, but this
case was not distinguished from fil_system.space_list.end().

4. IOPS are allocated before searching for tablespaces, wasting resources
during iteration when no I/O occurs.

5. Encryption threads use fil_crypt_threads_cond for two different purposes:
waiting for encryption work and waiting for IOPS allocation. When
fil_crypt_return_iops() or fil_crypt_realloc_iops() broadcasts after
releasing IOPS, it wakes ALL threads including those correctly waiting
for work, causing spurious wakeups and CPU waste.

6. When innodb_encrypt_tables or innodb_encryption_rotate_key_age is changed
during encryption thread iteration, threads continue with stale configuration
values, potentially missing tablespaces that should be encrypted or rotated
under the new settings.

7. The InnoDB encryption thread could deadlock with DROP TABLE and purge
operations in a three-way deadlock scenario:

- DROP TABLE thread holds lock_sys.latch and waits for the tablespace
pending reference count to reach zero before dropping the space.

- Encryption thread holds a tablespace reference and waits to acquire
the tablespace allocation latch in exclusive mode to call
fseg_page_is_allocated() for checking if a page is allocated before
encrypting it.

- Purge coordinator thread holds the tablespace allocation latch
in exclusive mode and waits to acquire lock_sys.latch
in shared mode for record lock operations.

This creates a circular dependency and leads to deadlock.

Solution:
=========
1. Implement timed wait with exponential backoff:

When space == fil_system.space_list.end() (applies to both default_encrypt_list
and space_list iteration when no acquirable spaces are found):
- First timeout: 5 seconds
- Subsequent timeouts: (timed_wait_count + 1) * 5 seconds (10s, 20s, 40s, 60s)
- After 5 consecutive timeouts (~135 seconds total), continue with 60-second
timed waits to ensure threads periodically recheck for state changes
- Timeout counter resets to 0 when woken by signal or when work is found

2. Move IOPS allocation from before tablespace search to after finding a space
that needs rotation. If allocation fails, set recheck=true to skip waiting
and immediately try next space.

Encryption threads would hold space references while waiting in
fil_crypt_alloc_iops(), blocking DROP TABLE. To prevent this,
encyption should do release-wait-reacquire pattern.

fil_crypt_alloc_iops(): Added nowait parameter (default false). When
nowait=true, returns immediately if IOPS not available instead of waiting.

fil_crypt_thread(): Try non-blocking IOPS allocation first with nowait=true.
Only if IOPS not immediately available:
- Save space ID and release space reference
- Wait for IOPS with nowait=false
- Reacquire space using fil_space_get_by_id() and space->acquire()
- If space dropped or stopping, release IOPS and skip

This ensures encryption threads never hold space references while waiting
for IOPS, allowing DROP TABLE operations to proceed without deadlock.

3. Introduce separate condition variable fil_crypt_iops_cond specifically for
IOPS allocation synchronization to prevent spurious wakeups:

- fil_crypt_threads_cond: Used in wait_for_work() for waiting when no
tablespaces need encryption. Signaled when settings change, new
tablespaces are created, or thread count changes.

- fil_crypt_iops_cond: Used in fil_crypt_alloc_iops() for waiting
when IOPS limit is reached. Signaled when IOPS are returned via
fil_crypt_return_iops(), released via fil_crypt_realloc_iops(), or
when srv_n_fil_crypt_iops is increased.

4. Added atomic version counter fil_crypt_settings_version that is incremented
whenever innodb_encrypt_tables or innodb_encryption_rotate_key_age changes.
Encryption threads capture the version at iteration start and check for
changes during iteration. If config changed, threads immediately restart
iteration from the beginning to ensure complete coverage with new settings.

New fields:
- fil_crypt_settings_version: Atomic counter to track configuration changes
- rotate_thread_t::timed_wait_count (uint8_t): Counts consecutive timeouts
  for exponential backoff
- rotate_thread_t::wait_for_work(): Implements timed/indefinite wait strategy
- rotate_thread_t::settings_version: Compares with fil_crypt_settings_version
  to restart encryption from the beginning

5. Fix three-way deadlock with trylock mechanism:
fseg_page_is_allocated(): Added optional caller_mtr parameter. Encryption
thread passes an mtr, allocation bitmap page latch is now correctly held
in that mtr until the caller commits it.

fil_crypt_get_page_throttle(): Use fil_space_t::x_lock_try() to acquire
the tablespace allocation latch non-blockingly. If the trylock fails,
back off and retry the same page to avoid deadlock. The bitmap page
S-latch is added to the mtr and held throughout the page read and
encryption operations. Release the tablespace exclusive latch immediately
after the allocation check to minimize contention, while the bitmap page
S-latch remains held in the mtr.

fil_crypt_rotate_page(): After 10 retries on page latch acquisition,
release IOPS, sleep 10ms, then try to reacquire IOPS with nowait=true.
If IOPS not immediately available, exit gracefully by setting first=true.
This periodic IOPS release allows other encryption threads working on
tablespaces being dropped to make progress and release their space
references, helping to break the deadlock scenario.

rotate_state.aborted: set under crypt_data->mutex when a
thread bails in fil_crypt_rotate_pages(). Gate should_flush in
fil_crypt_complete_rotate_space() on !aborted so a partial pass
cannot commit min_key_version. Cleared when the next pass
initializes the fresh pass restarts.
Dave Gosselin
MDEV-39856: innochecksum help never stops printing whitespace

The help printer wraps each option description at spaces so it fits a
fixed column.  When a single word is wider than that column there is
no space to break on, so it stops advancing and prints blank,
indented lines without end.  Running innochecksum with no arguments
showed this after a long documentation link was placed in an option
description, but any tool with a similar description is affected.

When a word does not fit the column, print it whole on its own line
instead of looping.  If it is the last word in the description, leave
it for the final write so the help does not end on a blank line.
Thirunarayanan Balathandayuthapani
MDEV-34358  Encryption threads consume CPU and deadlock with DROP TABLE/purge

Problem:
========
1. Encryption threads busy-wait when no work is available:

When reaching fil_system.space_list.end(), fil_crypt_return_iops() is called
with wake=true, causing pthread_cond_broadcast() to wake all threads
unnecessarily, leading to CPU waste.

2. Tablespaces with CLOSING/STOPPING flags skipped during iteration:

Since DDL completion doesn't wake encryption threads, these spaces may never
be encrypted if threads sleep indefinitely.

3. For default_encrypt_list iteration, when spaces exist but none are
acquirable, threads need to wake others for cooperative retry, but this
case was not distinguished from fil_system.space_list.end().

4. IOPS are allocated before searching for tablespaces, wasting resources
during iteration when no I/O occurs.

5. Encryption threads use fil_crypt_threads_cond for two different purposes:
waiting for encryption work and waiting for IOPS allocation. When
fil_crypt_return_iops() or fil_crypt_realloc_iops() broadcasts after
releasing IOPS, it wakes ALL threads including those correctly waiting
for work, causing spurious wakeups and CPU waste.

6. When innodb_encrypt_tables or innodb_encryption_rotate_key_age is changed
during encryption thread iteration, threads continue with stale configuration
values, potentially missing tablespaces that should be encrypted or rotated
under the new settings.

7. The InnoDB encryption thread could deadlock with DROP TABLE and purge
operations in a three-way deadlock scenario:

- DROP TABLE thread holds lock_sys.latch and waits for the tablespace
pending reference count to reach zero before dropping the space.

- Encryption thread holds a tablespace reference and waits to acquire
the tablespace allocation latch in exclusive mode to call
fseg_page_is_allocated() for checking if a page is allocated before
encrypting it.

- Purge coordinator thread holds the tablespace allocation latch
in exclusive mode and waits to acquire lock_sys.latch
in shared mode for record lock operations.

This creates a circular dependency and leads to deadlock.

Solution:
=========
1. Implement timed wait with exponential backoff:

When space == fil_system.space_list.end() (applies to both default_encrypt_list
and space_list iteration when no acquirable spaces are found):
- First timeout: 5 seconds
- Subsequent timeouts: (timed_wait_count + 1) * 5 seconds (10s, 20s, 40s, 60s)
- After 5 consecutive timeouts (~135 seconds total), continue with 60-second
timed waits to ensure threads periodically recheck for state changes
- Timeout counter resets to 0 when woken by signal or when work is found

2. Move IOPS allocation from before tablespace search to after finding a space
that needs rotation. If allocation fails, set recheck=true to skip waiting
and immediately try next space.

Encryption threads would hold space references while waiting in
fil_crypt_alloc_iops(), blocking DROP TABLE. To prevent this,
encyption should do release-wait-reacquire pattern.

fil_crypt_alloc_iops(): Added nowait parameter (default false). When
nowait=true, returns immediately if IOPS not available instead of waiting.

fil_crypt_thread(): Try non-blocking IOPS allocation first with nowait=true.
Only if IOPS not immediately available:
- Save space ID and release space reference
- Wait for IOPS with nowait=false
- Reacquire space using fil_space_get_by_id() and space->acquire()
- If space dropped or stopping, release IOPS and skip

This ensures encryption threads never hold space references while waiting
for IOPS, allowing DROP TABLE operations to proceed without deadlock.

3. Introduce separate condition variable fil_crypt_iops_cond specifically for
IOPS allocation synchronization to prevent spurious wakeups:

- fil_crypt_threads_cond: Used in wait_for_work() for waiting when no
tablespaces need encryption. Signaled when settings change, new
tablespaces are created, or thread count changes.

- fil_crypt_iops_cond: Used in fil_crypt_alloc_iops() for waiting
when IOPS limit is reached. Signaled when IOPS are returned via
fil_crypt_return_iops(), released via fil_crypt_realloc_iops(), or
when srv_n_fil_crypt_iops is increased.

4. Added atomic version counter fil_crypt_settings_version that is incremented
whenever innodb_encrypt_tables or innodb_encryption_rotate_key_age changes.
Encryption threads capture the version at iteration start and check for
changes during iteration. If config changed, threads immediately restart
iteration from the beginning to ensure complete coverage with new settings.

New fields:
- fil_crypt_settings_version: Atomic counter to track configuration changes
- rotate_thread_t::timed_wait_count (uint8_t): Counts consecutive timeouts
  for exponential backoff
- rotate_thread_t::wait_for_work(): Implements timed/indefinite wait strategy
- rotate_thread_t::settings_version: Compares with fil_crypt_settings_version
  to restart encryption from the beginning

5. Fix three-way deadlock with trylock mechanism:
fseg_page_is_allocated(): Added optional caller_mtr parameter. Encryption
thread passes an mtr, allocation bitmap page latch is now correctly held
in that mtr until the caller commits it.

fil_crypt_get_page_throttle(): Use fil_space_t::x_lock_try() to acquire
the tablespace allocation latch non-blockingly. If the trylock fails,
back off and retry the same page to avoid deadlock. The bitmap page
S-latch is added to the mtr and held throughout the page read and
encryption operations. Release the tablespace exclusive latch immediately
after the allocation check to minimize contention, while the bitmap page
S-latch remains held in the mtr.

fil_crypt_rotate_page(): After 10 retries on page latch acquisition,
release IOPS, sleep 10ms, then try to reacquire IOPS with nowait=true.
If IOPS not immediately available, exit gracefully by setting first=true.
This periodic IOPS release allows other encryption threads working on
tablespaces being dropped to make progress and release their space
references, helping to break the deadlock scenario.

rotate_state.aborted: set under crypt_data->mutex when a
thread bails in fil_crypt_rotate_pages(). Gate should_flush in
fil_crypt_complete_rotate_space() on !aborted so a partial pass
cannot commit min_key_version. Cleared when the next pass
initializes the fresh pass restarts.
bsrikanth-mariadb
Add "sql_buffer_result" to the opt_related_sys_vars list

This fixes main.variables test
Sergei Petrunia
Make Optimizer Context include "subquery_runs": [ NNN, ...].

This shows that subquery(ies) were executed at the optimization phase.
There is no way to replay this, so we do not parse this back.

The idea is to allow "--replay-server" detect this and not try
replaying such queries.
Rex Johnston
MDEV-39495 Parallel Query:comments and code cleanup
Yuchen Pei
MDEV-15621 [refactor] Partitioning cleanup

change p_column_list_val::fixed to a bool
remove redundant end label in partition_info::fix_column_value_functions
Thirunarayanan Balathandayuthapani
MDEV-24813 Drop low-value table_lock test combinations

The three deadlock_*_race tests cannot reach their DEBUG_SYNC race
under a table lock (it degenerates to a timeout), and the three I_S
tests only restate a lock-mode change already covered
by innodb_full_scan.test.
Brandon Nesterenko
MDEV-38796 Remove *_binlog_local_stmt_filter()

TODO Addressed:
  - Remove clear/reset xxx_binlog_local_stmt_filter() as this is now handled
      by binlog_state.
Yuchen Pei
MDEV-15621 Auto add RANGE COLUMNS partitions by interval

Allow auto partitioning by interval in PARTITION BY RANGE COLUMNS

PARTITION BY RANGE COLUMNS (col_name)
INTERVAL interval [AUTO]
(
  PARTITION partition_name VALUES LESS THAN (value)
  [, PARTITION partition_name VALUES LESS THAN (value) ... ]
)

where

- col_name is the name of one column of type DATE or DATETIME or
  TIMESTAMP

- at least one partition is supplied, and the highest partition cannot
  have MAXVALUE range

- INTERVAL interval is a positive time interval. it can be mariadb
  format or oracle NUMTODSINTERVAL/NUMTOYMINTERVAL format. Like
  versioning, the smallest unit is second, i.e. no subsecond like
  microsecond.

- DATE column cannot have interval with values less than a day

- Subpartitions are allowed, but restricted to existing subpartition
  types, i.e. [LINEAR] (KEY|HASH)

When performing one of the following DML statements on such a table,
it will first add partitions by the specified interval until the
partition covers the current time:

- INSERT
- INSERT ... SELECT
- LOAD
- UPDATE
- REPLACE
- REPLACE ... SELECT

Partition addition will not cause an implicit commit like DDL normally
does.

The partitions are named pN.

Otherwise the table behaves exactly the same as a normal RANGE COLUMNS
partitioned table.

Note that TIMESTAMP is not allowed as a type for PARTITION BY RANGE
COLUMNS otherwise.
Yuchen Pei
MDEV-39789 Fix compiling without perfschema
Yuchen Pei
MDEV-24813 Signal full scan to storage engines.

When starting to do a full table/index scan without a WHERE or JOIN
condition, tell the storage engine so and the corresponding
ulong-truncated LIMIT.

Include an innodb implementation: added an innodb switch
table_lock_on_full_scan, so that when the switch is on, on receiving
the full scan signal from the sql layer, if the truncated LIMIT is
ULONG_MAX (likely no LIMIT), attempt to acquire a table lock.

Updated tests that have different results with the switch on.

(Comment and code edited by Sergei Petrunia)
Brandon Nesterenko
MDEV-38796 Update BINLOG_STATE_FILTER at DB change time

Addresses TODO

    - Update binlog_state with BINLOG_STATE_FILTER only when database or filter
      is changed.  This means we do not have to call binlog_filter->db_ok() for
      every statement in decide_binlog_format().
Alexey Botchkov
next
Dave Gosselin
MDEV-35291 derived_with_keys not applied for UNION with const filter

The optimizer fails to generate a derived key when a materialized
UNION derived table is filtered by a constant equality, as in:

  SELECT dt.* FROM
    ((SELECT * FROM t1 LIMIT 100) UNION
    (SELECT * FROM t1 LIMIT 200)) dt
  WHERE id = 1;

The condition cannot be pushed into the underlying SELECTs because
LIMIT inside each branch blocks pushdown, so the only remaining
optimization is to build a derived key on id and access via
ref(const).  But add_key_part() registers KEYUSE entries for the
materialized derived only when the value side of the equality
references another table.  That excludes constants, leaving the
outer query no choice but to scan the full materialized result.

Investigation showed that this is the same defect that Rex Johnston
already fixed under MDEV-39499 (see the bb-11.4-MDEV-39499 branch).
This commit ports the add_key_part() change from MDEV-39499 to 10.11.
Sergei Petrunia
Amend --replay-server code: Allocate replay server port based on MTR_BUILD_THREAD

This allows to run multiple mtrs with --replay-server on the same machine.
Marko Mäkelä
MDEV-39861 innodb_log_recovery_target wrongly opens log in read-write mode

log_t::archived_switch_recovery_prepare(): Observe recv_sys.rpo
similar to what recv_sys_t::find_checkpoint() does.
Brandon Nesterenko
MDEV-38796 Test updates for forced row logging

Test binlog_spurious_ddl_errors is updated here because now
decide_logging_format doesn't error out when it can't log in statement
format, but just switches to row
(aa4b22cbea472e23cd32b47b15308cafae919a2f).

@@ -7259,7 +7262,15 @@ int THD::decide_logging_format(TABLE_LIST *tables)
                              wsrep_cs().mode() ==
                              wsrep::client_state::m_local),1))
          {
-            my_error((error= ER_BINLOG_STMT_MODE_AND_ROW_ENGINE), MYF(0), "");
+            if (is_current_stmt_binlog_format_stmt())
+            {
+              my_printf_error(ER_BINLOG_STMT_MODE_AND_ROW_ENGINE,
+                              "Statement cannot be logged as STATEMENT as at "
+                              "least one table is limited to row-based logging."
+                              " Switching temporarly to row format",
+                              MYF(ME_NOTE));
+              set_current_stmt_binlog_format_row();
+            }
          }
bsrikanth-mariadb
Generate context for table less queries as well.

Fixes func_str, ctype_utf8, ctype_latin1 tests
Marko Mäkelä
MDEV-39861 innodb_log_recovery_target wrongly opens log in read-write mode

log_t::archived_switch_recovery_prepare(): Observe recv_sys.rpo
similar to what recv_sys_t::find_checkpoint() does.
bsrikanth-mariadb
Disable replay for main.user_var

As user variables are not yet supported for replay
Mohammad Tafzeel Shams
MDEV-38305: Expose adaptive hash index statistics in ANALYZE FORMAT=JSON

Expose InnoDB's Adaptive Hash Index (AHI) statistics through ANALYZE
FORMAT=JSON output and global status variables to provide query-level
and system-level visibility into AHI usage and effectiveness.
This allows DBAs and developers to monitor how well
the adaptive hash index is serving their workloads on a per-query basis.

The r_ahi_stats object (nested inside r_engine_stats) now reports four
key metrics: ahi_searches (successful AHI lookups), ahi_searches_btree
(AHI misses requiring B-tree fallback), ahi_rows_added (rows inserted
into AHI), and ahi_pages_added (pages indexed by AHI). These same
metrics are exposed globally via innodb_status_variables.

Transaction-local tracking eliminates contention by accumulating
statistics in trx_t fields (n_sea, n_non_sea, n_ahi_rows_added,
n_ahi_pages_added) and aggregating them into global atomic counters
(btr_cur_n_sea, btr_cur_n_non_sea, btr_ahi_n_rows_added,
btr_ahi_n_pages_added) during trx_t::commit_cleanup() and trx_t::free(),
following the pattern established for buf_pool.stat.n_page_gets.

- btr_ahi_inc_searches(): Increment trx->n_sea when AHI lookup succeeds.
- btr_ahi_inc_searches_btree(): Increment trx->n_non_sea when AHI lookup
  fails and falls back to B-tree search.
- btr_ahi_inc_rows_added(): Increment trx->n_ahi_rows_added when rows
  are added to the adaptive hash index structure.
- btr_ahi_inc_pages_added(): Increment trx->n_ahi_pages_added when new
  pages are indexed by AHI.
- btr_cur_t::search_leaf(): Call btr_ahi_inc_searches() on successful
  AHI hit and btr_ahi_inc_searches_btree() on AHI miss.
- btr_cur_n_sea, btr_cur_n_non_sea: Replace sharded ib_counter_t with simple
  atomic counters.
- MONITOR_OVLD_ADAPTIVE_HASH_ROW_ADDED, MONITOR_OVLD_ADAPTIVE_HASH_PAGE_ADDED:
  Convert monitor counters to "overload" type that read from global atomic
  counters btr_ahi_n_rows_added and btr_ahi_n_pages_added respectively instead
  of maintaining separate statistics.
- trace_engine_stats(): Output r_ahi_stats object with all four AHI
  counters in JSON format when any AHI activity is detected during query
  execution.
- ha_handler_stats: Added ahi_searches, ahi_searches_btree, ahi_rows_added,
  and ahi_pages_added fields to track per-query AHI statistics.
- Add mtr parameter to btr_search_move_or_delete_hash_entries(),
  btr_cur_t::search_info_update(), btr_search_update_hash_on_insert(),
  btr_search_update_hash_ref(), btr_search_info_update_hash(), and
  btr_search_build_page_hash_index() to allow updating AHI statistics.
- ahi_stats.test: Comprehensive verification of AHI statistics reporting
  across different scenarios: insufficient accesses (no AHI build),
  threshold triggering (AHI construction), heavy warmup (full AHI
  utilization), and disabled AHI (verify zero statistics).
- check_ahi_status.inc: Reusable include file for executing queries with
  configurable warmup repetitions and extracting AHI statistics from
  ANALYZE FORMAT=JSON output using JSON path expressions.
Sergei Petrunia
Do optimizer context recording for all SQL commands that have a query plan

... except for INSERT DELAYED which damages the tables making it
impossible to dump their definitions.
Sergei Petrunia
Fix mtr --replay-server main.select_found

Make show_create_table_ex/get_default_field_value() use
Field::real_maybe_null(), not maybe_null().

Before, the difference didn't matter as show_create_table_ex() was
only invoked in SHOW CREATE TABLE where columns are not made NULLable.

Optimizer context code invokes show_create_tableas show_create_table_ex()
in all queries, so now the distinction matters.
Vladislav Vaintroub
MDEV-14443 DENY statement

Implements DENY/REVOKE DENY and associated tasks.
Dave Gosselin
MDEV-35291 derived_with_keys not applied for UNION with const filter

The optimizer fails to generate a derived key when a materialized
UNION derived table is filtered by a constant equality, as in:

  SELECT dt.* FROM
    ((SELECT * FROM t1 LIMIT 100) UNION
    (SELECT * FROM t1 LIMIT 200)) dt
  WHERE id = 1;

The condition cannot be pushed into the underlying SELECTs because
LIMIT inside each branch blocks pushdown, so the only remaining
optimization is to build a derived key on id and access via
ref(const).  But add_key_part() registers KEYUSE entries for the
materialized derived only when the value side of the equality
references another table.  That excludes constants, leaving the
outer query no choice but to scan the full materialized result.

Investigation showed that this is the same defect that Rex Johnston
already fixed under MDEV-39499 (see the bb-11.4-MDEV-39499 branch).

This commit ports the add_key_part() change from MDEV-39499 to
10.11.  It does not port the rest of Rex's commit because it depends
on KEY::rec_per_key_null_aware() and the KEY::all_nulls_key_parts
bitmap populated by the rewritten sql_statistics code, neither of
which exists in 10.11.  But it turns out that doesn't matter for
this bug and the add_key_part() is sufficient to fix it.
Yuchen Pei
MDEV-15621 Auto add RANGE COLUMNS partitions by interval

Allow auto partitioning by interval in PARTITION BY RANGE COLUMNS

PARTITION BY RANGE COLUMNS (col_name)
INTERVAL interval [AUTO]
(
  PARTITION partition_name VALUES LESS THAN (value)
  [, PARTITION partition_name VALUES LESS THAN (value) ... ]
)

where

- col_name is the name of one column of type DATE or DATETIME or
  TIMESTAMP

- at least one partition is supplied, and the highest partition cannot
  have MAXVALUE range

- INTERVAL interval is a positive time interval. it can be mariadb
  format or oracle NUMTODSINTERVAL/NUMTOYMINTERVAL format. Like
  versioning, the smallest unit is second, i.e. no subsecond like
  microsecond.

- DATE column cannot have interval with values less than a day

- Subpartitions are allowed, but restricted to existing subpartition
  types, i.e. [LINEAR] (KEY|HASH)

When performing one of the following DML statements on such a table,
it will first add partitions by the specified interval until the
partition covers the current time:

- INSERT
- INSERT ... SELECT
- LOAD
- UPDATE
- REPLACE
- REPLACE ... SELECT

Partition addition will not cause an implicit commit like DDL normally
does.

The partitions are named pN.

Otherwise the table behaves exactly the same as a normal RANGE COLUMNS
partitioned table.

Note that TIMESTAMP is not allowed as a type for PARTITION BY RANGE
COLUMNS otherwise.
Sergei Petrunia
Make --replay-server support "Optimizer Context Stopword": subquery_runs

If a stopword is present in the optimizer context, we do not try to
replay it.