Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Yuchen Pei
MDEV-40467 [to-squash] Improve Y2038/2106 problem error message

When nearing the end times TIMESTAMP_MAX_VALUE, auto-partition
creation may encounter overflow when calculating new partition range.
This patch improves the message of such an overflow
Oleksandr Byelkin
new CC 3.3
Sergei Golubchik
MDEV-40571 insufficient validation of frm data when opening a table

numerous checks that the frm is valid, no OOB reads,
values make sense (number of keyparts not less than number of keys,
no keys means no keyparts, number of long unique fields is not larger than
number of fields, fields values in the record don't overlap and don't
go over record ends, and so on). most asserts were changed to if()'s
Brandon Nesterenko
MDEV-38796 Clean up THD::decide_logging_format() checks

Continue the binlog_state refactoring by moving the remaining per
statement binary log checks into the cached THD::binlog_state, and
make the checks for whether binary logging is enabled consistent.

Changes:

* Track the binlog_do_db/binlog_ignore_db filter result for the
  current default database in binlog_state as BINLOG_STATE_FILTER_DB
  (renamed from BINLOG_STATE_FILTER, as the flag applies only to
  database filtering of statement format logging). The flag is
  updated in THD::set_db() and THD::reset_db() when the default
  database changes, so decide_logging_format() no longer calls
  binlog_filter->db_ok() for every statement. thd_binlog_filter_ok()
  now reads the flag instead of re-running the filter.

* Remove the THD local statement filter (m_binlog_filter_state and
  reset/clear/set/get_binlog_local_stmt_filter()). binlog_state now
  carries this information through BINLOG_STATE_FILTER_DB.

* Remove the 'silent' parameter from mysql_create_db_internal().
  mysql_upgrade_db() now suppresses binary logging of the internal
  CREATE DATABASE with tmp_disable_binlog() / reenable_binlog().

* Replace the remaining tests of mysql_bin_log.is_open() in main code
  paths with thd->binlog_ready() / binlog_ready_no_wsrep(). is_open()
  was tested without holding a mutex, which made it unreliable; it is
  now checked under mysql_bin_log.LOCK_log when committing to the
  binary log. WSREP related is_open() usage is left for MDEV-38865.

New tests:

* binlog.binlog_close_during_commit exercises statements racing with
  a binary log close between the thd->binlog_ready() check and the
  commit time acquisition of mysql_bin_log.LOCK_log, using the new
  debug sync point commit_after_binlog_ready_before_LOCK_log.

* rpl.rpl_lsu_binlog_filter verifies that a slave configured with
  log_slave_updates and binlog_do_db/binlog_ignore_db filters does
  not binlog filtered transactions, using a three server chain
  topology.

Test updates: ER_BINLOG_STMT_MODE_AND_ROW_ENGINE is no longer an
error. The statement is switched to row format for its duration and
a note is raised instead. Tests relying on the old error are updated
accordingly.

Reviewed by: Monty <[email protected]>
Sergei Golubchik
BUG#39449066 Refactor performance schema OBJECT_INSTANCE_BEGIN columns

Fix for MariaDB 10.6
forkfun
MDEV-39566 fix status_by_thread crash on live thread-count change

PFS_table_context snapshots the live thread/user/host/account
count at scan start and again on restore (filesort's second
rnd_init). If the count changed between the two, m_map_size
mismatched and the server aborted.

Skip the wasted re-sample on restore, bound each table's scan by
the frozen snapshot instead of the container's live count.
Yuchen Pei
MDEV-15621 Auto add RANGE COLUMNS partitions by interval

Allow auto partitioning by interval in PARTITION BY RANGE COLUMNS

PARTITION BY RANGE COLUMNS (col_name)
INTERVAL interval [AUTO]
(
  PARTITION partition_name VALUES LESS THAN (value)
  [, PARTITION partition_name VALUES LESS THAN (value) ... ]
)

where

- col_name is the name of one column of type DATE or DATETIME or
  TIMESTAMP

- at least one partition is supplied, and the highest partition cannot
  have MAXVALUE range

- INTERVAL interval is a positive time interval. it can be mariadb
  format or oracle NUMTODSINTERVAL/NUMTOYMINTERVAL format. Like
  versioning, the smallest unit is second, i.e. no subsecond like
  microsecond.

- DATE column cannot have interval with values less than a day

- Subpartitions are allowed, but restricted to existing subpartition
  types, i.e. [LINEAR] (KEY|HASH)

When performing one of the following DML statements on such a table,
it will first add partitions by the specified interval until the
partition covers the current time:

- INSERT
- INSERT ... SELECT
- LOAD
- UPDATE
- REPLACE
- REPLACE ... SELECT

Partition addition will not cause an implicit commit like DDL normally
does.

The partitions are named pN.

Otherwise the table behaves exactly the same as a normal RANGE COLUMNS
partitioned table.

Note that TIMESTAMP is not allowed as a type for PARTITION BY RANGE
COLUMNS otherwise.

Including the following fixes:

1.

MDEV-40088 Disallow subqueries in INTERVAL clause in range interval auto partitioning

This is consistent with system time (versioning) partitioning. Also
consistent is that both allow expressions otherwise.

2.

MDEV-40048 Allow trigger and LOCK TABLES to work with range interval auto partitioning

When a range interval auto partitioned table is the target of a
trigger, the triggering statement is not necessarily one that would
cause the auto-creation of new partitions, so we need to account for
that.

Also added support for LOCK TABLES ... WRITE.

Improved tests coverage by adapting tests from versioning.partition.

3.

MDEV-40467 Improve Y2038/2106 problem error message

When nearing the end times TIMESTAMP_MAX_VALUE, auto-partition
creation may encounter overflow when calculating new partition range.
This patch improves the message of such an overflow
Marko Mäkelä
MDEV-14992 BACKUP SERVER

The following SQL statements will be introduced:

BACKUP SERVER TO '/path/to/directory' [ 1 CONCURRENT ];
BACKUP SERVER WITH [ 1 CONCURRENT ] 'command';

In place of the 1, any positive number of threads may be specified.
For the first variant, '/path/to' must exist and '/path/to/directory'
must not exist; that is where the backup will be written to.

For the second variant, 'command' must be the name of a script or
command that will be executed in a child process. The standard input
of that command will be in a format that is compatible with
GNU tar --format=oldgnu (and also BSD tar variants that are also part of
Microsoft Windows and Apple macOS). The command is expected to optionally
compress and encrypt the stream and redirect it to a file on a local or
a remote server. The BACKUP SERVER WITH will append an additional argument,
a positive base-ten number in ASCII, starting with 1, to identify the
current thread. In this way, each concurrent stream can write a separate
file.

The backup or the first stream will contain a file backup.cnf, which
includes parameters needed for restoring the backup. Currently,
these are innodb_log_recovery_start and innodb_log_recovery_target.
If innodb_log_recovery_target>0, InnoDB will be in read-only mode,
not allowing any writes to persistent files other than via the log
application.

To restore a streaming backup made with BACKUP SERVER WITH, an empty
directory needs to be created and all streams be extracted there using
the standard tar utility of the operating system, optionally after
undoing any encryption or compression that had been added by the
backup command. Then, the backup is prepared or MariaDB server started
up on the extracted directory, similar to as if the BACKUP SERVER TO
statement had been used.

Note: The parameter innodb_log_recovery_start in backup.cnf is
STRICTLY NECESSARY TO AVOID CORRUPTION! By default, InnoDB crash recovery
starts from the latest available log checkpoint. However, for restoring
a backup, recovery must start from the checkpoint that was the latest
when the backup was started. Starting recovery from a possible later
checkpoint will result in a corrupted database!

The following will be implemented separately:

MDEV-39061 mariadb-backup compatible wrapper script for BACKUP SERVER
MDEV-40163 Partial backup and restore
MDEV-39091 Back up ENGINE=RocksDB
MDEV-39092 Less blocking backup of ENGINE=Aria

The implementation introduces a basic driver Sql_cmd_backup,
storage engine interfaces, and basic copying of the storage engines
InnoDB, Aria, MyISAM, MERGE (MyISAM), Archive, CSV.

backup_target: A structured data type to represent a target directory.
On Microsoft Windows, we must use directory paths because there is
no variant of CopyFileEx() that would work on file handles.

backup_sink: Wraps a per-thread output stream as well as storage engine
specific context.

handlerton::backup_start(), handlerton::backup_end(): Invoked at the
start or end of a backup phase, in the thread that executes a
BACKUP SERVER statement.

handlerton::backup_step(): A backup step that can be invoked from
multiple threads concurrently, between the execution of the corresponding
handlerton::backup_start() and handlerton::backup_end() of the same
phase.

copy_entire_file(): A file copying service for POSIX systems.

copy_file(): A partial or sparse file-copying service for all systems.

backup_stream_append(): Equivalent to copy_file(), but appending to
a stream. On Linux, this uses sendfile(2), which assumes that the
source data will not be changed before the data has been consumed
from the pipe.

backup_stream_append_async(): A variant of backup_stream_append()
where the source file region is guaranteed to be immutable after the
call returns. We must not use Linux sendfile(2) for copying data files
that may be modified in place, because it could introduce a race
condition between a page write that runs concurrently with a child process
that is reading the data from the pipe.

InnoDB_backup::context: Backup context, attached to backup_sink
so that context can continue to exist between the time a
BACKUP SERVER releases all locks and another BACKUP SERVER starts
executing, with innodb_backup pointing to the new backup, while
the old backup is still being finished.

InnoDB_backup::queue: Collection of tablespace IDs and payload sizes
at the start of the backup. If any file is created or extended while
the backup is executing, we must have the corresponding write-ahead-log
entries that we are copying since the latest checkpoint that was
completed when the backup started. If any tablespaces are deleted
during the backup, we may or may not copy them, and the application
of a FILE_DELETE record will remove them. Similarly, FILE_RENAME
or FILE_CREATE records will take care of renaming or creating files
during recovery (applying the backed-up log).

fil_space_t::write_or_backup: Keep track of in-flight page writes and
pending backup operation. We must not allow them concurrently, because
that could lead into torn pages in the backup.

fil_space_t::backup_end: The first page number that is not being backed up
(by default 0, to indicate that no backup is in progress).

fil_space_t::BACKUP_BATCH_SIZE: The number of preceding pages that will be
covered by fil_space_t::backup_end. This is the unit of "page range locking"
during InnoDB backup.

log_sys.backup: Whether BACKUP SERVER is in progress. The purpose of this
is to make BACKUP SERVER prevent the concurrent execution of
SET GLOBAL innodb_log_archive=OFF or SET GLOBAL innodb_log_file_size
when innodb_log_archive=OFF.

log_sys.archived_checkpoint: Keep track of the earliest available
checkpoint, corresponding to log_sys.archived_lsn. This reflects
SET GLOBAL innodb_log_recovery_start (which is settable now), for
incremental backup.

buf_flush_list_space(): Check for concurrent backup before writing each
page. This is inefficient, but this function may be invoked from multiple
threads concurrently, and it cannot be changed easily, especially for
fil_crypt_thread().

fil_system.have_all_spaces: Whether all tablespace metadata is guaranteed
to be known. To speed up startup, InnoDB does not normally open
all tablespace files.
Sergei Golubchik
Revert "MDEV-39622 OBJECT_INSTANCE_BEGIN in P_S are unstable, difficult to compare"

Let's use MySQL's fix for compatibility

This reverts commit 11c41cd93d2c3732862ba043afd18508021440c0.
Alexey Botchkov
MDEV-39750 ExtractValue does not control recursion depth.

Stack exhaustive test shouldn't be ran with the ASAN/UBSAN.
Rex Johnston
MDEV-39492 Parallel Query: ship base-table columns, not projected values

A worker used to run the whole select-project query over its chunk and ship the
projected select list. It now ships the base-table columns the query reads, in
table order, and the manager copies each one back into the field it came from in
its own table instances. Then, instead of the manager choosing a terminal
function itself, it hands each row to the plan's own: the last real table's
next_select, called exactly as sub_select() would call it.

The result is that everything downstream of the driving scan reads what a serial
scan would have left in those records, with nothing redirected. That deletes
three mechanisms rather than adding one. The reference-array slice built over the
result table is gone, and HAVING resolves through the records instead. The
per-item projection list the manager sent is gone, and the select list is
evaluated by end_send() as usual. The redirect that pointed each aggregate's
arguments()[0] at a shipped column is gone, and the aggregates read their own
arguments.

Carrying columns rather than values is what the next stage needs, not merely
tidier. An aggregation temp table is filled by copy_fields(), which walks
Copy_field pairs holding raw Field pointers into the base tables, so a temp-table
stage cannot be satisfied by re-pointing Items -- the values have to be in the
base-table records. Nothing in this commit has such a stage, since the gate still
refuses one, but this is the change that makes GROUP BY reachable.

Every kind of aggregate now runs in the workers, where before only COUNT and SUM
did, and the restriction that produced that list is gone with the redirect it
described. MIN and MAX read their argument through an Item_cache built when they
were fixed, AVG and STD keep their own counters, and the DISTINCT variants keep
the set in an Aggregator; none of that is the transport's business once the record
is right. HAVING beside an aggregate works for the same reason. GROUP_CONCAT is
refused, and now for the only reason left: its value is the order the rows arrived
in, and workers finish chunks in an order that varies between runs, so it would
answer differently each time.

Two gate checks went with the design they served. The select list is no longer
tested for clonability, because nothing clones it -- and probing it was actively
harmful, since pwt_item_is_clonable() copies an item to test it and copying an
Item_sum_min_max crashes: its copy constructor leaves cmp uninitialised while its
cleanup() deletes it. The refusal of a select list with hidden items is gone too;
it existed so that reference-array slot i and result-table column i lined up, and
neither is used now.

This commit was prepared with Claude Code: it identified that copy_fields() holds
raw Field pointers and so decided the transport shape, wrote the column shipping
and the copy-back, replaced the hardcoded terminals with the plan's own,
established by test that the aggregate-kind restriction was then unnecessary,
found that the gate's clonability probe was what crashed on MIN, and updated the
tests whose recorded expectations the wider capability changed.
Sergei Petrunia
Remove incorrectly added sql/opt_sum.cc.orig
Georgi (Joro) Kodinov
Updated README.md to contain a more friendly version.
Sergei Golubchik
Revert "MDEV-39622 OBJECT_INSTANCE_BEGIN in P_S are unstable, difficult to compare"

Let's use MySQL's fix for compatibility

This reverts commit 11c41cd93d2c3732862ba043afd18508021440c0.
Marko Mäkelä
squash! e16b1b4e2739be7bc8e91643b0a71e1f04a1b02f

innodb_backup_batch_wait(): Ensure that no conflicting write-fixed pages exist
Teemu Ollakka
Galera MTR: disable SSL by default for SST, guard tests missing stunnel

Set ssl-mode=DISABLED in [sst] across all top-level galera/galera_3nodes
topology .cnf files, so tests no longer implicitly require stunnel via
inherited ssl-ca/ssl-cert/ssl-key. Also add the missing have_stunnel.inc
skip guard to galera_ist_rsync_verify_ca and galera_sst_cn_injection's
rsync combination, which intentionally test SSL SST but were not
skipped when stunnel is absent.
Teemu Ollakka
wsrep_info MTR: disable SSL by default for SST
Georgi (Joro) Kodinov
MDEV-39718: Produce Markdown plugin API documentation

Generated the plugin API headers using a shell script.
Fixed some doxygen comment mistakes in the headers.
Added a cmake conveninence target to generate the docs into $BUILD_DIR/docs
Added a main page for the API docs.
Included all of the existing group .md files into the CMake target
Leveraged moxygen 2.1.11's fixes to produce the full API docs in a single go
Removed the list of output .md files from the CMake target and switched to a
stamp file to avoid unnecessary rebuilds of the docs when the list of .md
files changes.
Addressed various review comments.
Thirunarayanan Balathandayuthapani
MDEV-19574: innodb_stats_method is not honored when innodb_stats_persistent=ON

Problem:
=======
When persistent statistics are enabled (innodb_stats_persistent=ON),
the innodb_stats_method setting is not properly utilized during
statistics calculation.

The statistics collection functions always use a hardcoded default
behavior for NULL value comparison instead of respecting the
configured stats method (NULLS_EQUAL, NULLS_UNEQUAL, or
NULLS_IGNORED). This affects the accuracy of n_diff_key_vals
(distinct key count) and n_non_null_key_val estimates, particularly
for indexes with nullable columns containing NULL values. This
impacts the query optimizer, which makes decisions based on
inaccurate cardinality estimates.

Solution:
========
Introduced IndexLevelStats to collect statistics at a specific
B-tree level during index analysis.

Introduced PageStats to collect statistics for leaf page analysis.

Refactored the following functions:
dict_stats_analyze_index_level() to IndexLevelStats::analyze_level()
dict_stats_analyze_index_for_n_prefix() to IndexLevelStats::sample_leaf_pages()
dict_stats_analyze_index_below_cur() to PageStats::scan_below()
dict_stats_scan_page() to PageStats::scan()

Add the stats method name to stat_description when innodb_stats_method
has a non-default value.

Added the new stat name n_nonnull_fld01, n_nonnull_fld02, etc. with a
stats description, to indicate how many non-null values exist for the
nth field of the index. This value is retrieved and stored in the index
statistics in dict_stats_fetch_index_stats_step().

rec_get_n_blob_pages(): Calculate the number of externally stored pages
for a record. It uses ceiling division with the actual usable blob page
space (blob_part_size) and now correctly handles both compressed and
uncompressed table formats for accurate BLOB page counting.

When InnoDB scans the leaf page directly, assign the leaf page count as
the number of pages scanned for a multi-level index. For single-page
indexes, use 1. This change leads to multiple changes in existing
test cases.

Non-null values are only counted at the leaf level, since only leaf
pages hold actual records. Both nullable and NOT NULL columns are
estimated with the same leaf-sampling formula:

  n_ordinary_leaf_pages * (n_non_null_all_analyzed_pages
                          / n_leaf_pages_to_analyze)

For a NOT NULL column every record is counted, so this yields the
estimated record count (no NULLs).

dict_stats_save(): now static function in dict0stats.cc that
takes the innodb_stats_method value, and is removed from dict0stats.h.

Replaced btr_rec_get_externally_stored_len() with rec_get_n_blob_pages()
in dict0stats.cc. btr_rec_get_field_ref_offs() and btr_rec_get_field_ref(),
together with the BTR_BLOB_HDR_* macros, were moved from btr0cur.cc
to btr0cur.h so that rec_get_n_blob_pages()
can reuse them;

btr_rec_get_field_ref_offs() is now a noexcept function returning size_t.

Changed stat_n_diff_key_vals and stat_n_non_null_key_vals from
ib_uint64_t* to uint64_t*

Removed the unused UNIV_STATS_DEBUG build macro (univ.i) and turned the
DEBUG_PRINTF() helper in dict0stats.cc into an unconditional no-op.
Sergei Golubchik
BUG#39449066 Refactor performance schema OBJECT_INSTANCE_BEGIN columns

Fix for MariaDB 10.6
Sergei Golubchik
MDEV-40589 default exclude list for secure-file-priv

don't allow to access /proc if secure-file-priv="",
set secure-file-priv=/ to access everything and disable the exclude list

remove test for a conditon that can no longer happen
Yuchen Pei
MDEV-15621 [to-squash] Follow some gemini review comments

Also removed table_rows in a SELECT, to avoid an unrelated flaky failure
bsrikanth-mariadb
MDEV-40388: sequence.simple fails on replay

The problem is that, when recording is enabled for the query such as,
explain select * from seq_1_to_10;
it recorded the table context having a DDL definition as: -

CREATE TABLE `seq_1_to_10` (
    ->  `seq` bigint(20) unsigned NOT NULL,
    ->  PRIMARY KEY (`seq`)
    -> ) ENGINE=SEQUENCE DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci;

Now, when that context is replayed, the DDL statement is executed.
But, we cannot create such a table, and instead it errors out saying
ERROR 1050 (42S01): Table 'seq_1_to_10' already exists.

Solution is to use: -
  CREATE TABLE IF NOT EXISTS seq_1_to_10 ...;

=====

Also, there is a different way to use sequences as: -
  Create sequence s1;
  Explain select * from s1;

Here, we should be recording the DDL statement, but no need to store the
stats for it. However, we didn't record the DDL statement earlier.
Moreover, sequence's next value should be the same in the replay environment.

Solution here is to record the DDL for such a sequence as
  CREATE TABLE IF NOT EXISTS s1 ...;
and also set its start value as the recorded environment's previous value using
  SELECT SETVAL(s1, prev_value);
Georgi (Joro) Kodinov
MDEV-39572: Add marking requirements for AI-assisted contributions to COMMUNITY_CONTRIBUTIONS.md

Explained the use of git commit trailers.
Fixed some minor header typos in the document.
Addressed review comments.
Hemant Dangi
Fix galera SST-abort tests to allow exit code 0 on some platforms

Issue: galera_sst_mariabackup_missing_ssl and galera_sst_rsync_missing_stunnel
only accepted exit codes 1,134 when the joiner mariadbd aborts SST, causing
intermittent failures (e.g. on FreeBSD) where the process exits cleanly with 0.

Solution: accept exit code 0 as well, matching the precedent already
established in galera_sst_cn_injection.test / galera_sst_mariabackup_encrypt_with_key_server.test.
Georgi (Joro) Kodinov
Updated README.md to contain a more friendly version.
Yuchen Pei
MDEV-15621 [refactor] Partitioning cleanup

change p_column_list_val::fixed to a bool
remove redundant end label in partition_info::fix_column_value_functions
Sergei Golubchik
MDEV-40608 MariaDB-devel is incomplete for plugins

create and install mariadb-plugin-config.cmake
Sergei Golubchik
cleanup: sys_vars.secure_file_priv test
Oleksandr Byelkin
Merge branch '10.11' into 11.4
Rex Johnston
MDEV-39492 Parallel Query: GROUP BY

The manager hands each drained row to the plan's terminal function, and for a
GROUP BY that is sub_select_postjoin_aggr(): it writes the row into the server's
aggregation temp table keyed on the group, and at end of records reads the table
back and drives every stage after it. So the grouping, the aggregates, HAVING
over the groups and an ORDER BY of the groups are the server's own work, done on
the manager, and this commit adds none of them. What it does is stop refusing
them.

The gate no longer refuses group_list, and no longer refuses need_tmp. The shapes
a temp table is otherwise built for -- DISTINCT, ORDER BY, window functions, a
procedure -- are each refused on their own terms, which says what is excluded
instead of excluding a superset of it. need_tmp said little there in any case: the
gate runs from make_join_readinfo(), before make_aggr_tables_info() plans the
aggregation, so it still held its pre-planning value.

What a chunked scan cannot promise is row order, and two of the server's group
terminals need it. end_send_group and end_write_group find a new group by
comparing each row's group values with the previous row's, so given unordered rows
they would emit a group per run of equal neighbours. The optimizer picks one of
them when its plan delivers the rows already grouped, which EXPLAIN shows as
"Using filesort" without "Using temporary". end_update and end_unique_update look
each group up in the temp table by key and do not care what order the rows arrive
in, and that is the plan for a GROUP BY no index resolves -- the case worth
parallelising.

Which terminal was chosen is not known when the gate runs, for the same reason
need_tmp was not. So pwt_plan_needs_group_order() asks in run_worker_side_join(),
after the aggregation has been planned, and declines the parallel scan the way the
engine can when it will not do a consistent read. AGGR_OP gets a getter for
write_func so the question can be asked.

With no GROUP BY, end_send_group is the terminal and is safe: there is one
implicit group, group_fields is empty and no group change is ever reported. That
is the aggregate case, which continues to run in the workers.

An ORDER BY that reaches execution unrefused is sound only when there is an
aggregation temp table for the sort to be applied to, because then the order is
established on the manager after every row has arrived and the order they arrived
in does not matter. Asserted rather than assumed, since the gate cannot see such
an ORDER BY either.

Measured shapes are in the new test, including the two halves of the order
decision: a GROUP BY that gets the keyed temp table runs in the workers, and
COUNT(DISTINCT) per group and WITH ROLLUP get the sorted plan and are declined.
GROUP BY the primary key is refused earlier and for an unrelated reason -- the
plan reads the clustered index in order, and a driving table that is not a full
scan is not one the gate accepts.

This commit was prepared with Claude Code: it traced the terminal-function chain
to find that one call reaches every post-scan stage, established from
set_postjoin_aggr_write_func() which terminals depend on row order, wrote the
gate relaxation and the execution-time check, and measured each shape in the test
against its serial answer -- correcting two explanations it had first written,
about indexes forcing the ordered plan, that EXPLAIN then disproved.
Aleksey Midenkov
MDEV-40480 LOAD DATA leaves a stale STORED generated column after a BEFORE INSERT trigger changes its base column

On the LOAD DATA path the base columns are filled directly from the
input file and fill_record_n_invoke_before_triggers() is then called
with an empty field list (there is no SET clause). After the BEFORE
INSERT trigger changed a base column, the stored generated columns were
not recomputed, so they kept the value derived from the pre-trigger
input (e.g. g=24 instead of 40 for g=v*2 with v set to 20 by the
trigger). A regular INSERT was unaffected.

The recompute was guarded by "fields.elements". That condition is a
leftover from the original computed-columns implementation
(f7a75b999b4), where fill_record_n_invoke_before_triggers() had no
TABLE* argument and had to reverse-derive the table from the first
item of the field list:

    if (fields.elements)
    {
      fld= (Item_field*)f++;
      item_field= fld->field_for_view_update();
      table= item_field->field->table;
      ...
    }

With an empty field list there was no way to obtain the table, so the
recompute was silently skipped. Since bc4a456758c (MDEV-452) the
function receives TABLE* explicitly, which made the whole derivation
dead code (as the in-place DBUG_ASSERT(table == item_field->field->table)
confirmed). Recompute the virtual fields unconditionally on
table->vfield, the same way the Field** overload of
fill_record_n_invoke_before_triggers() already does.
Sergei Golubchik
BUG#39449066 Refactor performance schema OBJECT_INSTANCE_BEGIN columns

Fix for MariaDB 10.6
Jan Lindström
Fix test failure on galera_sst_cn_injection test case.

Test requires pkill so skip it is not found from system. Additionally
pkill may fail if socat is not anymore open when pkill executed
(or when socat is not actually used).
Sergei Golubchik
MDEV-40589 default exclude list for secure-file-priv

don't allow to access /proc if secure-file-priv="",
set secure-file-priv=/ to access everything and disable the exclude list

remove test for a conditon that can no longer happen
forkfun
MDEV-39566 fix status_by_thread crash on live thread-count change

PFS_table_context snapshots the live thread/user/host/account
count at scan start and again on restore (filesort's second
rnd_init). If the count changed between the two, m_map_size
mismatched and the server aborted.

Skip the wasted re-sample on restore, bound each table's scan by
the frozen snapshot instead of the container's live count.
Sergei Golubchik
win secure-file-priv
Thirunarayanan Balathandayuthapani
MDEV-19574: innodb_stats_method is not honored when innodb_stats_persistent=ON

Problem:
=======
When persistent statistics are enabled (innodb_stats_persistent=ON),
the innodb_stats_method setting is not properly utilized during
statistics calculation.

The statistics collection functions always use a hardcoded default
behavior for NULL value comparison instead of respecting the
configured stats method. This affects the accuracy of
n_diff_key_vals (distinct key count), particularly for
indexes with nullable columns containing NULL values.

Moreover, stat_n_non_null_key_vals[] was never computed for
persistent statistics; it stayed at the 0 that
dict_stats_empty_index() assigns.

With innodb_stats_method=nulls_ignored, innodb_rec_per_key()
therefore always found n_diff <= n_null and reported one record
per key for every index. This impacts the query optimizer,
which makes decisions based on inaccurate cardinality estimates.

Solution:
========
Introduced IndexLevelStats to collect statistics at a specific
B-tree level during index analysis.

Introduced PageStats to collect statistics for leaf page analysis.

Refactored the following functions:
dict_stats_analyze_index_level() to IndexLevelStats::analyze_level()
dict_stats_analyze_index_for_n_prefix() to IndexLevelStats::sample_leaf_pages()
dict_stats_analyze_index_below_cur() to PageStats::scan_below()
dict_stats_scan_page() to PageStats::scan()

The innodb_stats_method value is read once per table in
dict_stats_update_persistent() and passed down, so that all
indexes of a table are analyzed with the same method.

Add the stats method name to stat_description when
innodb_stats_method has a non-default value. The suffix is
dropped when the description is already full.

Added the new stat name n_nonnull_fld01, n_nonnull_fld02, etc.
with a stats description, to indicate how many non-null values
exist for the nth field of the index. This value is retrieved
and stored in the index statistics
in dict_stats_fetch_index_stats_step(). The counts are per
column, not per n-column prefix.

rec_get_n_blob_pages(): Calculate the number of
externally stored pages for a record, using ceiling division
by the usable BLOB page payload (blob_part_size), which differs
between ROW_FORMAT=COMPRESSED (zip_size minus FIL_PAGE_DATA) and
the other formats (srv_page_size minus the BLOB header and the
page trailer). For ROW_FORMAT=COMPRESSED the length in
the field reference is the uncompressed length, so the result is an
upper bound.

When the leaf level is scanned in full, the number of leaf pages that
were scanned is reported as n_leaf_pages for a multi level index.
Before, result.n_leaf_pages was overwritten with
index->stat_n_leaf_pages, which dict_stats_empty_index() had
just set to 1, so every index that took the full scan path
reported n_leaf_pages=1. Single page indexes report 1.
This changes cardinality estimates and
therefore leads to multiple changes in existing test cases.

Non-null values are counted only at the leaf level, since only leaf
pages hold actual records. A full scan of the leaf level counts them
exactly. When the level is sampled, the per column count is derived
from the sampled leaves with the same formula as n_diff:

  n_ordinary_leaf_pages * n_non_null_all_analyzed_pages
                        / n_leaf_pages_to_analyze

This is an estimate for NOT NULL columns as well: the sampled leaves
may hold fewer or more records than the average, and a dive that
stops at a boring page contributes nothing to the sum while still
counting in the divisor.

innodb_rec_per_key(): stat_n_non_null_key_vals[i] holds the
number of records in which the i-th indexed column alone is
not NULL, while what has to be excluded here is the number
of records whose first i+1 columns are all not NULL,
because that is the population which the n-column
prefix statistic stat_n_diff_key_vals[i] has to be
corrected against when innodb_stats_method=nulls_ignored:
with NULLs compared as unequal, every record carrying a NULL
anywhere in the prefix adds a distinct value of its own to n_diff.

PageStats::scan(): n_non_null is accumulated and assigned only
for leaf pages, so that a non-leaf scan cannot leave a node
pointer count behind when scan_below() stops at a boring page
without reaching a leaf.

IndexLevelStats::reset_for_level() also clears n_diff[], and
dict_stats_analyze_index() zero initializes the buffer backing it, so
that a level scan which finds no records (a failed
btr_pcur_open_level(), or a non-leaf page whose first record is not
marked as the leftmost one on the level) leaves n_diff[] at 0 instead
of stale values.

IndexLevelStats::sample_leaf_pages() returns early when the group
boundaries for the prefix are empty, which is the same condition.

dict_stats_fetch_index_stats_step() no longer resets
stat_n_non_null_key_vals[] while processing an n_diff_pfxNN row:
dict_stats_empty_table() has already cleared the array before the
fetch, and with n_nonnull_fldNN rows now being read too,
that reset would make the result depend on the order in which the
rows arrive.

dict_stats_save(): now static function in dict0stats.cc that
takes the innodb_stats_method value, and is removed from dict0stats.h.
dict_stats_update_persistent() saves the statistics itself, so its
callers no longer have to.

Replaced btr_rec_get_externally_stored_len() with
rec_get_n_blob_pages() in dict0stats.cc.

btr_rec_get_field_ref_offs() and btr_rec_get_field_ref(),
together with the BTR_BLOB_HDR_* macros, were moved from btr0cur.cc
to btr0cur.h so that rec_get_n_blob_pages() can reuse them;

btr_rec_get_field_ref_offs() is now a noexcept function
returning size_t.

Changed stat_n_diff_key_vals and stat_n_non_null_key_vals from
ib_uint64_t* to uint64_t*

len_is_stored(): simplified to a single comparison, which is
equivalent for the unsigned lengths that it is used with.

Removed the unused UNIV_STATS_DEBUG build macro (univ.i) and
turned the DEBUG_PRINTF() helper in dict0stats.cc into an
unconditional no-op