Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Sergei Petrunia
Inline the mvi_key variable into its only use

mysql_prepare_alter_table() computed it at the top of the key loop and read
it ~290 lines below, at the one place that wants it. Nothing in between can
change the answer, and the early call also ran for keys that never reach the
assignment.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Vladislav Vaintroub
GnuTLS: actually load CRLs from ssl_crlpath directory

GnuTLS has no gnutls_certificate_set_x509_crl_dir(), so
ssl_crlpath was silently ignored, letting revoked server
certificates through. Enumerate the directory and load each
regular file as a CRL instead.
Mohammad Tafzeel Shams
MDEV-37467: InnoDB Instant ALTER TABLE is not crash safe

The hidden metadata record of instant ALTER TABLE was not written
crash-safely, in two independent ways.

First, the metadata record may include externally stored BLOB
metadata. The existing BLOB storage path in
btr_store_big_rec_extern_fields() writes the clustered index record
first, with zero BLOB pointers, and only fills in the BLOB pointers
afterwards. If the server is killed after the mini-transaction that
wrote the (incomplete) metadata record was durably committed, but
before the BLOB pointers were written, the table could become
inaccessible on recovery.

Make metadata BLOB storage crash-safe by writing the BLOB pages and
computing their pointers before the metadata record itself is
inserted or updated, so that the record is always written with
complete BLOB pointers. If the server is killed before the metadata
record is written, the already-written BLOB pages are merely
orphaned, which is safe.

Second, trx_undo_report_row_operation() writes the undo log record in
a mini-transaction of its own, which is committed before the
mini-transaction that writes the metadata record. Because
innobase_instant_try() had already updated SYS_COLUMNS and SYS_TABLES
in earlier mini-transactions, a kill in between left a durable undo
log record for the table while the metadata record was unchanged. On
recovery, trx_resurrect_table_locks() would then load the table
definition before the incomplete transaction was rolled back. The
data dictionary described the table as it would be after the
operation, while the metadata record still described it as it was
before, and btr_cur_instant_init() failed on that disagreement.

Write the undo log record of the metadata record in the same
mini-transaction that inserts or updates the record, so that the two
cannot be separated by a crash: until that mini-transaction is
committed, neither of them is durable.

An undo log record is never split between pages. If the DEFAULT
values of the columns being added are large enough that the undo log
record for updating the metadata record would not fit on one page,
innobase_instant_try() would fail. Determine this before the operation
starts, so that it can be performed by another algorithm instead.

- btr_store_big_rec_metadata():
  New function to store the off-page columns of a metadata record
  ahead of time. Each BLOB page is allocated and linked in its own
  mini-transaction, and the resulting BLOB pointers are written
  directly into the (heap-resident) index entry. On failure, it frees
  any pages it already allocated and resets the pointers to zero.

- btr_free_big_rec_metadata():
  New helper to free the BLOB pages written by
  btr_store_big_rec_metadata() and reset the entry's BLOB pointers
  to zero, used both on failure inside that function and by its
  callers when the metadata record ends up not being written.

- row_ins_clust_index_entry_low():
  For a metadata entry that needs external storage, convert it to a
  big record and call btr_store_big_rec_metadata() (with
  log_free_check() allowed, since no latches are held yet) before
  inserting the record. On failure, free the metadata BLOBs and
  convert the entry back.

- btr_cur_pessimistic_update():
  When updating a metadata record that requires external storage,
  call btr_store_big_rec_metadata() (without log_free_check(),
  since index and page latches are held) before modifying the record,
  and free the temporary big_rec vector via btr_free_big_rec_metadata()
  or dtuple_big_rec_free() on the various failure/success paths.

- btr_cur_optimistic_insert():
  Remove the special-cased jump to convert_big_rec for metadata
  entries, since their BLOBs are now always stored ahead of time by
  the caller; assert that a metadata entry never needs external
  storage at this point.

- innobase_instant_try():
  Since btr_cur_pessimistic_update() now stores metadata BLOBs
  before updating the record, big_rec is always NULL here; assert
  this instead of calling btr_store_big_rec_extern_fields().

- trx_undo_report_row_operation():
  New parameter caller_mtr. If it is specified, the undo log record
  is written in that mini-transaction, which is never committed or
  restarted here. An undo log page is added within the same
  mini-transaction if the record does not fit on the current one. A
  temporary table never uses the caller's mini-transaction, because
  that would require changing its logging mode. All other callers
  pass NULL and are unaffected.

- btr_cur_ins_lock_and_undo(), btr_cur_upd_lock_and_undo():
  For an instant ALTER TABLE metadata record, pass the
  mini-transaction that is going to insert or modify the record.

- trx_undo_max_rec_size():
  New function to determine the maximum size of an undo log record,
  that is, the space available on an empty undo log page.

- ha_innobase::check_if_supported_inplace_alter():
  Refuse ALGORITHM=INSTANT if the metadata record already exists and
  the undo log record for updating it would exceed
  trx_undo_max_rec_size(). trx_undo_page_report_modify() stores the
  DEFAULT value of each column that is being added in that record in
  full, inline. No such limit applies when the metadata record is
  being inserted, because trx_undo_page_report_insert() writes
  TRX_UNDO_INSERT_METADATA and no field data.

- Added test in innodb.instant_alter and innodb.instant_alter_crash
  to test normal working of INSTANT ALTER, crash safety and full table.
Vladislav Vaintroub
MDEV-40382 Support --ssl-crl on server builds with WolfSSL

--ssl-crl was silently discarded on WolfSSL: sslopt-case.h nulled
opt_ssl_crl after option parsing, and new_VioSSLFd() skipped CRL
loading for HAVE_WOLFSSL. A revoked certificate could still
authenticate a REQUIRE X509 / REQUIRE SUBJECT account.

WolfSSL's OpenSSL-compat layer already implements
X509_STORE_load_locations/X509_STORE_set_flags with real CRL
enforcement, so remove both restrictions.

Also fix X509_STORE_set_flags()'s success check: unlike OpenSSL,
WolfSSL can return negative error codes on failure, which == 0
missed.

Existing CRL tests were fixed to run with WolfSSL as well.
Vladislav Vaintroub
MDEV-40382 Support --ssl-crl on server builds with WolfSSL

--ssl-crl was silently discarded on WolfSSL: sslopt-case.h nulled
opt_ssl_crl after option parsing, and new_VioSSLFd() skipped CRL
loading for HAVE_WOLFSSL. A revoked certificate could still
authenticate a REQUIRE X509 / REQUIRE SUBJECT account.

WolfSSL's OpenSSL-compat layer already implements
X509_STORE_load_locations/X509_STORE_set_flags with real CRL
enforcement, so remove both restrictions.

Also fix X509_STORE_set_flags()'s success check: unlike OpenSSL,
WolfSSL can return negative error codes on failure, which == 0
missed.

Existing CRL tests were fixed to run with WolfSSL as well.

Note: on Windows, a revoked cert sometimes surfaces as ECONNRESET
instead of a TLS alert; the test's regex handles this too.
Sergei Petrunia
Improve comments, formatting.
Vladislav Vaintroub
MDEV-40382 Support --ssl-crl on server builds with WolfSSL

--ssl-crl was silently discarded on WolfSSL: sslopt-case.h nulled
opt_ssl_crl after option parsing, and new_VioSSLFd() skipped CRL
loading for HAVE_WOLFSSL. A revoked certificate could still
authenticate a REQUIRE X509 / REQUIRE SUBJECT account.

WolfSSL's OpenSSL-compat layer already implements
X509_STORE_load_locations/X509_STORE_set_flags with real CRL
enforcement, so remove both restrictions.

Also fix X509_STORE_set_flags()'s success check: unlike OpenSSL,
WolfSSL can return negative error codes on failure, which == 0
missed.

Existing CRL tests were fixed to run with WolfSSL as well.
sjaakola
MDEV-41012 Galera appliers hang with foreign key of types UUID, INET4, INET6

wsrep_store_key_val_for_row() built the certification key of a row by
collating the column value whenever the field reports MYSQL_TYPE_STRING
or MYSQL_TYPE_VAR_STRING, taking the collation from Field::charset().

The data types implemented on Field_fbt - UUID, INET6 and INET4
report MYSQL_TYPE_STRING, and their charset() is my_charset_numeric,
which is latin1. Their values are however plain binary and accordingly
get_innobase_type_from_mysql_type() maps them to DATA_FIXBINARY. Their
keys were therefore run through latin1_swedish_ci, which
folds them. That corrupts the key in two ways:

1. A key mismatch for one and the same row. The reference key that
wsrep_rec_get_foreign_key() appends for the parent of a child INSERT is
built from the InnoDB record and is not collated, so it no longer
matched the primary key carried by the parent row's own writeset.
Certification saw no dependency between a child INSERT and a concurrent
parent UPDATE, and two appliers could apply them in parallel causing
a hang or crash.

2. A key collision between distinct rows. The folding is many to one, so
different values collapse onto one key, Certification compares keys byte
for byte, so unrelated rows were treated as the same row. Concurrent
transactions on them certified as a conflict and one was aborted with
ER_LOCK_DEADLOCK.

Fix is for  wsrep_store_key_val_for_row() to skip the collation for
fields that InnoDB stores as binary, using the same condition as
get_innobase_type_from_mysql_type(). This is a no-op for the types that
worked before.
Hemant Dangi
MDEV-40501: Assertion `info->type == READ_CACHE || info->type == WRITE_CACHE' failed in reinit_io_cache upon CHANGE MASTER

Issue:
CHANGE MASTER ... FOR CHANNEL with a channel name within
MAX_CONNECTION_NAME can still overflow the OS file name limit once
escaped into the relay log file name. Relay_log_info::init() then
fails to open the relay log, leaving its index file unopened, but
Master_info_index::remove_master_info() unconditionally calls
reset_logs() on it during CHANGE MASTER's error cleanup, which hits
the assertion in reinit_io_cache().

Solution:
Guard the reset_logs() call in remove_master_info() with is_open(),
so a relay log that was never opened is never passed to it. Also use
MY_SAFE_PATH in open_index_file() so an over-length name fails
deterministically instead of silently falling back to a mangled one.
Thirunarayanan Balathandayuthapani
MDEV-41022 Wildcard term returns no rows when combined with a phrase in boolean mode search

Problem:
========
MATCH(..) AGAINST('"zzzz" aut*' IN BOOLEAN MODE) fails to return
the rows which match the wildcard term when the matching word is still
in the FTS cache. fts_query_phrase_search() sets query->flags to
FTS_PHRASE or FTS_PROXIMITY, but never resets it before returning.
The subsequent wildcard term of the same query then finds the stale
flag in fts_query_cache() and fts_query_difference(), takes the
exact word lookup path instead of fts_cache_find_wildcard()
and misses the words present in the cache.

Solution:
=========
- Reset query->flags in fts_query_phrase_search() before returning,
so that the phrase or proximity state doesn't leak into the terms
evaluated later in the same query.
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

In one of the practical cloud MariaDB setups, a server node accesses its
datadir over the network, but also has a fast local SSD storage for
temporary data. The content of such temporary storage is lost when the
server container is destroyed.

The commit uses this ephemeral fast local storage (SSD) as an extension of
the portion of InnoDB buffer pool (DRAM) that caches persistent data
pages. This cache is separated from the persistent storage of data files
and ib_logfile0 and ignored during backup.

The following system variables were introduced:

innodb_extended_buffer_pool_size - the size of external buffer pool
file, if it equals to 0, external buffer pool will not be used;

innodb_extended_buffer_pool_path - the directory in which the external
buffer pool file is created, the data directory is used if the variable
is not set.

If innodb_extended_buffer_pool_size is not equal to 0, external buffer
pool file will be created on startup of a normal server instance. It is
created as a temporary file, so that the operating system removes it
when the server closes it or exits, and no stale file is left behind
after a crash. For this purpose row_merge_file_create_mode() was
generalized into pfs_create_temp_file() and moved to fil0fil.cc, so that
both the merge sort files and the external buffer pool file are created
by the same function.

Only clean pages will be flushed to external buffer pool file. There is
no need to flush dirty pages, as such pages will become clean after
flushing, and then will be evicted when they reach the tail of LRU list.
Freed pages are not written to the external buffer pool file either,
they are just evicted.

The general idea of this commit is to flush clean pages to external
buffer pool file when they are evicted.

A page can be evicted either by transaction thread or by background
thread of page cleaner. In some cases transaction thread is waiting for
page cleaner thread to finish its job. We can't do flushing in external
buffer pool file when transaction threads are waiting for eviction,
that would hurt performance. That's why the only case for flushing is
when page cleaner thread evicts pages in background and there are no
waiters. For this purpose buf_pool_t::done_flush_list_waiters_count
variable was introduced, we flush evicted clean pages only if the
variable is zeroed.

Clean pages are evicted in buf_flush_LRU_list_batch() to keep some
amount of pages in buffer pool's free list. That's why we flush every
second page to external buffer pool file, otherwise there could be not
enough amount of pages in free list to let transaction threads to
allocate buffer pool pages without page cleaner waiting. This might be
not a good solution, but this is enough for prototyping.

External buffer pool page is introduced to store information in buffer
pool page hash about the certain page can be read from external buffer
pool file. The first several members of such page must be the same as the
members of internal page. External page frame must be equal to the
certain value to distinguish external page from internal one. External
buffer pages are preallocated on startup in external pages array. We
could get rid of the frame in external page, and check if the page's
address belongs to the array to distinguish external and internal pages.

There are also external pages free and LRU lists. When some internal page
is decided to be flushed in external buffer pool file, a new external
page is allocated either from the head of external free list, or from
the tail of external LRU list. Both lists are protected with
buf_pool.mutex. It makes sense, because a page is removed from internal
LRU list during eviction under buf_pool.mutex.

Then internal page is locked and the allocated external page is attached
to io request for external buffer pool file, and when write request is
completed, the internal page is replaced with external one in page hash,
external page is pushed to the head of external LRU list and internal
page is unlocked. After internal page was removed from external free list,
it was not placed in external LRU, and placed there only
after write completion, so the page can't be used by the other threads
until write is completed.

Page hash chain get element function has additional template parameter,
which notifies the function if external pages must be ignored or not. We
don't ignore external pages in page hash in two cases, when some page is
initialized for read and when one is reinitialized for new page creating.

When an internal page is initialized for read and external page with the
same page id is found in page hash, the internal page is locked,
the external page in replaced with newly initialized internal page in the
page hash chain, the external page is removed from external LRU list and
attached to io request to external buffer pool file. When the io request
is completed, external page is returned to external free list,
internal page is unlocked. So during read external page is absent in both
external LRU and free lists and can't be reused.

When an internal page is initialized for new page creating and external
pages with the same page id is found in page hash, we just remove external
page from the page hash chain and external LRU list and push it to the
head of external free list. So the external page can be used for future
flushing.

The external buffer pool file is not represented by a fil_space_t. The
requests to it are issued by fil_system_t::ext_bp_io(), and the external
buffer pool page which a request refers to is stored in IORequest in
place of the fil_node_t. The pages are written to and read from the
external buffer pool file in the same form in which they are written to
their tablespaces, i.e. compressed and encrypted pages stay compressed
and encrypted in external buffer pool file.

If a write to the external buffer pool file fails, a warning is written
to the error log and the external buffer pool is disabled for the rest
of the server lifetime.

Currently the commit passed some local smoke tests, mtr and RQG tests
with external buffer pool turned on.

TODO:
1. Add some monitoring, i.e. how much pages are currently in external
  buffer pool, the percent of hits during reading, take a look at the
  current buffer pool monitoring and implement the general monitoring
  tools for external buffer pool.
2. Think about partial initialization of external pages array, as it was
  done for internal pages.
3. Take a look at compressed LRU list, it looks like currently it's not
  covered with eviction to external buffer pool file (I don't currently
  understand if we need it at all).
4. Think about more suitable algorithm for eviction to external buffer
  pool, currently just every second page is flushed.
Alexander Barkov
MDEV-39518 Allow prepared statements in stored functions in assignment right hand

Allowing prepared statements in stored functions when
a stored function is used in an assignment right hand.

Both DEFAULT clause of a variable initialization and
the right side of the SET statement are supported:

  CREATE PROCEDURE p1()
  BEGIN
    -- case 1: DEFAULT clause
    DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK

    -- case 2: SP variable assignment statement
    DECLARE spvar2 INT;
    SET spvar2= f1_with_ps(); -- OK
  END;

- Only assignments to SP variables works for now:
  * SET spvar= func_with_ps(); -- OK
  * SET @uvar= func_with_ps(); -- Error

- Only bare function calls are supported for now. Using a function in
  an expression does not make it PS-safe yet:
    SET v= f1()+0;

- The parser now does not reject PS statements in stored functions.
  PS applicability in stored functions is now detected at run time.
  Note, PS statements in triggers are still prohibited by the parser.

- Functions with PS do not acquire MDL locks on tables, and no MDL is
  taken on the routines themselves either. They work like procedures in
  terms of table opening and routine locking: a concurrent DROP FUNCTION
  can complete while such a function is executing.

- Functions with PS are not replicated as a single `SELECT f1()` call.
  They are replicated per-statement, like procedures.

Helper changes:
- Changing the return result for LEX::sp_variable_declarations_init()
  from void to bool to catch errors in the caller properly.

Misc:
- This patch incorporates fixes for the following bugs found during debugging:
  MDEV-40224,MDEV-40225,MDEV-40226,MDEV-40227,MDEV-40240,
  MDEV-40285,MDEV-40288,MDEV-40315,MDEV-40318,MDEV-40890,MDEV-40900,
  MDEV-40901,MDEV-40913,MDEV-40914,MDEV-41013,MDEV-41015,MDEV-41019

Assisted-by: Claude - reviews and minor clean-ups
Yuchen Pei
MDEV-40168 nested array handling and json validation
Sergei Petrunia
This is a combination of 5 commits.

Undo whitespace changes to reduce diff size

Inline the mvi_key variable into its only use

mysql_prepare_alter_table() computed it at the top of the key loop and read
it ~290 lines below, at the one place that wants it. Nothing in between can
change the answer, and the early call also ran for keys that never reach the
assignment.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

Move the multi-valued key part DDL out of the grammar

multi_valued_key_part: built a whole schema object in its action: the hidden
DB_MVI_<n> column with its MVI_ENCODE() vcol, the rewrite of the key into an
invisible fulltext one, and the key part naming the column. That is DDL, and
it belongs with the rest of the multi-valued index code, not in sql_yacc.yy
where nobody reading opt_multi_valued_index.cc will find it.

It becomes add_mvi_key_part(), which returns the key part or NULL if it
raised an error, and the production is three lines. check_mvi_key_type()
follows its only caller and turns static, so sql_table.h loses a declaration
and the forward `class Key;` that existed only for it.

No behaviour change.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

opt_mvi_jsonfuncs.cc: Move the code, const-ify, add comments.

Improve comments, formatting.
Yuchen Pei
MDEV-40168 Add testcases for when there's both a ft index and an mvi
Sergei Petrunia
Move the multi-valued key part DDL out of the grammar

multi_valued_key_part: built a whole schema object in its action: the hidden
DB_MVI_<n> column with its MVI_ENCODE() vcol, the rewrite of the key into an
invisible fulltext one, and the key part naming the column. That is DDL, and
it belongs with the rest of the multi-valued index code, not in sql_yacc.yy
where nobody reading opt_multi_valued_index.cc will find it.

It becomes add_mvi_key_part(), which returns the key part or NULL if it
raised an error, and the production is three lines. check_mvi_key_type()
follows its only caller and turns static, so sql_table.h loses a declaration
and the forward `class Key;` that existed only for it.

No behaviour change.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Alessandro Vetere
MDEV-40408 btr_page_reorganize_low() uses the buffer pool just to obtain a scratch block

Add buf_pool.scratch_buf, a pool of page frames that the page
reorganization operations use instead of taking a block from the global
buffer pool.

buf_pool_t::scratch_buffer: A singly linked list of chunks of
buf_tmp_buffer_t slots. A thread holds at most one slot at a time, and
it holds a page latch while doing so, which is why reserve() must not
wait for the buffer pool or for I/O. When every slot is in use,
reserve() appends a chunk that holds twice the slots of the last one, up
to a limit on the size of one chunk. Only grow() waits, on the mutex
that serializes it and on the allocator. A chunk is never moved or freed
before close(), so a thread can keep using the slot that it reserved
while another thread appends a chunk. Debug builds start with a single
slot, so that a second concurrent page reorganization exercises grow().

buf_pool_t::scratch_buffer::shrink(): Free the page frames of the slots
that are not in use. The slots and the chunks are kept, because a slot
costs 32 bytes while a page frame costs srv_page_size. A frame survives
the first pass, because that pass only clears the used flag. The master
thread calls this often while the server is idle and rarely while it is
active, and buf_pool_t::garbage_collect() calls it under memory
pressure, where it ignores the used flag, because releasing these frames
is much cheaper than shrinking the buffer pool.

buf_pool_t::io_buf_t::acquire(): Factor out the scan for an unreserved
slot, which io_buf_t::reserve() ran twice and the scratch buffer reuses.

btr_page_reorganize_low(), page_zip_reorganize(): Obtain the scratch
page frame from buf_pool.scratch_buf. This removes the buf_pool.mutex
acquisition and the free block wait that buf_block_alloc() could
perform while at least a page X-latch was being held.
btr_page_reorganize_low() also used to leak the block on its error
paths; a single exit now releases the slot.

page_copy_rec_list_end_no_locks(), lock_move_reorganize_page(): Take
the source page frame instead of a source buf_block_t, because the
source is no longer a buffer pool block. In
page_copy_rec_list_end_no_locks() the source page is page_align(rec) at
every call site, so only the record is passed.

buf_tmp_buffer_t::acquire(), buf_tmp_buffer_t::release(): Use acquire
and release memory ordering, so that the page frame pointer of a slot
is published to the next thread that reserves the slot. acquire() also
reads the flag before the exchange, so that a scan across an array of
slots does not write to the slots that it finds reserved. release()
also marks the page frame undefined for Valgrind and MSan, because the
frame stays allocated for the next reserver and no deallocation marks
it.
Yuchen Pei
MDEV-40168 Add some DDL tests
Marko Mäkelä
fixup! 215c4e74e17bf5fb03af376c08159210330b9950
Sergei Petrunia
opt_mvi_jsonfuncs.cc: Move the code, const-ify, add comments.
Vladislav Vaintroub
MDEV-40382 Support --ssl-crl on server builds with WolfSSL

--ssl-crl was silently discarded on WolfSSL: sslopt-case.h nulled
opt_ssl_crl after option parsing, and new_VioSSLFd() skipped CRL
loading for HAVE_WOLFSSL. A revoked certificate could still
authenticate a REQUIRE X509 / REQUIRE SUBJECT account.

WolfSSL's OpenSSL-compat layer already implements
X509_STORE_load_locations/X509_STORE_set_flags with real CRL
enforcement, so remove both restrictions.

Also fix X509_STORE_set_flags()'s success check: unlike OpenSSL,
WolfSSL can return negative error codes on failure, which == 0
missed.

Existing CRL tests were fixed to run with WolfSSL as well.

Note: on Windows, a revoked cert sometimes surfaces as ECONNRESET
instead of a TLS alert; the test's regex handles this too.
Hemant Dangi
MDEV-40501: Assertion `info->type == READ_CACHE || info->type == WRITE_CACHE' failed in reinit_io_cache upon CHANGE MASTER

Issue:
CHANGE MASTER ... FOR CHANNEL with a channel name within
MAX_CONNECTION_NAME can still overflow the OS file name limit once
escaped into the relay log file name. Relay_log_info::init() then
fails to open the relay log, leaving its index file unopened, but
Master_info_index::remove_master_info() unconditionally calls
reset_logs() on it during CHANGE MASTER's error cleanup, which hits
the assertion in reinit_io_cache().

Solution:
Guard the reset_logs() call in remove_master_info() with is_open(),
so a relay log that was never opened is never passed to it. Also use
MY_SAFE_PATH in open_index_file() so an over-length name fails
deterministically instead of silently falling back to a mangled one.
Yuchen Pei
MDEV-40168 Add some DDL tests
Vladislav Vaintroub
MDEV-40382 Narrow ssl_crl.test's regex, don't mask unrelated failures

The previous fix matched any text after "TLS/SSL error:", too broad.
Require "revoked" or "10054" (WSAECONNRESET, seen on AppVeyor) so an
unrelated 2026 error still fails the test.
Oleksandr Byelkin
Fix the version
Sergei Petrunia
Undo whitespace changes to reduce diff size
Thirunarayanan Balathandayuthapani
MDEV-19574: innodb_stats_method is not honored when innodb_stats_persistent=ON

Problem:
=======
When persistent statistics are enabled (innodb_stats_persistent=ON),
the innodb_stats_method setting is not properly utilized during
statistics calculation.

The statistics collection functions always use a hardcoded default
behavior for NULL value comparison instead of respecting the
configured stats method. This affects the accuracy of
n_diff_key_vals (distinct key count), particularly for
indexes with nullable columns containing NULL values.

Moreover, stat_n_non_null_key_vals[] was never computed for
persistent statistics; it stayed at the 0 that
dict_stats_empty_index() assigns.

With innodb_stats_method=nulls_ignored, innodb_rec_per_key()
therefore always found n_diff <= n_null and reported one record
per key for every index. This impacts the query optimizer,
which makes decisions based on inaccurate cardinality estimates.

Solution:
========
Introduced IndexLevelStats to collect statistics at a specific
B-tree level during index analysis.

Introduced PageStats to collect statistics for leaf page analysis.

Refactored the following functions:
dict_stats_analyze_index_level() to IndexLevelStats::analyze_level()
dict_stats_analyze_index_for_n_prefix() to IndexLevelStats::sample_leaf_pages()
dict_stats_analyze_index_below_cur() to PageStats::scan_below()
dict_stats_scan_page() to PageStats::scan()

The innodb_stats_method value is read once per table in
dict_stats_update_persistent() and passed down, so that all
indexes of a table are analyzed with the same method.

Add the stats method name to stat_description when
innodb_stats_method has a non-default value. The suffix is
dropped when the description is already full.

Added the new stat name n_nonnull_fld01, n_nonnull_fld02, etc.
with a stats description, to indicate how many non-null values
exist for the nth field of the index. This value is retrieved
and stored in the index statistics
in dict_stats_fetch_index_stats_step(). The counts are per
column, not per n-column prefix.

rec_get_n_blob_pages(): Calculate the number of
externally stored pages for a record, using ceiling division
by the usable BLOB page payload (blob_part_size), which differs
between ROW_FORMAT=COMPRESSED (zip_size minus FIL_PAGE_DATA) and
the other formats (srv_page_size minus the BLOB header and the
page trailer). For ROW_FORMAT=COMPRESSED the length in
the field reference is the uncompressed length, so the result is an
upper bound.

When the leaf level is scanned in full, the number of leaf pages that
were scanned is reported as n_leaf_pages for a multi level index.
Before, result.n_leaf_pages was overwritten with
index->stat_n_leaf_pages, which dict_stats_empty_index() had
just set to 1, so every index that took the full scan path
reported n_leaf_pages=1. Single page indexes report 1.
This changes cardinality estimates and
therefore leads to multiple changes in existing test cases.

Non-null values are counted only at the leaf level, since only leaf
pages hold actual records. A full scan of the leaf level counts them
exactly. When the level is sampled, the per column count is derived
from the sampled leaves with the same formula as n_diff:

  n_ordinary_leaf_pages * n_non_null_all_analyzed_pages
                        / n_leaf_pages_to_analyze

This is an estimate for NOT NULL columns as well: the sampled leaves
may hold fewer or more records than the average, and a dive that
stops at a boring page contributes nothing to the sum while still
counting in the divisor.

innodb_rec_per_key(): stat_n_non_null_key_vals[i] holds the
number of records in which the i-th indexed column alone is
not NULL, while what has to be excluded here is the number
of records whose first i+1 columns are all not NULL,
because that is the population which the n-column
prefix statistic stat_n_diff_key_vals[i] has to be
corrected against when innodb_stats_method=nulls_ignored:
with NULLs compared as unequal, every record carrying a NULL
anywhere in the prefix adds a distinct value of its own to n_diff.

PageStats::scan(): n_non_null is accumulated and assigned only
for leaf pages, so that a non-leaf scan cannot leave a node
pointer count behind when scan_below() stops at a boring page
without reaching a leaf.

IndexLevelStats::reset_for_level() also clears n_diff[], and
dict_stats_analyze_index() zero initializes the buffer backing it, so
that a level scan which finds no records (a failed
btr_pcur_open_level(), or a non-leaf page whose first record is not
marked as the leftmost one on the level) leaves n_diff[] at 0 instead
of stale values.

IndexLevelStats::sample_leaf_pages() returns early when the group
boundaries for the prefix are empty, which is the same condition.

IndexLevelStats::analyze_level(): Instead of copying the last record
of the page, retain the latch on the page until the record has been
compared with the first record of the next page

dict_stats_fetch_index_stats_step() no longer resets
stat_n_non_null_key_vals[] while processing an n_diff_pfxNN row:
dict_stats_empty_table() has already cleared the array before the
fetch, and with n_nonnull_fldNN rows now being read too,
that reset would make the result depend on the order in which the
rows arrive.

dict_stats_save(): now static function in dict0stats.cc that
takes the innodb_stats_method value, and is removed from dict0stats.h.
dict_stats_update_persistent() saves the statistics itself, so its
callers no longer have to.

Replaced btr_rec_get_externally_stored_len() with
rec_get_n_blob_pages() in dict0stats.cc.

btr_rec_get_field_ref_offs() and btr_rec_get_field_ref(),
together with the BTR_BLOB_HDR_* macros, were moved from btr0cur.cc
to btr0cur.h so that rec_get_n_blob_pages() can reuse them;

btr_rec_get_field_ref_offs() is now a noexcept function
returning size_t.

Changed stat_n_diff_key_vals and stat_n_non_null_key_vals from
ib_uint64_t* to uint64_t*

len_is_stored(): simplified to a single comparison, which is
equivalent for the unsigned lengths that it is used with.

Removed the unused UNIV_STATS_DEBUG build macro (univ.i) and
turned the DEBUG_PRINTF() helper in dict0stats.cc into an
unconditional no-op
Vladislav Vaintroub
MDEV-40382 Fix ssl_crl.test regex, AppVeyor got a connection reset

On AppVeyor's Windows CI, the client saw a raw WSAECONNRESET (via
libmariadb's Schannel backend, reported as error 2026) rather than a
parsed TLS alert. Match any text after "TLS/SSL error:" instead of
requiring "revoked", using [^\r\n]* instead of .* so the match can't
swallow the line's trailing newline.
Thirunarayanan Balathandayuthapani
MDEV-26057 Assertion `!vcol->v_indexes.empty() in trx_undo_log_v_idx

Problem:
========
-  Rollback of an INPLACE ALTER TABLE is executed while holding only a
shared metadata lock on the table, so DML can run concurrently.
rollback_inplace_alter_table() resets dict_col_t::ord_part in a
critical section of its own, after row_merge_drop_indexes() already
removed the aborted indexes from the dictionary cache and emptied
dict_v_col_t::v_indexes. During this time, DML statement can see a
virtual column with ord_part set and an empty v_indexes, which
makes assert failure in trx_undo_report_insert_virtual().

Solution:
========
row_merge_reset_ord_part(): Added a function to reset
dict_col_t::ord_part for the columns that are no longer a field of
any index remaining in the dictionary cache.
For virtual columns the decision is based on dict_v_col_t::v_indexes
being empty, and no element is ever removed from that list.

row_merge_drop_indexes(): Added a call to row_merge_reset_ord_part()
in the branch that removes the indexes from the cache, in the same
dict_sys.latch critical section. That branch is taken only when
MDL_EXCLUSIVE is held or when this is the only handle to the table,
so no concurrent DML can observe the intermediate state.
In the lazy drop branch the indexes and their v_indexes entries
stay in the cache and nothing is reset; that is done later,
when the indexes are dropped while holding MDL_EXCLUSIVE.

check_col_exists_in_indexes(): Removed the only_committed parameter,
which no longer has any caller.

row_quiesce_col_ord_part(): Added a function to get
dict_col_t::ord_part and dict_col_t::max_prefix of a column
from the committed indexes that are
present in the dictionary cache.

row_quiesce_write_table(): Write the row_quiesce_col_ord_part() return
values to the .cfg file instead of the cached dict_col_t fields,
because a rolled back ADD INDEX leaves ord_part set until the
aborted index is removed by a later DDL, and
max_prefix is never reset when an index is dropped, which makes
IMPORT TABLESPACE reject the tablespace with a bogus schema mismatch.
sjaakola
MDEV-41012 Galera appliers hang with foreign key of types UUID, INET4, INET6

Added second test for testing key collisions from transactions modifying
separate rows
sjaakola
MDEV-41012 Galera appliers hang with foreign key of types UUID, INET4, INET6

Added a deterministic test for reproducing the issue
sjaakola
MDEV-41012 Galera appliers hang with foreign key of types UUID, INET4, INET6

Bumped application protocol version to level 5
Otto Kekäläinen
Promote getting GitHub stars in client prompt

Ask users to give MariaDB a star by having an extra line in the MariaDB
client prompt:

    Welcome to the MariaDB monitor.  Commands end with ; or \g.
    Your MariaDB connection id is X
    Server version: Y
    Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

    Help others discover MariaDB. Star it on GitHub: https://github.com/MariaDB/server

    Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

    MariaDB [(none)]>

Test file updated with:
nano --noconvert --nonewlines mysql-test/main/mysql-interactive.result

This change is similar to 346c7afe9b7071ce9c47892a83d69944b608b3da
applied on 'main', but without the SERVER_MATURITY_LEVEL check in order
to have this text visible in stable releases and actually help get the
GitHub star count up, and without the server log entry.
Vladislav Vaintroub
CONC-852 GnuTLS: actually load CRLs from ssl_crlpath directory

GnuTLS has no gnutls_certificate_set_x509_crl_dir(), so
ssl_crlpath was silently ignored, letting revoked server
certificates through. Enumerate the directory and load each
regular file as a CRL instead.
Marko Mäkelä
squash! b0944c9a38c3a3d5d7b9c87d3b137f67236bd744

BACKUP SERVER WITH 'command': Prepend the command with mariadb-backup-
and expect it to be in the search PATH. We distribute no such scripts;
the user has to write one.
Thirunarayanan Balathandayuthapani
MDEV-38942  i_s_dict_fill_sys_tables() aborts when reading INNODB_SYS_TABLES after innodb_force_recovery

Problem:
========
  A query on INFORMATION_SCHEMA.INNODB_SYS_TABLES crashes when
SYS_TABLES contains a record that was inserted by a transaction
which has not been committed. This can happen after a
crash while a CREATE TABLE was in progress, if the server is
restarted with innodb_force_recovery=4 or greater,
because trx_rollback_recovered() is then skipped and the
recovered transaction remains ACTIVE.

dict_sys_tables_rec_read() returns READ_NOT_FOUND for such a
record, and dict_load_table_low() returns that as success with no
error message and setting *table to nullptr.
i_s_sys_tables_fill_table() checks only the error
message and passes the nullptr table to i_s_dict_fill_sys_tables(),
which dereferences it.

Solution:
========
i_s_sys_tables_fill_table(): Skip the SYS_TABLES record when
dict_load_table_low() reports success but returns no table, because
such a record is not visible.
Thirunarayanan Balathandayuthapani
MDEV-39061 mariadb-backup compatible wrapper for BACKUP SERVER

This adds a shell script that lets users keep using their existing
mariadb-backup commands while the real work is done by the new
server-side BACKUP SERVER command. The goal is "drop-in": users should
not have to change their backup scripts.

extra/mariabackup/scripts/mariadb-backup-server.sh (plain POSIX sh)
understands the usual mariadb-backup modes and translates each one.
A companion helper, extra/mariabackup/scripts/mbstream-server.sh,
lets streamed backups be unpacked by pipelines that expect the
mbstream CLI. Both are documented in
extra/mariabackup/scripts/README.md.

--backup
========
Connects with the mariadb client and runs "BACKUP SERVER TO '<dir>'".
Connection options (--user, --host, --port, --socket, --defaults-file,
ssl, ...) are passed through to the client;

--parallel=N becomes the "<N + 1> CONCURRENT": mariadb-backup runs
a dedicated log_copying_thread() besides its N-data copy threads.
Minimum value is 2

After the backup it writes backup-prepare.cnf into the backup
directory, recording what --prepare needs later: where
mariadbd lives, the InnoDB parameters (page size, data file path,
undo tablespaces, checksum algorithm, log file size), and if
the server is encrypted then how to reload the encryption key
plugin (the file_key_management variables),
so an encrypted backup can be prepared without extra input.

--backup --stream
=================
Runs "BACKUP SERVER WITH '<command>'", where <command> writes the
tar into a fifo that the wrapper drains to its own stdout, so the
backup reaches the consumer as it is produced and never lands on
local disk.

backup-prepare.cnf is appended as a final tar afterwards. The
server's tar carries no end-of-archive marker;
only that trailing archive adds one, so the whole stream
extracts with a plain "tar -x".

--parallel is ignored here, with a warning. CONCURRENT selects the
number of output sinks as well as workers, and N self-contained
tars cannot be interleaved on one stdout, so a stream needs one writer.

For parallel streaming use BACKUP SERVER WITH N CONCURRENT directly,
giving each stream its own destination via the appended index.

Three properties follow from how BACKUP SERVER streams,
all differing from mariadb-backup:
- local: the stream command runs inside the server, so the wrapper
  must share its filesystem;
---target-dir is optional in stream mode (scratch for the per-stream.
- tar only: any --stream=<format> (including xbstream) yields tar;
- single-threaded: one worker, so no parallel read either.
--target-dir is optional in stream mode; nothing is spooled there,
it only holds the helper script, the fifo and backup-prepare.cnf
(a mktemp dir is used otherwise).

mbstream-server.sh maps the mbstream CLI onto a plain "tar -x"/"tar -c", so
existing "mbstream -x"/"-c" pipelines keep working on the wrapper's
stream. mbstream-only flags (-p/--parallel, ...) are accepted and
ignored; any other unknown option is rejected.

Environment overrides (mainly for testing): MARIADB (client),
MARIADBD (the --prepare bootstrap server) and TAR (the tar
implementation, e.g. TAR=bsdtar) can each be overridden. To run the
bootstrap under rr, put it in MARIADBD and let rr's own _RR_TRACE_DIR
choose the trace location, e.g.
  _RR_TRACE_DIR=/dev/shm/rr MARIADBD='rr record mariadbd'

--prepare
=========
Starts "mariadbd --bootstrap" on the backup directory using
backup-prepare.cnf as its defaults file, replays the archived redo
log between the start and target LSN read from backup.cnf,
then builds a fresh ib_logfile0 so a normal server can start
on the directory. mariadbd is taken from the path recorded in
backup-prepare.cnf if that binary exists, otherwise by searching
/libexec, /sbin, /bin and the configured install directories.
PATH is not searched; set MARIADBD to point elsewhere.

User --defaults-file/-extra-file and encryption options are
layered onto the bootstrap.

--copy-back / --move-back
=========================
Copy or move a prepared backup into the datadir. The datadir
is created if missing, a non-empty datadir is refused unless
--force-non-empty-directories is given, and a chown
reminder is printed.

If --aria-log-dir-path is given, the Aria logs (aria_log_control,
aria_log.*) are relocated into that directory.

Packaging
=========
The wrapper is not installed by default and never replaces the
real mariadb-backup / mbstream binaries.
1. cmake -DWITH_MARIABACKUP_WRAPPER=ON (default OFF) controls it.
2. When ON, the scripts install as /usr/bin/mariadb-backup-server
and /usr/bin/mbstream-server, tagged COMPONENT Backup so they
ship in the mariadb-backup package.
3. RPM: nothing extra to do. the component handles it.
4. DEB: not wired. debian/rules uses --fail-missing and does not
enable the option, so the -server binaries are not listed.
To ship via DEB, make a paired change: add
-DWITH_MARIABACKUP_WRAPPER=ON in debian/rules and list both
usr/bin/mariadb-backup-server and
usr/bin/mbstream-server in debian/mariadb-backup.install together.
5. The real mariadb-backup/mbstream binaries and the
mariabackup symlink are left untouched; opt in via an alias or a
symlink early in PATH.

Limitations (not supported yet)
===============================
1) Incremental backup & prepare (--incremental-basedir,
  --incremental-dir, --apply-log-only)
2) --rollback-xa
3) Partial backup (--databases, --tables, --tables-file)
4) Output compression and encryption (--compress, --encrypt)
5) --export is accepted but only warns and runs a plain recovery
6) --extra-lsndir is ignored
7) --parallel is ignored with --stream
8) Windows: POSIX sh only, not installed on Windows


Behaviour differences from native mariadb-backup
================================================
- The wrapper needs the mariadb client on PATH for --backup,
and mariadbd on PATH (or recorded in backup-prepare.cnf)
--backup; --prepare needs mariadbd recorded in backup-prepare.cnf,
in a standard install directory, or named by MARIADBD
- BACKUP SERVER refuses an already-existing target directory
- BACKUP SERVER does copy the data file as raw pages without
checksum validation, so a corrupted table is not detected
at backup time
- --prepare only works on a wrapper-made backup. It
needs backup-prepare.cnf)
- --stream is tar, not xbstream, local-only and single-threaded

Tests
=====
include/have_mariabackup_wrapper.inc redirects $XTRABACKUP to
mariadb-backup-server.sh and $XBSTREAM to mbstream-server.sh, skipping
when a wrapper or the mariadb client is unavailable.
include/have_mariabackup_combination.inc runs a test under both the
[CLIENT] mariadb-backup binary and the [SERVER] wrapper.
Vladislav Vaintroub
MDEV-40287 AES_ENCRYPT() and KDF() return NULL with OpenSSL 4.0

MyCTX faked an EVP_CIPHER_CTX in a stack buffer instead of allocating it
with EVP_CIPHER_CTX_new(). OpenSSL 4.0 rejects a context that was not
created that way for ciphers that use an IV: EVP_CipherInit_ex() fails
with "invalid iv length", so AES_ENCRYPT() and KDF() in CBC/CTR/GCM
modes return NULL. ECB has no IV and still works, which is why only the
non-ECB modes broke.

Allocate the context with EVP_CIPHER_CTX_new()/EVP_CIPHER_CTX_free().
Also remove check_openssl_compatibility() (it only validated the old
buffer's size, which was never the problem here - the context is 184
bytes) and the EVP_CIPHER_CTX_SIZE / EVP_CIPHER_CTX_init macros.

Verified against OpenSSL 4.0.1: the mysys aes-t test fails on the
CBC/CTR/GCM cases with the stack buffer and passes with
EVP_CIPHER_CTX_new().

No visible performance degradation: the extra allocation costs
~12 ns/call (WolfSSL) and ~30 ns (OpenSSL) on Windows at a 30-byte
payload, nothing at 16 KB, and nothing on Linux/glibc; sysbench OLTP
over encrypted tables and redo log is unchanged.

Assisted-by: Claude:claude-opus-4-8
Dimitri John Ledkov
MDEV-26015 ssl: remove insecure fixed DH params (mostly unused)- #5639

WolfSSL code path already operates without fixed DH parameters. OpenSSL code path still sets fixed static precomputed DH params, which is now prohibited by IETF.

Also OPENSSL_init_ssl is not required since OpenSSL 1.1.0, for over 10 years now. Also cleaned up at the same time.

https://www.rfc-editor.org/rfc/rfc10015.html#section-2:
> Clients MUST NOT offer and servers MUST NOT select non-ephemeral FFDH cipher suites in (D)TLS 1.2 connections.

https://www.rfc-editor.org/rfc/rfc10015.html#section-3:
> Clients MUST NOT offer and servers MUST NOT select FFDHE cipher suites in (D)TLS 1.2 connections.

And the depreciated tables include all ciphersuites that can use SSL_CTX_set_tmp_dh as part of the connection.

Also for a very long time OpenSSL was handling these automatically anyway, back when DHE was still recommended.
bsrikanth-mariadb
MDEV-36356 Server crash in Item::save_int_in_field with Window functions

A tail (ORDER BY / LIMIT / locking clause) that follows a parenthesized
query expression is parsed while the select inside the parentheses is
the current one, so everything the tail registers - its window
functions, the window specifications they introduce and the units of
its subqueries - ends up registered in that inner select.

When the parenthesized query expression already has a tail of its own,
LEX::add_tail_to_query_expression_body_ext_parens() wraps it into a
derived table and attaches the new tail to the wrapping select. The
registrations were left behind in the inner select, so the ORDER BY of
the wrapping select contained window function items that no select had
registered: they never got a Window_funcs_sort, were never computed,
and their result field was read unset. The subqueries of the tail kept
the inner select as their master and name resolution context.

Move these registrations to the wrapping select in the new
move_tail_registrations():

- The window functions of the tail are the ones its ORDER BY items
  contain, looked up with Item::walk() and walk_subquery == FALSE so
  that a window function belonging to a subquery of the tail is not
  found and stays registered where it was parsed. Each one takes along
  the window specification it introduced; "OVER win_name" has none of
  its own, it is looked up by name at fix_fields() time and never
  reaches window_specs.

- Item::walk() does not descend into a window specification, so the
  items of a moved specification's PARTITION BY and ORDER BY lists are
  not re-targeted by Lex_order_limit_lock::set_to() and are pointed at
  the wrapping select's name resolution context here.

- The subquery units of the tail are the ones registered in front of
  the first unit the inner select had when the tail started to be
  parsed; the grammar remembers it in a mid-rule action. There is no
  equivalent of the walk above for them: a unit records neither the
  clause it came from nor its position in the parsed text. They are
  excluded from the inner select, re-registered in the wrapping select
  and their Item_subselect::parent_select is updated.

Per-select parse state that is not a registration (n_sum_items,
with_sum_func, with_rownum, ftfunc_list, uncacheable, and the
select_n_where_fields of the tail's own fields) is deliberately left
behind, where it is merely over-counted.

Also remove the unreachable !unit check - unit is dereferenced on
entry - and fix the indentation of the surrounding block.