Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Rucha Deodhar
MDEV-39049: Memory corruption & crash in check_key_in_list upon using
JSON_KEYS after modifying character set name/collation

Analysis:
Since the length of string is 0, accessing out of boundry memory,
leads to crash

Fix:
If string length is empty, return success.
Dave Gosselin
MDEV-33616:  Match the macOS dlopen error in plugins.multiauth

The client reports why it could not load client_ed25519, and macOS names
every path that dlopen() tried.  Two expressions are added, one for the
chunk that holds the start of that message and one for the chunk that
holds the rest of it.

The line runs to 563 bytes, 52 of prefix and the 511 that the client
error buffer holds, while do_exec() reads the output with fgets() into a
512 byte buffer and runs the replacements on each chunk on its own.  A
long enough vardir therefore splits the line, because the path appears
four times in the dlopen text.  The second chunk is the tail of a path
and carries no colon, where the first chunk keeps the colons of the
mysqltest prefix.  That chunk also holds the only line terminator the
error line gets, so the expression captures the newline and the
replacement puts it back.  A replacement is inserted as written, so a \n
spelled there would reach the output as a backslash and an n.

Both expressions stop at a newline.  reg_replace compiles with
REG_DOTALL, so an unrestricted .* runs past the line terminator whenever
the whole message reaches the replacement in one chunk, and the error
line then joins the line after it.
Rex Johnston
MDEV-35168 subselects with outer references to derived tables may be incorrectly evaluated as constant

Subselects with outer references to derived tables may be incorrectly
evaluated as having no table references.  This can lead to these
subselects being marked as constant, leading to an incorrect
result.

During the calculation of the tables used in a subselect, we construct a
table map of outer references in our (not necessarily new) "new_parent"
select.  This is currently done purely by finding Item_fields in our tree
and using the attached table to update our bitmap.  It can be that a
reference to a derived table also needs to have it's table added to this
map.  If the derived table can be null, this is the case.

We add a new processor to our item walk system,
enumerate_table_refs_processor which is defined at this stage only
for Item_direct_view_ref items.
This called alongside enumerate_field_refs_processor in
Item_subselect::recalc_used_tables().

Coverage of the merged_into() function is provided at the end of the
tests in subselect4.test.  Be aware that they are not a regression test,
we could not find anything that provided an incorrect output.
Aleksey Midenkov
MDEV-40821 SIGSEGV in Window_funcs_sort::setup

Window_funcs_sort failed accessing win_func->window_spec as window
spec was not defined in the query. Usually this is checked by fixing
item win_func, but it was not done by setup_conds.

Actually, earlier check by vers_setup_conds() must fail on DELETE
HISTORY from non-versioned table, but TABLE_LIST for t1 has no
versioning conditions.

The cause was the parser assigning versioning conditions to wrong
table pointed by last_table() which was already switched to another
table from SYSTEM_TIME expression (cs1).

The fix assigns versioning conditions to the correct table stored to
correspondent_table by delete_single_table branch of the parser.
Aleksey Midenkov
MDEV-40854 Use of uninitialized table_will_be_deleted in federated engine

MemorySanitizer report:

==851408==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 ha_federated::end_bulk_insert() storage/federated/ha_federated.cc:2035:30
    #1 mysql_insert(THD*, ...) sql/sql_insert.cc:1258:11
    ...
  Memory was marked as uninitialized
    #0 __msan_allocated_memory
    #1 my_malloc mysys/my_malloc.c:116:7
SUMMARY: MemorySanitizer: use-of-uninitialized-value ... end_bulk_insert()

table_will_be_deleted is a handler member with no in-class
initializer, and the handler object itself is heap-allocated via
my_malloc(), so it starts out as garbage. It was only ever set in
extra(HA_EXTRA_PREPARE_FOR_DROP) and in external_lock().  For a
TEMPORARY table, statement execution can reach
end_bulk_insert()/write_row(), which reads the flag, without
external_lock() having run first, so the read sees uninitialized
memory.

The fix initializes table_will_be_deleted in reset(), which runs at
the start of every statement regardless of locking path.
Marko Mäkelä
MDEV-41152: Fix FILE_CREATE recovery

fil_name_process(): Treat FILE_CREATE in the same way as FILE_MODIFY
that led to a FIL_LOAD_DEFER return. Remove the parameter lsn,
and return file_name_t& in which the caller may assign create_lsn
when processing a FILE_CREATE record.

deferred_spaces.reinit_all(): Never create anything for deleted
tablespaces. Doing so could cause a legitimate file to be deleted
if files are being deleted and re-created with the same name.

deferred_space.create(): Remove some duplicated code. Missing
tablespace files will be created in fil_node_open_file_low()
starting with
commit 759e3523e3d832b174cf0a612704da38b2557b40 (MDEV-38026).

deferred_spaces::item::lsn: Remove. Starting with
commit 37d8577aee3bd87b5b04464144d064063b169039 (MDEV-40728)
each FILE_ record is parsed only once.

recv_sys_t::parse_store_if_exists(): Tell the caller to skip
tablespaces for which both FILE_CREATE and FILE_DELETE was parsed.
This improves performance, not correctness.

recv_validate_tablespace(): Avoid duplicated tablespace lookup
and remove a redundant deferred_spaces.add(); fil_name_process
already keeps deferred_spaces in sync with recv_spaces.

fil_space_t::rename(): If !log, assert !replace and that
the target file name does not exist.

os_file_rename_func(): Do not check that the target path
does not exist. This is already checked by every caller.
This fixes a debug assertion failure that could otherwise
occur when recovering from a crash in
fil_space_t::rename() between the write of the FILE_RENAME
and the actual rename.

Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Saahil Alam

(cherry picked from commit bdff5b8f2455acd226d242e16e8c2107f741d3da)
Marko Mäkelä
MDEV-41152: Fix FILE_CREATE recovery

fil_name_process(): Treat FILE_CREATE in the same way as FILE_MODIFY
that led to a FIL_LOAD_DEFER return. Remove the parameter lsn,
and return file_name_t& in which the caller may assign create_lsn
when processing a FILE_CREATE record.

deferred_spaces.reinit_all(): Never create anything for deleted
tablespaces. Doing so could cause a legitimate file to be deleted
if files are being deleted and re-created with the same name.

deferred_space.create(): Remove some duplicated code. Missing
tablespace files will be created in fil_node_open_file_low()
starting with
commit 759e3523e3d832b174cf0a612704da38b2557b40 (MDEV-38026).

deferred_spaces::item::lsn: Remove. Starting with
commit 37d8577aee3bd87b5b04464144d064063b169039 (MDEV-40728)
each FILE_ record is parsed only once.

recv_sys_t::parse_store_if_exists(): Tell the caller to skip
tablespaces for which both FILE_CREATE and FILE_DELETE was parsed.
This improves performance, not correctness.

recv_validate_tablespace(): Avoid duplicated tablespace lookup
and remove a redundant deferred_spaces.add(); fil_name_process
already keeps deferred_spaces in sync with recv_spaces.

fil_space_t::rename(): If !log, assert !replace and that
the target file name does not exist.

os_file_rename_func(): Do not check that the target path
does not exist. This is already checked by every caller.
This fixes a debug assertion failure that could otherwise
occur when recovering from a crash in
fil_space_t::rename() between the write of the FILE_RENAME
and the actual rename.

Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Saahil Alam
Dmitry Shulga
MDEV-30645: CREATE TRIGGER FOR { STARTUP | SHUTDOWN }

Follow-up patch to fix missing call to my_error() in case not all
mandatory columns present in the table mysql.event
Thirunarayanan Balathandayuthapani
MDEV-41207 Acquire metadata locks for recovered transaction

Problem:
=======
A transaction being rolled back during recovery holds LOCK_IX on the
table(not the metadata locks), and the rollback thread holds
a reference on it.

An online ALTER TABLE on that table falls back to acquiring LOCK_S,
which conflicts and fails. prepare_inplace_alter_table_dict()
asserted that the reference count is 1 before checking whether
the table lock was acquired, so the reference still held by the
rollback thread makes the assertion fail.

Solution:
========
trx_recovery_thd: A background connection that owns the metadata
locks of the recovered transactions. One connection is shared by
all of recovering transaction.

trx_lists_init_at_db_start(): Create trx_recovery_thd, before any
metadata lock can be acquired.

trx_resurrect_table_locks(): Acquire a shared metadata lock on each
table that the recovered transaction had modified, and remember the
locks in trx_recovery_mdl. Recovered XA PREPARED transactions are
excluded, because they are completed by a user connection.

trx_recovery_mdl: The metadata locks of the recovered transactions,
by transaction. They are kept outside trx_t. Only a recovered
transaction ever has an entry.

trx_recovery_mdl_exists: Whether trx_recovery_mdl is not empty.
It is read for every transaction in trx_t::free(),
so that the map will only be consulted while some recovered
transaction still holds metadata locks. It becomes false as
soon as the rollback of the recovered transactions has been
completed, long before trx_recovery_thd is
destroyed.

trx_t::free(): Release the metadata locks of a recovered transaction,
once its rollback has been completed. This is the only place that
releases them, so that a transaction which was rolled back by
trx_rollback_recovered(false) will not keep its locks until shutdown.

trx_recovery_thd_destroy(): Destroy trx_recovery_thd.
It is invoked by innodb_shutdown() only, after trx_sys.close()
has freed any recovered transaction that was left.

row_undo_mod(): Add the debug injection rollback_wait
Oleksandr Byelkin
MDEV-32401 expression cache lead to crash if table of wrong type created and the cache switched off

1) take into account TMP_TABLE_ALL_COLUMNS when
  we are modifying agg_item->result_field
2) remove unused now "bool materialized_subquery;"
Dave Gosselin
MDEV-33616:  Normalize the strerror text in innodb_fts.index_table

The injected deadlock reaches the client as ER_GET_ERRNO carrying errno
11, and the text comes from my_strerror().  11 is EAGAIN on Linux and
EDEADLK on macOS, so the message reads "Resource temporarily
unavailable" on one and "Resource deadlock avoided" on the other.
Replace the quoted text so the test does not depend on it.
Dave Gosselin
MDEV-33616:  Skip the redo log upgrade tests without sparse file support

innodb.log_upgrade and innodb.log_upgrade_101_flags build 8GB redo log
files by seeking past the end of an empty file and writing a single
byte.  That needs a filesystem which leaves the skipped range
unallocated.  HFS on macOS allocates every block of it instead, so the
write fails with ENOSPC and the test reports a perl failure.

include/have_sparse_files.inc probes a directory the caller names,
writing one byte 64MB into an empty file there and comparing the
allocated block count against that offset.
Dave Gosselin
MDEV-33616:  Make two tests independent of lower_case_table_names

macOS puts the data directory on a case insensitive file system, so
lower_case_table_names is 2 and both tests recorded an answer that only
holds for 0.

period.i_s_notembedded looked up I_S.PERIODS and I_S.KEY_PERIOD_USAGE by
the schema name TEST.  That comparison follows the table name
comparison, so it finds the table under 1 and 2 and finds nothing under
0.  Those four queries move to the new test period.i_s_case_sensitive,
which requires lower_case_table_names=0.  The win rdiff of
period.i_s_notembedded covered the same difference and is no longer
needed.

atomic.drop_db_long_names generated table and view names in upper case
and compared the DROP statements that DDL recovery writes to the binary
log.  Under 2 the names come back from the directory in lower case.
Generating them in lower case to begin with gives the same names on
every setting.  Lower case also changes where the view name sorts
against its table name for the letters after v, which moves one view
between two of the recorded DROP VIEW statements.
Oleksandr Byelkin
MDEV-39993 Use CREATE OR REPLACE for sys schema routines

mariadb-upgrade silently dropped EXECUTE grants on sys schema stored
functions and procedures. The sys schema install scripts reinstalled
every routine with DROP FUNCTION/PROCEDURE IF EXISTS followed by
CREATE. DROP cascades to delete the routine's rows in
mysql.procs_priv, so any EXECUTE grant a DBA had issued on e.g.
sys.table_exists or sys.quote_identifier was lost every time
mariadb-upgrade reinstalled the sys schema, even though the routine
itself came back unchanged.

Fix: replace DROP ... IF EXISTS + CREATE with CREATE OR REPLACE in
all 53 sys_schema function/procedure files and in the two templates
(templates/function.sql, templates/procedure.sql) so future routines
follow the same pattern. CREATE OR REPLACE PROCEDURE/FUNCTION goes
through sp_drop_routine_internal(), which only deletes the
mysql.proc row and never reaches sp_revoke_privileges() (that is
only called from the explicit DROP PROCEDURE/FUNCTION statement),
so mysql.procs_priv is left untouched and existing grants survive.
This mirrors the pattern already used by sys schema views
(CREATE OR REPLACE ... VIEW, since MDEV-9077), which never had this
problem.

Four of the converted files (functions/format_path.sql,
functions/ps_is_account_enabled_57.sql,
procedures/ps_setup_reset_to_default.sql,
procedures/ps_trace_thread_57.sql) are not referenced by
scripts/sys_schema/CMakeLists.txt; they were converted anyway for
consistency and have no behavioural effect.

As a side effect, a pre-existing UDF whose name collides with a sys
routine name is no longer destroyed before the reinstall fails:
DROP FUNCTION IF EXISTS resolved the UDF namespace first, silently
dropping the UDF and then failing on ER_SP_ALREADY_EXISTS anyway;
CREATE OR REPLACE fails immediately on ER_UDF_EXISTS with the UDF
intact.

scripts/maria_add_gis_sp.sql.in and the sys_config triggers were
deliberately left untouched: the GIS procedures are only
re-installed at bootstrap time (mariadb-upgrade instead patches
their definer in place via UPDATE), and triggers carry no
procs_priv rows, so neither is on the code path this bug is about.

Added mysql-test/main/mysql_upgrade_sys_routine_grants.test, which
grants EXECUTE on sys.table_exists and sys.quote_identifier,
overwrites both routine bodies with a marker to prove the upgrade
actually reinstalls them (rather than the sys schema install being
skipped), runs mariadb-upgrade, and checks both that the grants
survived and that the real routine bodies came back.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Dave Gosselin
MDEV-33616:  Exclude innodb_log_file_mmap from sys_vars.sysvars_innodb

Its default value depends on the operating system, ON where the log can
be memory mapped and OFF elsewhere, so the recorded row only holds on
some platforms.  The other variables whose default depends on the
operating system are already excluded the same way.
Dave Gosselin
MDEV-33616:  Allocate the recovery buffer from the heap

recv_sys.tmp_buf comes from malloc() rather than from the large page
allocator.

main.large_pages fails on macOS with "Warning: Memory not freed: 16375"
at shutdown.  recv_sys_t::find_checkpoint() asks for 1048585 bytes,
my_large_malloc() rounds that up to 1064960 and charges the rounded
figure to the server memory accounting, and recv_sys_t::tmp_free()
credits back the 1048585 that was requested.  ut_malloc_dontdump() takes
the size by value, so it has nowhere to report what my_large_malloc()
wrote back.

The rounding happens whenever my_next_large_page_size() finds a reported
large page size at or below the request.  macOS has no huge page
interface for my_get_large_page_sizes() to consult, so its fallback
branch reports the ordinary page size, 16384 on Apple silicon, and the
request is always rounded.  Linux reads the sizes from
/sys/kernel/mm/hugepages, where the smallest entry is usually 2 MiB, and
a 1 MiB request then gets no large page and no rounding.

The buffer has no alignment requirement.  recv_sys_t::parse() copies a
mini-transaction into it when the record is encrypted in the
FORMAT_ENC_11 log, where it is then decrypted in place, or when the
record wraps around the end of the log file, and reads it back as a byte
sequence.  tmp_free() calls std::free() because the member function
recv_sys_t::free() hides the one from <cstdlib>.

log_sys.buf and log_sys.flush_buf keep the large page allocator.  They
round the same way, so a server started with --large-pages
--innodb-log-buffer-size=2101248 still reports 24576 on macOS.  The core
dump exclusion that recv_sys.tmp_buf gives up applies only where
MADV_DONTDUMP exists, so nothing changes on macOS, while a release build
on Linux would now include the buffer in a dump.  tmp_free() overwrites
the redo log records that innodb_encrypt_log decrypted before releasing
the memory, through a volatile function pointer because GCC removes a
plain memset() that is followed by free().

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Khaled Riyad
MDEV-38861 heap-use-after-free in Prepared_statement::execute()

DROP PROCEDURE and CREATE OR REPLACE PROCEDURE executed from inside the
routine itself removed it from the SP cache. sp_head::destroy() then freed
the memory root that the running sp_head, its LEX and its instructions
live in, and the caller kept using them.

Skip the removal while the routine is being executed. sp_cache_invalidate()
above has already bumped the cache version, so the stale entry is removed by
the next lookup, after IS_INVOKED has been cleared.
Oleg Smirnov
MDEV-28869 Show tables removed by table elimination in EXPLAIN

Tables removed by table elimination were omitted from EXPLAIN and
EXPLAIN FORMAT=JSON entirely. A user comparing the query text with
the plan could not tell whether a table had been eliminated or the
optimizer had lost it, and after MDEV-26278 a whole derived table
could disappear the same way.

Such tables are now shown with eliminated in the type column
(in access_type for FORMAT=JSON), and an eliminated derived table
also prints its contents, every table of it marked the same way.
They are printed after the tables that take part in the plan.
Alessandro Vetere
MDEV-41036 buf_page_peek_if_young() does not properly compute and wrap page age in young calculation

buf_page_t::freed_page_clock is a 31-bit bitfield holding the low bits
of buf_pool.freed_page_clock. buf_page_peek_if_young() evaluated the
condition now < stamp + window in size_t arithmetic, which failed to
account for wraparound within the 31-bit clock space.

When the global clock wrapped past a stamp (particularly when stamp was
high, near 2^31), the comparison evaluated to true for an extended
period even though the page was older than the window. During these
wraparound windows, pages were falsely reported as young.

Because returning true causes InnoDB to skip moving pages to the MRU
head (to reduce lock contention), these falsely "young" pages were not
promoted.
As a result, active/hot pages could sink into the old portion of the LRU
list and be evicted prematurely. Additionally, buf_read_ahead_random()
over-counted recently accessed pages during wraparound intervals.

Compute age as (now - stamp) & clock_mask using 31-bit modular
arithmetic and compare age < window. A page then cleanly leaves the
young window after the intended number of evictions regardless of clock
wraparound.
Thirunarayanan Balathandayuthapani
MDEV-40324 use-of-uninitialized-value after creation of FULLTEXT table failure

Problem:
=======
For fulltext index, row_create_index_for_mysql() calls
fts_create_index_tables(). If creating FTS auxiliary table fails,
error handling performs trx->rollback() of the dictionary
transaction. Rollback removes the parent table from
dictionary cache and frees it. After that,
convert_error_code_to_mysql() reads table->flags after table->heap.
This leads to read of freed memory.

Solution:
========
create_index(): Read table->flags into a local variable before
calling row_create_index_for_mysql()
Oleksandr Byelkin
MDEV-39993 Use CREATE OR REPLACE for sys schema routines

mariadb-upgrade silently dropped EXECUTE grants on sys schema stored
functions and procedures. The sys schema install scripts reinstalled
every routine with DROP FUNCTION/PROCEDURE IF EXISTS followed by
CREATE. DROP cascades to delete the routine's rows in
mysql.procs_priv, so any EXECUTE grant a DBA had issued on e.g.
sys.table_exists or sys.quote_identifier was lost every time
mariadb-upgrade reinstalled the sys schema, even though the routine
itself came back unchanged.

Fix: replace DROP ... IF EXISTS + CREATE with CREATE OR REPLACE in
all 53 sys_schema function/procedure files and in the two templates
(templates/function.sql, templates/procedure.sql) so future routines
follow the same pattern. CREATE OR REPLACE PROCEDURE/FUNCTION goes
through sp_drop_routine_internal(), which only deletes the
mysql.proc row and never reaches sp_revoke_privileges() (that is
only called from the explicit DROP PROCEDURE/FUNCTION statement),
so mysql.procs_priv is left untouched and existing grants survive.
This mirrors the pattern already used by sys schema views
(CREATE OR REPLACE ... VIEW, since MDEV-9077), which never had this
problem.

Four of the converted files (functions/format_path.sql,
functions/ps_is_account_enabled_57.sql,
procedures/ps_setup_reset_to_default.sql,
procedures/ps_trace_thread_57.sql) are not referenced by
scripts/sys_schema/CMakeLists.txt; they were converted anyway for
consistency and have no behavioural effect.

As a side effect, a pre-existing UDF whose name collides with a sys
routine name is no longer destroyed before the reinstall fails:
DROP FUNCTION IF EXISTS resolved the UDF namespace first, silently
dropping the UDF and then failing on ER_SP_ALREADY_EXISTS anyway;
CREATE OR REPLACE fails immediately on ER_UDF_EXISTS with the UDF
intact.

scripts/maria_add_gis_sp.sql.in and the sys_config triggers were
deliberately left untouched: the GIS procedures are only
re-installed at bootstrap time (mariadb-upgrade instead patches
their definer in place via UPDATE), and triggers carry no
procs_priv rows, so neither is on the code path this bug is about.

Added mysql-test/main/mysql_upgrade_sys_routine_grants.test, which
grants EXECUTE on sys.table_exists and sys.quote_identifier,
overwrites both routine bodies with a marker to prove the upgrade
actually reinstalls them (rather than the sys schema install being
skipped), runs mariadb-upgrade, and checks both that the grants
survived and that the real routine bodies came back.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Aleksey Midenkov
MDEV-40821 SIGSEGV in Window_funcs_sort::setup

Window_funcs_sort failed accessing win_func->window_spec as window
spec was not defined in the query. Usually this is checked by fixing
item win_func, but it was not done by setup_conds.

Actually, earlier check by vers_setup_conds() must fail on DELETE
HISTORY from non-versioned table, but TABLE_LIST for t1 has no
versioning conditions.

The cause was the parser assigning versioning conditions to wrong
table pointed by last_table() which was already switched to another
table from SYSTEM_TIME expression (cs1).

The fix assigns versioning conditions to the correct table stored to
correspondent_table by delete_single_table branch of the parser.
Aleksey Midenkov
MDEV-40854 Use of uninitialized table_will_be_deleted in federated engine

MemorySanitizer report:

==851408==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 ha_federated::end_bulk_insert() storage/federated/ha_federated.cc:2035:30
    #1 mysql_insert(THD*, ...) sql/sql_insert.cc:1258:11
    ...
  Memory was marked as uninitialized
    #0 __msan_allocated_memory
    #1 my_malloc mysys/my_malloc.c:116:7
SUMMARY: MemorySanitizer: use-of-uninitialized-value ... end_bulk_insert()

table_will_be_deleted is a handler member with no in-class
initializer, and the handler object itself is heap-allocated via
my_malloc(), so it starts out as garbage. It was only ever set in
extra(HA_EXTRA_PREPARE_FOR_DROP) and in external_lock().  For a
TEMPORARY table, statement execution can reach
end_bulk_insert()/write_row(), which reads the flag, without
external_lock() having run first, so the read sees uninitialized
memory.

The fix initializes table_will_be_deleted in reset(), which runs at
the start of every statement regardless of locking path.
Dave Gosselin
MDEV-33616:  Routines of a mixed case database are not listed

At lower_case_table_names=2 this returns nothing.

  CREATE DATABASE Db1;
  CREATE FUNCTION Db1.f1(a INT) RETURNS INT RETURN a;
  SELECT ROUTINE_NAME FROM information_schema.ROUTINES
  WHERE ROUTINE_SCHEMA='Db1';

mysql.proc records the function's database as db1, in lower case.
Creating a routine lower-cases its database name whenever
lower_case_table_names is anything but 0, at sql/sp_head.h:121.  The
datadir, SCHEMATA and DATABASE() all keep Db1.

CALL Db1.f1() still works, because calling a routine lower-cases the
database name too and then searches mysql.proc for db1.  The query
above never lower-cases it.  It searches for Db1, and mysql.proc.db
collates utf8mb3_bin, so the comparison runs byte for byte and no row
matches.

At setting 1 the server lower-cases the filter value as well, at
sql/sql_show.cc:4394, and lower-cases every name it stores, so the
query and the table always agree.  Setting 2 lower-cases the routine's
copy and nothing else.

The fix lower-cases the filter value before the search.

Sorting the same query brings the row back.

  SELECT ROUTINE_NAME FROM information_schema.ROUTINES
  WHERE ROUTINE_SCHEMA='Db1' ORDER BY ROUTINE_NAME;

The sort keeps the filter from reaching that search.  The server reads
all of mysql.proc instead, then applies the WHERE to ROUTINE_SCHEMA,
which compares case insensitively.  That shape answered correctly all
along.

The same search fills PARAMETERS and backs SHOW FUNCTION STATUS, SHOW
PROCEDURE STATUS, SHOW PACKAGE STATUS and SHOW PACKAGE BODY STATUS.
Every one returned nothing for Db1.  mariadb-dump lists routines with
SHOW FUNCTION STATUS WHERE Db=..., at client/mysqldump.cc:2859, which
is the main.mysqldump failure.

Setting 0 keeps Db1 and db1 as two databases holding two routines.  A
case sensitive volume confirms both stay distinct before and after this
change.  beb9a5459d4 (MDEV-20609) added the search in 10.11.1.
main.lowercase_routines runs both query shapes.
Dave Gosselin
MDEV-33616:  Take the read lock many times in perfschema.func_mutex

The wait timer can have a granularity coarser than the time an
uncontended read lock is held, so the recorded duration of one lock can
be zero, which reads back as NULL.  This can cause the test to fail with
a false negative.

Take the lock twenty more times at each measurement point, with the
extra statements silent so the recorded result does not change.  The
mutex part of the test already works this way, since one SELECT
produces ten THR_LOCK::mutex events.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Dave Gosselin
MDEV-41212:  multi_source.status_vars fails on MacOS platform

Replace the two recorded reads of Slave_received_heartbeats with an
assertion that the counter is nonzero.

The counter advances once per heartbeat period for as long as the
connection is running.  The test waited for it to reach 2 and then
read it again in a separate query, so a heartbeat arriving between
those two queries recorded an unexpected value.

The same wait timed out when the counter was already past 2 at the
first poll, so it now accepts any value at or above the target.
Dave Gosselin
MDEV-33616:  Only one of two routines named in a statement is found

With lower_case_table_names 0 the server can have databases Db1 and db1,
each with a function f1.  A single statement naming both databases, like
SELECT Db1.f1(), db1.f1(), reported that db1.f1 does not exist.

The set of routines a statement uses compared its entries without regard
to case.  Only one routine was loaded but the reference to the other
found nothing.  The set now compares its entries exactly, as the routine
cache and the lock manager already do.
Marko Mäkelä
MDEV-41166 --backup --innodb-log-checkpoint-now may copy too much

xtrabackup_backup_func(): Request for a checkpoint synchronously
so that recv_sys.find_checkpoint() will observe the effect.

Reviewed by: Thirunarayanan Balathandayuthapani

(cherry picked from commit 98970415ed7193dad736460a8f3e53b9282d6bd2)
Marko Mäkelä
WIP: log tracking BACKUP SERVER TO ... CONCURRENT (for HAVE_INNODB_PMEM)

backup_sink::id: The thread identifier (0 to CONCURRENT-1)

innodb_backup_checkpoint_pmem(): Copy the old log file.

InnoDB_backup::log_track(), InnoDB_backup::log_track_pmem():
Keep copying the log until we run out of InnoDB data files to copy.

InnoDB_backup::checkpoint_complete_pmem(): Copy the remaining
part of an old log file right before it is being released.

InnoDB_backup::commit(): In log tracking backup, copy the rest of
the HAVE_INNODB_PMEM log.

FIXME: Implement the non-PMEM code path with minimal blocking.
Dave Gosselin
MDEV-33616:  MTR flag to mark tests as incompatible with macOS

Introduces a new MTR include, not_mac.inc, which when included at the
top of a test, prevents that test from running on macOS.

sys_vars.sysvars_readonly_debug is the first user.  It expects the
server to fault when a read only sysvar is written behind the sysvar
interface.  That protection needs the ro_after_init section, which a
linker script places and ld64 has no option to take, so
HAVE_RO_AFTER_INIT stays undefined on macOS.  Without it no variable is
moved into the read only root either, so neither of the two assignments
is refused.
Rex Johnston
MDEV-41228 Test deep clones of Item_cache items

Deep clones of Item items are created in very few places, so most of the
clone code has no test coverage at all, although Parallel Query relies on it
heavily. One of the places where clones are created is the generation of the
key parts for a lookup into a split materialized table, in
TABLE::add_splitting_info_for_key_field().

An Item_cache reaches that place through the IN->EXISTS transformation.
Item_in_optimizer::fix_left() wraps the left expression of the predicate in
an Item_cache, and the equality injected into the subquery refers to it, so
the key field value being cloned is an Item_direct_ref over that cache. Two
more things are needed for the clone to happen: the grouping field the
equality matches has to be the first component of some index of the
underlying table, otherwise it is not among spl_opt_info->spl_fields and the
function returns before cloning, and the subquery must not be converted to a
semi-join. The latter is achieved by the shape of the query, a UNION in the
subquery, rather than by turning optimizer switches off, so that the plan is
the one a user gets with a default optimizer_switch.

Add Item::check_deep_copy(), which validates a clone against its original in
a debug build. It walks both item trees and reports, as notes, whether they
have the same shape with the same Item class at every node, and whether the
clone shares an Item object with the original. Sharing is what distinguishes
a shallow copy from a deep one: an item that is shallow by design, Item_field
for instance, still produces a separate object and only shares a Field, which
is not an Item.

Call it from TABLE::add_splitting_info_for_key_field() under the
"split_materialized_clones" debug flag, which additionally makes the
optimizer use a clone as the value of the generated key part. The clone then
has to work both in the condition pushed into the materialized table and as
the value looked up in the filled table. Note that the clone built for the
pushed condition cannot be reused for this, as it has already been made
dependent on the select that specifies the materialized table.

With the check in place, the clone of the cache turned out not to be a deep
one: every Item_cache_* class implemented deep_copy() as a plain shallow
copy. That is wrong beyond sharing the example item. A cache is filled by the
store()/cache_value() calls of the item that owns it, Item_in_optimizer here,
and nobody does that for a clone, so a clone that inherited the cached value
of the original kept returning that value for the rest of the query.

Implement Item_cache::deep_copy() once for all cache classes instead. The
clone is given an empty cache, so that it computes the value itself out of
the item the value is read from, and a copy of that item. The exception is an
example containing an aggregate or a window function: those are not clonable
yet, as a copy of one shares the per-execution data of the original and both
would free it, so such an example is shared and the check reports the clone
as not fully deep. Item_cache_row is not clonable at all now, as a copy of it
shared the values[] array of element caches with the original.

The test uses a single row in the outer table, because the answer to the same
query with more rows is wrong for an unrelated reason, MDEV-41251: a split
materialized table in a dependently executed subquery is never refilled when
the outer row changes.

This commit was prepared with Claude Code (Opus 5), which located the query
shape that makes the split key part generation clone an Item_cache by
instrumenting TABLE::add_splitting_info_for_key_field() and searching for a
shape that reaches it, wrote Item::check_deep_copy() and the debug hook,
implemented Item_cache::deep_copy(), bisected the crash in Item_sum::cleanup()
that cloning the example item first caused to the aggr and cmp pointers shared
by an Item_sum copy, and separated the remaining wrong result into MDEV-41251
by showing that it survives the clone fix and does not depend on Item_cache.
Aleksey Midenkov
MDEV-40854 Use of uninitialized table_will_be_deleted in federated engine

MemorySanitizer report:

==851408==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 ha_federated::end_bulk_insert() storage/federated/ha_federated.cc:2035:30
    #1 mysql_insert(THD*, ...) sql/sql_insert.cc:1258:11
    ...
  Memory was marked as uninitialized
    #0 __msan_allocated_memory
    #1 my_malloc mysys/my_malloc.c:116:7
SUMMARY: MemorySanitizer: use-of-uninitialized-value ... end_bulk_insert()

table_will_be_deleted is a handler member with no in-class
initializer, and the handler object itself is heap-allocated via
my_malloc(), so it starts out as garbage. It was only ever set in
extra(HA_EXTRA_PREPARE_FOR_DROP) and in external_lock().  For a
TEMPORARY table, statement execution can reach
end_bulk_insert()/write_row(), which reads the flag, without
external_lock() having run first, so the read sees uninitialized
memory.

The fix initializes table_will_be_deleted in reset(), which runs at
the start of every statement regardless of locking path.
Yuchen Pei
Do the same thing as the parent commit to quick_mode_{1,3}
Oleksandr Byelkin
MDEV-41094 KDF() aliases large iteration/width to weak 32-bit values

Item_func_kdf::val_str() read the PBKDF2 iteration count as a signed
64-bit longlong but only rejected values <= 0 before narrowing it to
the 32-bit int expected by PKCS5_PBKDF2_HMAC(). Iteration counts that
differ by 2^32 therefore aliased to the same 32-bit value and derived
identical keys: e.g. 4294968296 silently did the work of 1000. Since
the value is reproducible as (iter mod 2^32), any key derived this way
was already only as strong as the aliased low iteration count, so no
previously-derived ciphertext is orphaned by rejecting the alias now;
it simply reports the weak request instead of silently honouring it.

Item_func_kdf::fix_length_and_dec() had the identical bug for the key
width argument: `key_length= (uint)args[4]->val_int()` narrows before
the range check, and because the result is cached as a constant, the
runtime guard in val_str() (which uses a wider type and is otherwise
safe) is never reached for a constant width argument. This let a
width like 4294967552 silently alias to 256, and let negative widths
alias to a plausible positive one, both without warning.

Both call sites now validate the argument's true 64-bit value before
narrowing, reusing the existing invalid_argument_error() and NULL
result already used for other invalid KDF() arguments.
Oleksandr Byelkin
MDEV-41094 KDF() aliases large iteration/width to weak 32-bit values

Item_func_kdf::val_str() read the PBKDF2 iteration count as a signed
64-bit longlong but only rejected values <= 0 before narrowing it to
the 32-bit int expected by PKCS5_PBKDF2_HMAC(). Iteration counts that
differ by 2^32 therefore aliased to the same 32-bit value and derived
identical keys: e.g. 4294968296 silently did the work of 1000. Since
the value is reproducible as (iter mod 2^32), any key derived this way
was already only as strong as the aliased low iteration count, so no
previously-derived ciphertext is orphaned by rejecting the alias now;
it simply reports the weak request instead of silently honouring it.

Item_func_kdf::fix_length_and_dec() had the identical bug for the key
width argument: `key_length= (uint)args[4]->val_int()` narrows before
the range check, and because the result is cached as a constant, the
runtime guard in val_str() (which uses a wider type and is otherwise
safe) is never reached for a constant width argument. This let a
width like 4294967552 silently alias to 256, and let negative widths
alias to a plausible positive one, both without warning.

Both call sites now validate the argument's true 64-bit value before
narrowing, reusing the existing invalid_argument_error() and NULL
result already used for other invalid KDF() arguments.
Dave Gosselin
MDEV-33616:  Detect select() on macOS

macOS declares select() in sys/select.h, which the HAVE_SELECT probe did
not include.  clang rejects a call to an undeclared function, so the
probe failed and HAVE_SELECT was left undefined.

my_sleep() then took its last fallback, a busy loop on time() that
rounds the requested interval up to a whole second.  Every sub-second
sleep in the server became a one second spin on a CPU, which is what
made rpl.rpl_perfschema_applier_status_by_worker,
rpl.rpl_shutdown_sighup and rpl.rpl_semi_sync_shutdown_await_ack fail.
Oleg Smirnov
MDEV-28869 Show tables removed by table elimination in EXPLAIN

Tables removed by table elimination were omitted from EXPLAIN and
EXPLAIN FORMAT=JSON entirely. A user comparing the query text with
the plan could not tell whether a table had been eliminated or the
optimizer had lost it, and after MDEV-26278 a whole derived table
could disappear the same way.

Such tables are now shown with eliminated in the type column
(in access_type for FORMAT=JSON), and an eliminated derived table
also prints its contents, every table of it marked the same way.
They are printed after the tables that take part in the plan.
Dave Gosselin
MDEV-33616:  Widen the block count filter in the buffer pool resize test

The test replaces the number of buffer pool blocks with a fixed value so
that the message is stable.  The pattern only accepted 5.., and macOS
builds without a futex use SUX_LOCK_GENERIC, which enlarges buf_block_t
enough to bring the count down into 4...
Aleksey Midenkov
Change insert target from t1 to t2 in federated test

Co-authored-by: Copilot Autofix powered by AI <[email protected]>