Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Monty
Update backup code to use new my_dir and my_copy interfaces
Oleksandr Byelkin
MDEV-39993 Force InnoDB in the sys routine grants upgrade test

vaintroub (PR #5698): without an explicit engine requirement, this
test could run in a config where InnoDB isn't the checked table's
engine, making mariadb-upgrade's "Checking and upgrading" phase
print misleading "Unknown storage engine 'InnoDB'" / "error :
Corrupt" lines for mysql.innodb_index_stats, mysql.innodb_table_stats
and mysql.transaction_registry, and a spurious "Repairing tables"
section. Add --source include/have_innodb.inc so the test only runs
with InnoDB available, and re-record the now-clean result.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Dave Gosselin
MDEV-33616:  Allocate the recovery buffer from the heap

recv_sys.tmp_buf comes from aligned_malloc() rather than from the large
page allocator.

main.large_pages fails on macOS with "Warning: Memory not freed: 16375"
at shutdown.  recv_sys_t::find_checkpoint() asks for 1048585 bytes,
my_large_malloc() rounds that up to 1064960 and charges the rounded
figure to the server memory accounting, and recv_sys_t::tmp_free()
credits back the 1048585 that was requested.  ut_malloc_dontdump() takes
the size by value, so it has nowhere to report what my_large_malloc()
wrote back.

Only macOS rounds.  my_get_large_page_sizes() has no huge page interface
to consult there, so its fallback branch reports the ordinary page size
of 16384 as the only large page size.  Linux offers 2 MiB and larger,
and my_next_large_page_size() returns only sizes at or below the
request, so this buffer is never rounded there.

The alignment is srv_page_size because recv_dblwr_t::validate_page()
uses the start of the buffer as two page frames.

log_sys.buf and log_sys.flush_buf keep the large page allocator and stay
excluded from core dumps.  They round the same way, so a server started
with --large-pages --innodb-log-buffer-size=2101248 still reports 24576
on macOS.  recv_sys.tmp_buf can now appear in a core dump, holding redo
log records that innodb_encrypt_log has decrypted.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Sergei Golubchik
don't install config.h

high chance of name conflict. not used by any other headers
identical to my_config.h (which is used by other headers), so redundant.
Monty
Fixed that translog_walk_filenames() in Aria properly recognized aria
log filenames.
Vladislav Vaintroub
MDEV-40608 propagate DBUG_OFF, ENABLED_DEBUG_SYNC and SAFE_MUTEX to external plugins

they affect ABI, but aren't in headers, so must be passed separately
Thirunarayanan Balathandayuthapani
MDEV-41207 Assertion `user_table->get_ref_count() == 1' failed during ALTER TABLE conflicting with rollback of recovered transaction

Problem:
=======
A transaction being rolled back during recovery holds LOCK_IX on the
table, and the rollback thread holds a reference on it.
An online ALTER TABLE on that table falls back to acquiring LOCK_S,
which conflicts and fails.
prepare_inplace_alter_table_dict() asserted that the reference
count is 1 before checking whether the table lock was acquired,
so the reference still held by the rollback thread makes the
assertion fail.

Solution:
========
prepare_inplace_alter_table_dict(): Move the assertion after the
table lock error handling, so that it is evaluated only when
the table locks were acquired successfully.
Val Doroshchuk
MDEV-41154: Report an error when the engine fails in ha_rnd_pos

DELETE skips the row without reporting anything when error is returned by handler::ha_rnd_pos()

Report every error except the two that genuinely mean "the row is not there
any more".
Sergei Golubchik
fix mariadb-plugin-defines.cmake for multi-config
Sergei Golubchik
initial support for plugin .tar.gz/.zip metadata
Oleksandr Byelkin
fix flaky spider.quick_mode_0/quick_mode_2 join-plan tests

Spider caches remote table stats (STS) and index cardinality (CRD)
per share, refreshed only every spider_sts_interval/spider_crd_interval
seconds (10s by default). These tests reuse the same tbl_a/tbl_b
table names and srv links as other tests earlier in the suite, so
under heavy parallel CI load the cache can still be within that
window when the test runs, handing the optimizer stale cardinality
and causing it to pick a nested-loop eq_ref plan (one remote
"where pkey = N" lookup per row) instead of the expected
block-nested-loop plan (one bulk "order by pkey" fetch), which is
what the test's general_log assertions check for.

Force synchronous, always-fresh stats for the session before the
join queries so the plan is deterministic regardless of prior test
activity or CI load.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Monty
Fixed that translog_walk_filenames() in Aria properly recognized aria
log filenames.
Thirunarayanan Balathandayuthapani
MDEV-41207 Acquire metadata locks for recovered transaction

Problem:
=======
A transaction being rolled back during recovery holds LOCK_IX on the
table(not the metadata locks), and the rollback thread holds
a reference on it.

An online ALTER TABLE on that table falls back to acquiring LOCK_S,
which conflicts and fails. prepare_inplace_alter_table_dict()
asserted that the reference count is 1 before checking whether
the table lock was acquired, so the reference still held by the
rollback thread makes the assertion fail.

Solution:
========
trx_recovery_thd: A background connection that owns the metadata
locks of the recovered transactions. One connection is shared by
all of recovering transaction.

trx_lists_init_at_db_start(): Create trx_recovery_thd, before any
metadata lock can be acquired.

trx_resurrect_table_locks(): Acquire a shared metadata lock on each
table that the recovered transaction had modified, and remember the
locks in trx_recovery_mdl. Recovered XA PREPARED transactions are
excluded, because they are completed by a user connection.

trx_recovery_mdl: The metadata locks of the recovered transactions,
by transaction. They are kept outside trx_t. Only a recovered
transaction ever has an entry.

trx_recovery_mdl_exists: Whether trx_recovery_mdl is not empty.
It is read for every transaction in trx_t::free(),
so that the map will only be consulted while some recovered
transaction still holds metadata locks. It becomes false as
soon as the rollback of the recovered transactions has been
completed, long before trx_recovery_thd is
destroyed.

trx_t::free(): Release the metadata locks of a recovered transaction,
once its rollback has been completed. This is the only place that
releases them, so that a transaction which was rolled back by
trx_rollback_recovered(false) will not keep its locks until shutdown.

trx_recovery_thd_destroy(): Destroy trx_recovery_thd.
It is invoked by innodb_shutdown() only, after trx_sys.close()
has freed any recovered transaction that was left.

row_undo_mod(): Add the debug injection rollback_wait
Oleksandr Byelkin
Fix cursor protocol
Sergei Golubchik
fix the build for -G "Ninja Multi-Config"
Vladislav Vaintroub
MDEV-40608 build mysqlservices without an embedded CRT requirement

mysqlservices only exposes a thin C API, no CRT state crosses it, so
don't force whatever CRT/config built the server onto a plugin linking
it. Without /Zl, a plugin built in a config with no matching installed
mysqlservices variant (CMake silently substitutes one - verified with
a toy project) gets an ignorable but noisy LNK4098 warning.

Assisted-by: Claude:claude-5-sonnet
Vladislav Vaintroub
MDEV-40608 build mysqlservices without an embedded CRT requirement

mysqlservices only exposes a thin C API, no CRT state crosses it, so
don't force whatever CRT/config built the server onto a plugin linking
it. Without /Zl, a plugin built in a config with no matching installed
mysqlservices variant (CMake silently substitutes one - verified with
a toy project) gets an ignorable but noisy LNK4098 warning.

Assisted-by: Claude:claude-5-sonnet
Dave Gosselin
MDEV-33616:  Skip the redo log upgrade tests without sparse file support

innodb.log_upgrade and innodb.log_upgrade_101_flags build 8GB redo log
files by seeking past the end of an empty file and writing a single
byte.  That needs a filesystem which leaves the skipped range
unallocated.  HFS on macOS allocates every block of it instead, so the
write fails with ENOSPC and the test reports a perl failure.

include/have_sparse_files.inc probes a directory the caller names,
writing one byte 64MB into an empty file there and comparing the
allocated block count against that offset.
Sergei Golubchik
don't INCLUDE(CPack) automatically
Sergei Golubchik
misc
Rex Johnston
MDEV-35168 subselects with outer references to derived tables may be incorrectly evaluated as constant

Subselects with outer references to derived tables may be incorrectly
evaluated as having no table references.  This can lead to these
subselects being marked as constant, leading to an incorrect
result.

During the calculation of the tables used in a subselect, we construct a
table map of outer references in our (not necessarily new) "new_parent"
select.  This is currently done purely by finding Item_fields in our tree
and using the attached table to update our bitmap.  It can be that a
reference to a derived table also needs to have it's table added to this
map.  If the derived table can be null, this is the case.

We add a new processor to our item walk system,
enumerate_table_refs_processor which is defined at this stage only
for Item_direct_view_ref items.
This called alongside enumerate_field_refs_processor in
Item_subselect::recalc_used_tables().

Coverage of the merged_into() function is provided at the end of the
tests in subselect4.test.  Be aware that they are not a regression test,
we could not find anything that provided an incorrect output.
Sergei Golubchik
MDEV-40608 MariaDB-devel is incomplete for plugins

This works on Linux and on Windows, with rpm/deb/tar.gz/zip
installations.

For rpm/deb it just works, for tar.gz/zip there is no
standard location, so one needs to configure plugin with

  -DCMAKE_PREFIX_PATH=/pah/to/mariadb/basedir

after that, `cmake --install .` works too, installing in the same
basedir.

`cmake --build . --target package` works, creating rpm/deb/targz/zip
depending on whether it's Linux or Windows and whether -DRPM or -DDEB
was specified.

* create and install mariadb-plugin-config.cmake
* for now it only supports one plugin per project, error out
  if there are many
* deb: move all headers that plugins need to libmariadb-dev,
  together with libmysqlservices.a. At least until we'll
  create mariadb-plugin-dev. Nobody should need huge
  libmariadbd-dev to develop a plugin
* rpm: all in MariaDB-devel already, no changes here
* install wsrep headers too, THD layout depends on WITH_WSREP
* show DBUG_OFF, ENABLED_DEBUG_SYNC, and SAFE_MUTEX to plugins, same
  reason (it doesn't happen automatically as they're not in my_config.h)
* but don't install config.h - high chance of name conflict with other
  projects and it's an exact copy of my_config.h anyway.
* adjust plugin.cmake to work for external plugins
* move server-internal part to top-level CMakeLists.txt
* remove double-defined macros from unireg.h (the guard doesn't help
  if unireg.h is included first)

ColumnStore, until fixed, needs a backward-compatibility workaround
Sergei Golubchik
initial support for plugin .tar.gz/.zip metadata
Monty
Implement copying of Aria files with reading of big blocks

The new interface for copy transactional Aria files:

Copying of an aria tables starts and end with these calls:
int aria_open_files_for_backup(THD *thd,
                              const char *path, my_bool trans_type,
                              ARIA_BACKUP_CONTEXT *context);
void aria_close_files_for_backup(ARIA_BACKUP_CONTEXT *context);

Reading through an aria file is done wih these calls:
longlong aria_read_index_file(ARIA_BACKUP_CONTEXT *context,
                              uchar *buffer, size_t buff_length);
longlong aria_read_data_file(ARIA_BACKUP_CONTEXT *context,
                            uchar *buffer, size_t buff_length);
One should call both in a loop until the return is <= 0
0 means end of file, a negative value means error.

storage/maria/test_ma_backup.c tests the code and can be used as an example
of how to use the functions.
Monty
fixup!
Thirunarayanan Balathandayuthapani
MDEV-40324 use-of-uninitialized-value after creation of FULLTEXT table failure

Problem:
=======
For fulltext index, row_create_index_for_mysql() calls
fts_create_index_tables(). If creating FTS auxiliary table fails,
error handling performs trx->rollback() of the dictionary
transaction. Rollback removes the parent table from
dictionary cache and frees it. After that,
convert_error_code_to_mysql() reads table->flags after table->heap.
This leads to read of freed memory.

Solution:
========
create_index(): Read table->flags into a local variable before
calling row_create_index_for_mysql()
Sergei Golubchik
MDEV-40608 MariaDB-devel is incomplete for plugins

This works on Linux and on Windows, with rpm/deb/tar.gz/zip
installations.

For rpm/deb it just works, for tar.gz/zip there is no
standard location, so one needs to configure plugin with

  -DCMAKE_PREFIX_PATH=/pah/to/mariadb/basedir

after that, `cmake --install .` works too, installing in the same
basedir.

`cmake --build . --target package` works, creating rpm/deb/targz/zip
depending on whether it's Linux or Windows and whether -DRPM or -DDEB
was specified.

* create and install mariadb-plugin-config.cmake
* deb: move all headers that plugins need to libmariadb-dev,
  together with libmysqlservices.a. At least until we'll
  create mariadb-plugin-dev. Nobody should need huge
  libmariadbd-dev to develop a plugin
* rpm: all in MariaDB-devel already, no changes here
* adjust plugin.cmake to work for external plugins
* move server-internal part to top-level CMakeLists.txt
* remove WITH_WSREP from my_config.h (it upsets external plugins)
* disable DBUG in plugins, can be enabled with -UDBUG_OFF, if needed
* remove double-defined macros from unireg.h (the guard doesn't help
  if unireg.h is included first)

ColumnStore, until fixed, needs a backward-compatibility workaround
Kristian Nielsen
MDEV-40729: Add functionality to verify if a transaction is committed or not after failure

Basic proof-of-concept patch, only implements the basics to put trx_id into
the binlog file and search for it with trx_status().

A test case rpl.rpl_client_trx_id demonstrates the functionality.

Some limitations/considerations from this patch:

There is no support for MARIADB_TRX_IN_PROGRESS (do we want/need such
functionality?).

Based on 10.11, however pushing to stable 10.11 will be controversial as it
changes GTID event format.

Since 10.11 has no GTID indexes, linear scan of at least one entire binlog
file will be required. An in-memory cache may be needed to handle client
reconnect-storm after a crash or network outage that caused many ongoing
commits to fail.

When starting gtid is not specified for trx_status(), it is not possible to
distinguish between MARIADB_TRX_ABORTED or MARIADB_TRX_UNKNOWN. In this
case, we return MARIADB_TRX_ABORTED.

Knowing where to start scanning binlogs is important, for performance (to
avoid scanning _entire_ binlog history), and to distinguish
MARIADB_TRX_ABORTED from MARIADB_TRX_UNKNOWN. It is somewhat tricky
though:

- A GTID position is multi-dimensional. The starting GTID _must_ be with
  the same domain_id as the transaction being searched for, otherwise
  searching on a slave may start too late in the slave's binlog and wrongly
  return MARIADB_TRX_ABORTED for a committed transaction.

- The client will need to obtain a starting GTID for the very first
  transaction done on the connection. One possible way could be to
  SELECT @@GLOBAL.gtid_binlog_pos and pick out the one with the domain_id
  which will be used for subsequent transactions.

Signed-off-by: Kristian Nielsen <[email protected]>
Oleksandr Byelkin
MDEV-35959 fix flaky spider.mdev_35959 test under parallel CI load

The 1-second spider_net_read_timeout is meant to trip during the
CREATE OR REPLACE TABLE ... LIKE t network read, but under heavy
parallel CI load it can instead expire while Spider is still
connecting to the "srv" server, surfacing as
ER_CONNECT_TO_FOREIGN_DATA_SOURCE instead of the already-tolerated
ER_LOCK_WAIT_TIMEOUT/ER_NET_READ_INTERRUPTED. Accept it as well,
consistent with how other spider bugfix tests treat connect-timeout
races.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Dave Gosselin
MDEV-33616:  Match the macOS dlopen error in plugins.multiauth

The client reports why it could not load client_ed25519, and macOS names
every path that dlopen() tried.  Two expressions are added, one for the
chunk that holds the start of that message and one for the chunk that
holds the rest of it.

The line runs to 563 bytes, 52 of prefix and the 511 that the client
error buffer holds, while do_exec() reads the output with fgets() into a
512 byte buffer and runs the replacements on each chunk on its own.  A
long enough vardir therefore splits the line, because the path appears
four times in the dlopen text.  The second chunk is the tail of a path
and carries no colon, where the first chunk keeps the colons of the
mysqltest prefix.  That chunk also holds the only line terminator the
error line gets, so the expression captures the newline and the
replacement puts it back.  A replacement is inserted as written, so a \n
spelled there would reach the output as a backslash and an n.

Both expressions stop at a newline.  reg_replace compiles with
REG_DOTALL, so an unrestricted .* runs past the line terminator whenever
the whole message reaches the replacement in one chunk, and the error
line then joins the line after it.
Sergei Golubchik
fixup: do not remove WITH_WSREP

this creates ABI incompatiility. install wsrep headers instead
Sergei Golubchik
misc fixes

1. set PLUGIN_HEX_VERSION correctly for bundled plugins
2. don't add GenError dependency for external plugins
3. don't change the policy globally
4. only do EXTERNAL_PLUGIN_POST() if EXTERNAL_PLUGIN_PRE() was done
Rucha Deodhar
MDEV-41181: ASAN heap-buffer-overflow after SELECT JSON_SCHEMA_VALID
Dmitry Shulga
MDEV-40072: Server crashes in my_hash_first/name_hash_search after load_system_triggers

Implementation of on startup/shutdown triggers didn't change security context
before running trigger body that resulted in crash in case such triggers
were fired on behalf a regular (not privileged) user. Moreover, on loading
on startup/on shutdown triggers from the table mysql.event the CREATE PROCEDURE
statement is recreated instead of CREATE TRIGGER since system triggers is
considered internally as a stored procedures, that is the method
sp_head::execute_procedure() is invoked on firing triggers ons startup
and shutdown. Additionally, fixed memory leaks in case parse error happens on
running on startup/shutdow triggers. This fix is the follow-up changeset for
the bug report MDEV-40064.
Dave Gosselin
MDEV-33616:  Take the read lock many times in perfschema.func_mutex

The wait timer can have a granularity coarser than the time an
uncontended read lock is held, so the recorded duration of one lock can
be zero, which reads back as NULL.  This can cause the test to fail with
a false negative.

Take the lock twenty more times at each measurement point, with the
extra statements silent so the recorded result does not change.  The
mutex part of the test already works this way, since one SELECT
produces ten THR_LOCK::mutex events.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Sergei Golubchik
fix errmsg-utf8.txt dependencies for Ninja generator

GenError's custom command must specify headers as OUTPUT,
otherwise ninja cannot deduce that mysqld.cc depends on errmsg-utf8.txt

As a bonus, BYPRODUCTS lists generated files for `ninja clean`
Dave Gosselin
MDEV-41212:  multi_source.status_vars fails on MacOS platform

Replace the two recorded reads of Slave_received_heartbeats with an
assertion that the counter is nonzero.

The counter advances once per heartbeat period for as long as the
connection is running.  The test waited for it to reach 2 and then
read it again in a separate query, so a heartbeat arriving between
those two queries recorded an unexpected value.
Sergei Golubchik
debian, again
Monty
Added new mysys functions my_open_dir, my_win_open and improved my_copy

- Improved my_copy() using copy_file_range and memmap
- Improve performance of my_win_open() by taking free entries from a list
  instead of searching after a free space in an array.

New mysys copy functions:

int my_copy_file(File from, File to, myf MyFlags);
int my_copy_file_range(File from, File to, my_off_t start,
                        my_off_t end, myf MyFlags);

Added compatibility functions for easy converting a windows HANDLE to
a File and back:

File my_convert_handle_to_file(my_native_file handle, int oflag);
void my_detach_file(File fd);
which complements the existing my_native_file_handle(fd)

New functions for looping over files in a directory:

MY_NO_CACHE_DIR *my_dir_open();
int my_dir_read_next()
int my_dir_rewind();
int my_dir_close();

Other things
- Fixed #ifdef's in sql_backup.cc to use the new define
  HAVE_COPY_FILE_RANGE
Sergei Golubchik
fix the build for -G "Ninja Multi-Config"