Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Sergei Golubchik
compiler warning: unused variable
Rudi Heitbaum
[CONC-805] fix assignment discards 'const' qualifier

For ISO C23, the functions bsearch, memchr, strchr, strpbrk, strrchr, strstr, wcschr, wcspbrk, wcsrchr, wcsstr and wmemchr that return pointers into their input arrays now have definitions as macros that return a pointer to a const-qualified type when the input argument is a pointer to a const-qualified type.

Fixes:
../libmariadb/mariadb_lib.c: In function 'mysql_real_connect':
../libmariadb/mariadb_lib.c:1543:20: error: assignment discards 'const' qualifier from pointer target type [-Werror=discarded-qualifiers]
1543 |      (host && (end= strstr(host, "://"))))
      |                    ^
  • cc-x-codbc-windows: 'dojob pwd if '3.4' == '3.4' ls win32/test SET TEST_DSN=master SET TEST_DRIVER=master SET TEST_PORT=3306 SET TEST_SCHEMA=odbcmaster if '3.4' == '3.4' cd win32/test if '3.4' == '3.4' ctest --output-on-failure' failed -  stdio
Aleksey Midenkov
MDEV-37275 Cannot remove default value of NOT NULL column

Run-time has semantics duplication in unireg_check, default_value and
flags, so all three must be in sync before FRM creation. Special
unireg_check values for temporal field types was introduced by
32b28f92980 WL#1266 "Separate auto-set logic from TIMESTAMP type."
Sergei Golubchik
MDEV-36787 Error 153: No savepoint with that name upon ROLLBACK TO SAVEPOINT, assertion failure

InnoDB was rolling back a transaction internally, while
the server thought the transaction stayed open.

this was fixed
in 10.11 by 387fe5ecc3a to rollback the transaction in the server
and in 12.3 by d228f237f27 to not rollback in InnoDB

let's keep 12.3 behavior, update test results to match.
but combine two nearly indentical test cases into one.
Alexey Yurchenko
MDEV-38383 Fix MDEV-38073 MTR test warning

MDEV-38073 MTR test started to fail with a warning after upstream merge
from 11.4 a7528a6190807281d3224e4e67a9b76083a202a6 because THD responsible
for creating SST user became read-only when the server was started with
--transaction-read-only=TRUE.
make sure the readonly flag on THDs created for wsp::thd utility class is
cleared regardless of the --transaction-read-only value as it is intended
only for client-facing THDs.
Vladislav Vaintroub
MDEV-37424 main.connect fails sporadically with a diff

Wait for disconnects to really finish before FLUSH GLOBAL STATUS
and testing max_used_connections.

The finished "disconnect" can't be relied on when testing this
variable, it is decremented after the socket is closed on server.
Sergei Golubchik
Merge branch '11.8' into 12.2
Hemant Dangi
MDL BF-BF conflict on ALTER and INSERT with multi-level foreign key parents

Issue:
On galera write node INSERT statements does not acquire MDL locks on it's all child
tables and thereby wsrep certification keys are also added for limited tables, but
on applier nodes it does acquire MDL locks for all child tables. This can result
into MDL BF-BF conflict on applier node when transactions referring to parent and
child tables are executed concurrently. For example:

Tables with foreign keys: t1<-t2<-t3<-t4
Conflicting transactions: INSERT t1 and DROP TABLE t4

Wsrep certification keys taken on write node:
- for INSERT t1: t1 and t2
- for DROP TABLE t4: t4

On applier node MDL BF-BF conflict happened between two transaction because
MDL locks on t1, t2, t3 and t4 were taken for INSERT t1, which conflicted
with MDL lock on t4 taken by DROP TABLE t4.
The Wsrep certification keys helps in resolving this MDL BF-BF conflict by
prioritizing and scheduling concurrent transactions. But to generate Wsrep
certification keys it needs to open and take MDL locks on all the child tables.

On applier nodes Write_rows event is implicitly a REPLACE, deleting all conflicting
rows which can cause cascading FK actions and locks on foreign key children tables.

Solution:
For Galera applier nodes the Write_rows event is considered pure INSERT
which will never cause cascading FK actions and locks on foreign key children tables.
Marcin Juszkiewicz
mark RISC-V64 as 64-bit architecture

Signed-off-by: Marcin Juszkiewicz <[email protected]>
Dave Gosselin
MDEV-38747:  ASAN errors in Optimizer_hint_parser::Identifier::to_ident_cli

Summary:
A trigger specifying a hint where the hint has a query block name will cause
an ASAN failure because hint resolution occurs after query parsing, not
during query parsing.  The trigger execution logic uses a stack-local
string to hold the query and hint text during parsing.  During trigger
execution, query parsing and query execution happen in different function
contexts, so the query string used during parsing goes out of scope, freeing
its memory.  But as hint resolution occurs after parsing is complete (and
hints merely point into the query string, they don't copy from it), the hints
refer into a deallocated query string upon hint resolution.

Details:
Prior to the commit introducing this bug, hint resolution was done via a call
to `LEX::resolve_optimizer_hints_in_last_select` when parsing the
`query_specification:` grammar rule.  This meant that any string containing
the query (and hints) was in scope for the entire lifetime of query parsing
and hint resolution.

In the patch introducing this bug, `resolve_optimizer_hints_in_last_select`
was replaced with `handle_parsed_optimizer_hints_in_last_select`, changing
the parsing such that it merely cached hints for resolution during query
execution.  Later, after parsing ends and upon query execution,
`mysql_execute_command` calls `LEX::resolve_optimizer_hints` to resolve hints.
When executing a typical SQL command trigger, `sp_lex_instr::parse_expr`
reparses the query associated with the trigger and does so using a stack-local
String variable to hold the query text.  `sp_lex_instr::parse_expr` returns after
query parsing completes but before hint resolution begins.  Since
the string holding the query was stack-local in `sp_lex_instr::parse_expr` and
destroyed when the method returned, the query string (and hints with it) were
deallocated, leading to the ASAN failure on hint resolution.  When executing
the trigger, `sp_instr_stmt::exec_core` calls `mysql_execute_command` which
calls `LEX::resolve_optimizer_hints` to complete hint resolution but the query
string that the hints depends on no longer exists at this point.

As noted, the stack-local `query_string` variable in `sp_lex_inst::parse_expr`
goes out-of-scope and is freed when the `sp_lex_instr::parse_expr` returns.  In
contrast, in the general case, when a `COM_QUERY` is processed during
`dispatch_command`, the query string lives on the `THD` for the lifetime of
the query independent of some particular function's scope.

For triggers, the necessary lifetime of that query string needs to be as long
as `sp_lex_keeper::validate_lex_and_exec_core` which covers both the query
string parsing via `sp_lex_instr::parse_expr` and the procedure's execution
during `reset_lex_and_exec_core`.  Consequently, this patch lifts the
`query_string` buffer up out of `parse_expr` and onto the `sp_lex_instr` itself
which guarantees that its lifetime is as long as the instruction, which also
guarantees the query string's lifetime extends across parsing and execution,
including hint resolution.  This also covers any cases where the trigger is
successfully executed consecutive times but not reparsed between those
executions.

QB_NAME is not the only affected hint kind; hints with some query block
identifier text for the query block, like
```
NO_MERGE(`@select#1`)
```
will also cause the crash while `NO_MERGE()` will not.
Thirunarayanan Balathandayuthapani
MDEV-19574: innodb_stats_method ignored with persistent statistics

Problem:
========
When persistent statistics are enabled (innodb_stats_persistent=ON),
the innodb_stats_method setting was not being properly passed
through the statistics calculation chain. This caused NULL handling
to always use the default behavior, regardless of the
configured stats method.

The issue affected query optimization for queries involving
NULL values, as the statistics collection wasn't respecting
the user's preference for NULL value treatment
(NULLS_EQUAL, NULLS_UNEQUAL, or NULLS_IGNORED).

Solution:
========
1. Passed innodb_stats_method parameter through the
statistics calculation chain:
- dict_stats_scan_page()
- dict_stats_analyze_index_below_cur()
- dict_stats_analyze_index_for_n_prefix()
- dict_stats_analyze_index_level
- dict_stats_analyze_index
- dict_stats_update_persistent
- dict_stats_save

2. Conditionally set n_non_null_key_vals based on the NULLS_IGNORED
method.

3. Introduced IndexLevelStats which is to collect statistics
at a specific B-tree level during index analysis

4. Introduced LeafPageStats which is to collect statistics
for leaf page analysis

5. IndexLevelStats, LeafPageStats replaces multiple individual
parameters in function signatures.

6. Add stats method name to stat_description in case of non
default innodb_stats_method variable value

7. Added the new stat name like n_nonnull_fld01, n_nonull_fld02 etc
with stats description to indicate how many non-nulls value exist
for nth field of the index. This value is properly retrieved and
stored in index statistics in dict_stats_fetch_index_stats_step().

8. When InnoDB scan the leaf page directly, assign leaf page
count as number of pages scanned in case of multi-level index.
For single page indexes, use 1. This change leads to multiple
changes in existing test case.
Sergei Golubchik
update rpm/deb cnf files to 12.3
Georg Richter
Fix for CONC-805

Fix const-correctness in _mariadb_set_conf_option for glibc 2.43 / C23

- Make a writable copy of `config_option` before replacing underscores with dashes
  to avoid modifying a string literal or const memory.
- Addresses build errors caused by glibc 2.43 / ISO C23 changes where functions
  like strchr now return a pointer to a const-qualified type if the input is const.
- Ensures compatibility with strict compiler warnings (-Werror=discarded-qualifiers).

The second part of CONC-805 was fixed by PR-301
bsrikanth-mariadb
MDEV-38701: hook records_in_range, and const tables

When the optimizer_record_context=ON save the following into the trace:
-
1. the records_in_range() call arguments, and its output records count in the
  method ror_scan_selectivity()
2. the const table rows from join_read_const() using an INSERT statement

Also, when the optimizer_replay_context is SET, read the stored
optimizer context from the trace, and do the following: -
1. execute the INSERT statements to add rows into the respective tables,
2. fetch output records count using the arguments to records_in_range() method
  and use them to instead of actually making a call to records_in_range() call
  from ror_scan_selectivity().
Sergei Golubchik
Merge branch '10.6' into 10.11
Monty
MDEV-38246 aria_read index failed on encrypted database during backup

The backup of encrypted Aria tables was not supported.
Added support for this. One complication is that the page checksum is
for the not encrypted page. To be able to verify the checksum I have to
temporarly decrypt the page.
In the backup we store the encrypted pages.

Other things:
- Fixed some (not critical) memory leaks in mariabackup
Sergei Golubchik
MDEV-38744 remove galera dependency from server packages
Sergei Golubchik
MDEV-38246 aria_read index failed on encrypted database during backup

Skip an all-zero pages in the index file.
They can happen normally if the ma_checkpoint_background
thread flushes some later page first (e.g. page 50 before page 48).

Also:
* don't do alloca() in a loop
* correct the check in ma_crypt_index_post_read_hook(),
  the page can be completely full
* compilation failure in ma_open.c:1289:
  comparison is always false due to limited range of data type
Sergei Golubchik
sporadic failures of mdev38431
Sergei Golubchik
Merge branch '11.4' into 11.8
Sergei Golubchik
MDEV-38709 ASAN heap-buffer-overflow in my_convert_using_func

Don't forget up to update stored_rec_length
when extending temp table reclength.

Followup for 4f9a13e9ecf2
Yuchen Pei
MDEV-36230 Fix SERVER port field bound check

The Port field in the system table mysql.servers has type INT,
which translates to Field_long.

During parsing it is parsed as ulong_num, and in this patch we add
bound checks there.
Yuchen Pei
MDEV-24813 Signal full scan to storage engines, with innodb implementation
Marko Mäkelä
MDEV-38589: SELECT unnecessarily waits for log write

The design of "binlog group commit" involves carrying some state across
transaction boundaries. This includes trx_t::commit_lsn, which keeps track
of how much write-ahead log needs to be written. Unfortunately, this
field was not reset in a commit where a log write was elided. That would
cause an unnecessary wait in a subsequent read-only transaction that
happened to reuse the same transaction object.

trx_deregister_from_2pc(): Reset trx->commit_lsn so that
an earlier write that was executed in the same client connection
will not result in an unnecessary wait during a subsequent read
operation.

trx_commit_complete_for_mysql(): Unless we are inside a binlog
group commit, reset trx->commit_lsn.

unlock_and_close_files(): Reset trx->commit_lsn after durably
writing the log, and remove a redundant log write call from some
callers.

trx_t::rollback_finish(): Clear commit_lsn, because a rolled-back
transaction will not need to be durably written.

trx_t::clear_and_free(): Wrapper function to suppress a debug check
in trx_t::free().

Also, remove some redundant ut_ad(!trx->will_lock) that will be checked
in trx_t::free().

Reviewed by: Vladislav Vaintroub
Sergei Golubchik
MDEV-38755 ST_COLLECT(1) IS NULL is false

always evaluate the item before checking null_value
bsrikanth-mariadb
MDEV-35815: use-after-poison_in_get_hash_symbol

In find_field_in_view(), we call field_it.create_item() which
creates item on a statement mem_root.
Then we set its name. Make sure the name is allocated on a statement
mem_root, too.
Thirunarayanan Balathandayuthapani
MDEV-32067 InnoDB linear read ahead had better be logical

The traditional linear read-ahead, enabled by innodb_read_ahead_threshold=56,
only works if pages are allocated on adjacent page numbers, which is not
always the case for B-tree leaf pages.

After this change, the exact nonzero values of
innodb_read_ahead_threshold matter only for the read-ahead of
undo log pages.

Introduced Multi-Range Read (MRR) aware read-ahead that collects
actual leaf page numbers during B-tree traversal

buf_read_ahead_undo(): Renamed from buf_read_ahead_linear().
This function will no longer be invoked on any BLOB pages
(for which FIL_PAGE_PREV and FIL_PAGE_NEXT were not initialized
consistently) nor on any index pages. For index leaf pages,
we will introduce buf_read_ahead_one() and buf_read_ahead_pages().

buf_read_ahead_one(): Read ahead one (sibling leaf) page.
This logic cannot be disabled.

buf_read_ahead_pages(): Read ahead B-tree index leaf pages.

buf_read_ahead_random(): Split the function into two parts: one
that determines which range of pages should be read, and another
that actually initiates a read of the pages.

btr_pcur_move_to_next_page(): Invoke buf_read_ahead_one()
instead of buf_read_ahead_linear().

btr_pcur_move_backward_from_page(): Implement a fast path of
trying to acquire a latch on the previous page without waiting,
and invoke buf_read_ahead_one() on the preceding page, with the
assumption that we may be accessing that page in the near future.

btr_copy_blob_prefix(): Simplify the logic. On other than
ROW_FORMAT=COMPRESSED BLOB pages, the FIL_PAGE_NEXT field is not
meaningfully initialized. The FIL_PAGE_PREV field is not pointing
to anything meaningful either. buf_read_ahead_linear() expects
these to be set meaningfully. Only the non-default setting
innodb_random_read_ahead=ON might be meaningful here.

btr_cur_t::search_leaf(): Add MRR read-ahead context to collect
leaf page numbers at PAGE_LEVEL=1 during B-tree traversal.
The collected page numbers represent actual leaf pages that
will be accessed, enabling more targeted
read-ahead than linear page number assumptions.

mrr_readahead_ctx_t: New structure for passing MRR context
through the call chain from ha_innobase -> row_search_mvcc()
-> btr_pcur_open() -> search_leaf() and it has
READ_AHEAD_PAGES=64 limit.
Thirunarayanan Balathandayuthapani
MDEV-36436  Avoid undo log on insertion of intermediate table during Ignore DDL

Problem:
=========
An assertion failure occurs in InnoDB during consecutive
ALTER TABLE operations using the COPY algorithm. The crash
happens during the table rename phase because the
source table (dict_table_t::n_ref_count) is non-zero, despite
the thread holding an exclusive Metadata Lock (MDL).

Reason:
========
When ALTER IGNORE TABLE is executed via the COPY algorithm,
it generates undo logs for every row inserted into the
intermediate table (e.g., #sql-alter-...). The background Purge
Thread, responsible for cleaning up these undo logs, attempts
to take an MDL on the table to prevent the table from
being dropped while in use.

Race condition:
==================
First ALTER: Creates #sql-alter-, copies data, and renames it to t1.

Purge Activation: The Purge thread picks up the undo logs from step 1.
It takes an MDL on the temporary name (#sql-alter-) and increments
the table's n_ref_count.

Identity Shift: InnoDB renames the physical table object to t1, but
the Purge thread still holds a reference to this object.

Second ALTER: Starts a new copy process. When it attempts to rename
the "new" t1 to a backup name, it checks if n_ref_count == 0.
Because the Purge thread is still "pinning" the object to
clean up logs from the first ALTER, the count is > 0,
triggering the assertion failure.

Solution:
========
ALTER IGNORE TABLE needs row-level undo logging to easily
roll back the last inserted row in case of duplicate key errors.
By discarding the last undo log record after inserting each row,
purge will not process any log records generated by
ALTER IGNORE TABLE, preventing unexpected access from the purge
subsystem during subsequent DDL operations.

Make skip_alter_undo (1-bit) to (2-bit enum)
to support different ALTER operation modes:

- NO_UNDO (0): Normal mode with standard undo logging
- SKIP_UNDO (1): ALTER mode that skips undo logging
- IGNORE_UNDO (2): ALTER IGNORE mode that rewrites undo blocks

trx_undo_report_row_operation(): Add ALTER IGNORE undo
rewriting logic. Store old undo record info before writing
new records for IGNORE_UNDO mode. Reset undo top_offset
to maintain only latest insert undo
Kristian Nielsen
MDEV-38776 [ERROR] Slave worker thread retried transaction 10 time(s) in vain, giving up

Partially revert this commit:

commit 6a1cb449feb1b77e5ec94904c228d7c5477f528a
Author: Sergei Golubchik <[email protected]>
Date:  Mon Jan 18 18:02:16 2021 +0100

    cleanup: remove slave background thread, use handle_manager thread instead

This restores running the parallel replication deadlock killing in its own
dedicated thread, not in the manager thread shared with other unrelated
processing.

When a parallel replication conflict is detected, multiple threads can be
waiting for each other, potentially in a loop. It is critical for
correctness (as well as performance) that the blocking thread is killed
immediately to allow other threads to continue. If one of the threads being
blocked was the manager thread itself in some unrelated job, the kill could
end up being blocked indefinitely, causing replication to hang, usually
eventually timing out on innodb_lock_wait_timeout and failing replication
with an error like:

[ERROR] Slave worker thread retried transaction 10 time(s) in vain, giving up

Signed-off-by: Kristian Nielsen <[email protected]>
Georg Richter
Merge pull request #295 from hrw/3.4

mark RISC-V64 as 64-bit architecture
Sergei Golubchik
bump the VERSION
Sergei Golubchik
MDEV-38710 Assertion is_lock_owner on error returning from auto-create in mysql_admin_table

don't auto-add new partitions if we're already at TIMESTAMP_MAX_VALUE
Sergei Golubchik
fix parts.key_compare_result_on_equal --cursor
Sergei Golubchik
Merge branch '12.2' into 12.3
Sergei Golubchik
Merge branch '10.11' into 11.4
Yuchen Pei
MDEV-24813 Signal full scan to storage engines, with innodb implementation
Brandon Nesterenko
MDEV-25039: MDL BF-BF conflict because of foreign key

Fix rpl suite tests added by MDEV-25039.

rpl_foreign_key_lock_table_insert.test is removed altogether because it
is unclear what the purpose of the test is. The changes of the patch
were done on the slave, yet all operations in the test were done on the
master. Nothing different could happen on the slave because it is
configured to be serial, so all transactions would run sequentially
anyway, and no validations were performed.

rpl_foreign_key_ddl_insert.test was renamed to
rpl_row_foreign_key_mdl.test and the test itself was re-written to be
a minimal test case to ensure that MDL locking behavior is different
pre- and post- patch. A few problems with the original test:
* No foreign-key locking was done on the slave because the table
  engine was not InnoDB.
* rpl_fk_ddl.inc had inconsistent validation checking. I.e., the child
  query validation checks were done on the master (which is incorrect)
  and because the slave was configured to be serial, the two
  transactions could not run concurrently on the slave anyway.
Georg Richter
Merge pull request #301 from heitbaum/patch-1

[CONC-805] fix assignment discards 'const' qualifier
  • cc-x-codbc-windows: 'dojob pwd if '3.4' == '3.4' ls win32/test SET TEST_DSN=master SET TEST_DRIVER=master SET TEST_PORT=3306 SET TEST_SCHEMA=odbcmaster if '3.4' == '3.4' cd win32/test if '3.4' == '3.4' ctest --output-on-failure' failed -  stdio
Sergei Golubchik
MDEV-32317 fix the test for --view
Sergei Golubchik
Merge branch '10.11' into 11.4