Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Kristian Nielsen
Binlog-in-engine: Disable test binlog_in_engine.rpl_gtid_index for Valgrind

The test requires large amounts of CPU and runs too slowly in Valgrind to
make sense, can even occasionally time out on loaded machines.

Signed-off-by: Kristian Nielsen <[email protected]>
Marko Mäkelä
fixup! e3e2274b556dce61585917d298587ba61c2ed084
Marko Mäkelä
Implement 2-file recovery

TODO: Implement multi-file recovery

TODO: Make innodb_log_file_mmap=ON work
Alexander Barkov
MDEV-10152 Add support for TYPE .. IS REF CURSOR

Version#2

In progress
Hemant Dangi
MDL BF-BF conflict on ALTER and INSERT with multi-level foreign key parents

Issue:
On galera write node INSERT statements does not acquire MDL locks on it's all child
tables and thereby wsrep certification keys are also added for limited tables, but
on applier nodes it does acquire MDL locks for all child tables. This can result
into MDL BF-BF conflict on applier node when transactions referring to parent and
child tables are executed concurrently. For example:

Tables with foreign keys: t1<-t2<-t3<-t4
Conflicting transactions: INSERT t1 and DROP TABLE t4

Wsrep certification keys taken on write node:
- for INSERT t1: t1 and t2
- for DROP TABLE t4: t4

On applier node MDL BF-BF conflict happened between two transaction because
MDL locks on t1, t2, t3 and t4 were taken for INSERT t1, which conflicted
with MDL lock on t4 taken by DROP TABLE t4.
The Wsrep certification keys helps in resolving this MDL BF-BF conflict by
prioritizing and scheduling concurrent transactions. But to generate Wsrep
certification keys it needs to open and take MDL locks on all the child tables.

On applier nodes Write_rows event is implicitly a REPLACE, deleting all conflicting
rows which can cause cascading FK actions and locks on foreign key children tables.

Solution:
For Galera applier nodes the Write_rows event is considered pure INSERT
which will never cause cascading FK actions and locks on foreign key children tables.
Rex Johnston
MDEV-38258 No error thrown when CTE columns updated in updates set clause

CTE's are read only i.e It cannot have their columns updated in updates
set clause. Attempting to do so , should throw error
ERROR 1288 (HY000): The target table cte of the UPDATE is not updatable

with cte as (select * from t1 where c < 5)
  update cte set cte.a =(select a from cte);
Here, conversion from a single table to a multi table update bypasses
the normal check by setting the TABLE_LIST object representing cte
to be merge derived.  This is incorrect, so we prohibit this conversion
in TABLE_LIST::init_derived.

Approved by: Sanja Byelkin ([email protected])
Marko Mäkelä
Multi-file checkpoint parsing for pread()

log_t::archived_switch_recovery(): Switch files in the pread() code path

recv_sys_t::find_checkpoint_archived(): For now, return an error
if multi-file recovery would be needed.
Kristian Nielsen
Binlog-in-engine: Fix uninitialized function parameters

Don't pass uninitialized values in function call. MSAN complains about this
(even when the called function never accesses the uninitialized values, and
even when the function is constexpr .oO).

Signed-off-by: Kristian Nielsen <[email protected]>
Rex Johnston
MDEV-38272 Sig11 in LEX::resolve_references_to_cte at sql/sql_cte.cc

Lex::save_list contents remain from a previous query that has invalid
syntax and isn't reset when processing a new query.
We initialize this structure along with it's peer in LEX::start.

MDEV-38295 Recursive CTE usage leaks memory when not used in a top level select

During st_select_lex::cleanup() we assume that leaf tables and any
associated recursive table references can only be present when a join
structure is present.  After MDEV-37220 this is no longer true.
We shift our cleanup routine out of the test for a join.

We add Multidelete_prelocking_strategy::handle_end() in the same manner
as Multiupdate_prelocking_strategy::handle_end() (but with minimal
processing as yet) to populate these leaf_tables in the outermost
select when it is a delete statement.

We add in a call to free_underlaid_joins() to the error processing section
of Sql_cmd_dml::execute to free up temporary tables that were allocated
in prepare, but not freed up when execute is bypassed because an error
occured during prepare.

Sergei Petrunia: Better comments for MDEV-38272, MDEV-38272

Approved by: Sergei Petrunia ([email protected])
Marko Mäkelä
Make remove_file work on Windows
Alexander Barkov
MDEV-38698 mysql_upgrade does not fix charset and collation for mysql.user

In case if the view mysql.user was created (e.g. in 10.6) with a
pre- 18edb0959fe5384673ebd36d72029fa8003ff5d9 server with an unexpected
character_set_client or collation_connection, e.g.
utf8mb3 and utf8mb3_general_ci, mysql_upgrade did not fix it to the
expected latin1 and latin1_swedish_ci.

Since 11.8 this could often lead to "Illegax mix of collations" errors
when querying mysql.user, because since 11.8 the default collation for
utf8mb3 is utf8mb3_uca1400_ai_ci, according to the default
@@character_set_collations. For consistency, it's better to fix the
problem starting from 11.4.

For example:

MariaDB [test]> select user,host,is_role from
    -> mysql.user where is_role='N';
ERROR 1267 (HY000): Illegal mix of collations (utf8mb3_general_ci,COERCIBLE)
                    and (utf8mb3_uca1400_ai_ci,COERCIBLE) for operation '='

Fixing mariadb_system_tables_fix.sql to drop the view if it has non-standard
character set or collations, so it gets recreated again correctly by
the CREATE statement in mariadb_system_tables.sql
Monty
MDEV-38246 aria_read index failed on encrypted database during backup

The backup of encrypted Aria tables was not supported.
Added support for this. One complication is that the page checksum is
for the not encrypted page. To be able to verify the checksum I have to
temporarly decrypt the page.
In the backup we store the encrypted pages.

Other things:
- Fixed some (not critical) memory leaks in mariabackup
Marko Mäkelä
MDEV-38730 innodb_log_file_mmap=ON does not work on AMD64, ARMv8, POWER

log_mmap(): If the MAP_SYNC|MAP_SHARED_VALIDATE operation (PMEM)
failed and the path is not in /dev/shm (which we treat as PMEM),
proceed to try regular MAP_SHARED read-only mapping. This allows
somewhat more efficient crash recovery, basically with an I/O
buffer that is not limited by innodb_log_buffer_size.
Sergei Golubchik
MDEV-38246 aria_read index failed on encrypted database during backup

Skip an all-zero pages in the index file.
They can happen normally if the ma_checkpoint_background
thread flushes some later page first (e.g. page 50 before page 48).

Also:
* don't do alloca() in a loop
* correct the check in ma_crypt_index_post_read_hook(),
  the page can be completely full
* compilation failure in ma_open.c:1289:
  comparison is always false due to limited range of data type
Sergei Golubchik
Merge branch '11.4' into 11.8
Sergei Golubchik
MDEV-38709 ASAN heap-buffer-overflow in my_convert_using_func

Don't forget up to update stored_rec_length
when extending temp table reclength.

Followup for 4f9a13e9ecf2
Yuchen Pei
MDEV-38694 [check-ci] Clean up unfinished spider bg thread in index_end
Oleg Smirnov
MDEV-38728 Improve join size estimation for ref access

When estimating number of rows produced by a join after `ref` access,
the optimizer assumes all driving table values will find matches
in the inner table. This causes overestimation when the driving
table has more distinct values than the inner table's key.

Fix: use number of distinct values (NDV) for columns in the
join predicate to calculate match probability:
  match_prob = min(1.0, NDV(inner) / NDV(driving))
The expected number of records after `ref` access is then multiplied
by match probability to provide more accurate estimate.

Limitations:
- EITS must be available for both columns in the join predicate
- both columns must be real table fields
- only single-column ref access is supported
- only first key part of the inner table's index is used

TODO:
- WHERE filter on the driving table may reduce NDV and affect estimation.
  Currently, it is handled only basically
    (driving_ndv must be <= number of records of current partial join)

This commit overwrites only those test results which have been verified,
i.e. provided better join size estimation. Other failing tests are not
yet verified.
Marko Mäkelä
fixup! 7356c5e209c4fd4bda2a5a1ad3bea26a8cf65d95

Avoid file sharing violation on Windows
Thirunarayanan Balathandayuthapani
MDEV-32067 InnoDB linear read ahead had better be logical

The traditional linear read-ahead, enabled by innodb_read_ahead_threshold=56,
only works if pages are allocated on adjacent page numbers, which is not
always the case for B-tree leaf pages.

After this change, the exact nonzero values of
innodb_read_ahead_threshold matter only for the read-ahead of
undo log pages.

Introduced Multi-Range Read (MRR) aware read-ahead that collects
actual leaf page numbers during B-tree traversal

buf_read_ahead_undo(): Renamed from buf_read_ahead_linear().
This function will no longer be invoked on any BLOB pages
(for which FIL_PAGE_PREV and FIL_PAGE_NEXT were not initialized
consistently) nor on any index pages. For index leaf pages,
we will introduce buf_read_ahead_one() and buf_read_ahead_pages().

buf_read_ahead_one(): Read ahead one (sibling leaf) page.
This logic cannot be disabled.

buf_read_ahead_pages(): Read ahead B-tree index leaf pages.

buf_read_ahead_random(): Split the function into two parts: one
that determines which range of pages should be read, and another
that actually initiates a read of the pages.

btr_pcur_move_to_next_page(): Invoke buf_read_ahead_one()
instead of buf_read_ahead_linear().

btr_pcur_move_backward_from_page(): Implement a fast path of
trying to acquire a latch on the previous page without waiting,
and invoke buf_read_ahead_one() on the preceding page, with the
assumption that we may be accessing that page in the near future.

btr_copy_blob_prefix(): Simplify the logic. On other than
ROW_FORMAT=COMPRESSED BLOB pages, the FIL_PAGE_NEXT field is not
meaningfully initialized. The FIL_PAGE_PREV field is not pointing
to anything meaningful either. buf_read_ahead_linear() expects
these to be set meaningfully. Only the non-default setting
innodb_random_read_ahead=ON might be meaningful here.

btr_cur_t::search_leaf(): Add MRR read-ahead context to collect
leaf page numbers at PAGE_LEVEL=1 during B-tree traversal.
The collected page numbers represent actual leaf pages that
will be accessed, enabling more targeted
read-ahead than linear page number assumptions.

mrr_readahead_ctx_t: New structure for passing MRR context
through the call chain from ha_innobase -> row_search_mvcc()
-> btr_pcur_open() -> search_leaf() and it has
READ_AHEAD_PAGES=64 limit.
Rex Johnston
MDEV-38258 No error thrown when CTE columns updated in updates set clause

CTE's are read only i.e It cannot have their columns updated in updates
set clause. Attempting to do so , should throw error
ERROR 1288 (HY000): The target table cte of the UPDATE is not updatable

with cte as (select * from t1 where c < 5)
  update cte set cte.a =(select a from cte);
Here, conversion from a single table to a multi table update bypasses
the normal check by setting the TABLE_LIST object representing cte
to be merge derived.  This is incorrect, so we prohibit this conversion
in TABLE_LIST::init_derived.

Approved by: Sanja Byelkin ([email protected])
Marko Mäkelä
fixup! eb8948e64c565af73b7dca163a76af7d328f4752
Rex Johnston
MDEV-38272 Sig11 in LEX::resolve_references_to_cte at sql/sql_cte.cc

Lex::save_list contents remain from a previous query that has invalid
syntax and isn't reset when processing a new query.
We initialize this structure along with it's peer in LEX::start.

MDEV-38295 Recursive CTE usage leaks memory when not used in a top level select

During st_select_lex::cleanup() we assume that leaf tables and any
associated recursive table references can only be present when a join
structure is present.  After MDEV-37220 this is no longer true.
We shift our cleanup routine out of the test for a join.

We add Multidelete_prelocking_strategy::handle_end() in the same manner
as Multiupdate_prelocking_strategy::handle_end() (but with minimal
processing as yet) to populate these leaf_tables in the outermost
select when it is a delete statement.

We add in a call to free_underlaid_joins() to the error processing section
of Sql_cmd_dml::execute to free up temporary tables that were allocated
in prepare, but not freed up when execute is bypassed because an error
occured during prepare.

Sergei Petrunia: Better comments for MDEV-38272, MDEV-38272

Approved by: Sergei Petrunia ([email protected])
Rex Johnston
MDEV-38272 Sig11 in LEX::resolve_references_to_cte at sql/sql_cte.cc

Lex::save_list contents remain from a previous query that has invalid
syntax and isn't reset when processing a new query.
We initialize this structure along with it's peer in LEX::start.

MDEV-38295 Recursive CTE usage leaks memory when not used in a top level select

During st_select_lex::cleanup() we assume that leaf tables and any
associated recursive table references can only be present when a join
structure is present.  After MDEV-37220 this is no longer true.
We shift our cleanup routine out of the test for a join.

We add Multidelete_prelocking_strategy::handle_end() in the same manner
as Multiupdate_prelocking_strategy::handle_end() (but with minimal
processing as yet) to populate these leaf_tables in the outermost
select when it is a delete statement.

We add in a call to free_underlaid_joins() to the error processing section
of Sql_cmd_dml::execute to free up temporary tables that were allocated
in prepare, but not freed up when execute is bypassed because an error
occured during prepare.

Sergei Petrunia: Better comments for MDEV-38272, MDEV-38272

Approved by: Sergei Petrunia ([email protected])
Marko Mäkelä
fixup! bb9fa15465cd3f38ac1f9776348638992c86e50f
Sergei Golubchik
MDEV-38710 Assertion is_lock_owner on error returning from auto-create in mysql_admin_table

don't auto-add new partitions if we're already at TIMESTAMP_MAX_VALUE
Yuchen Pei
MDEV-38694 [check-ci] Clean up unfinished spider bg thread in index_end
Brandon Nesterenko
MDEV-25039: MDL BF-BF conflict because of foreign key

Fix rpl suite tests added by MDEV-25039.

rpl_foreign_key_lock_table_insert.test is removed altogether because it
is unclear what the purpose of the test is. The changes of the patch
were done on the slave, yet all operations in the test were done on the
master. Nothing different could happen on the slave because it is
configured to be serial, so all transactions would run sequentially
anyway, and no validations were performed.

rpl_foreign_key_ddl_insert.test was renamed to
rpl_row_foreign_key_mdl.test and the test itself was re-written to be
a minimal test case to ensure that MDL locking behavior is different
pre- and post- patch. A few problems with the original test:
* No foreign-key locking was done on the slave because the table
  engine was not InnoDB.
* rpl_fk_ddl.inc had inconsistent validation checking. I.e., the child
  query validation checks were done on the master (which is incorrect)
  and because the slave was configured to be serial, the two
  transactions could not run concurrently on the slave anyway.
Marko Mäkelä
Fix innodb_log_file_mmap=ON

This was broken in 1a557d087c4977b555f6872850759d41be785a69
Yuchen Pei
MDEV-38694 Do not override false use_parallel when select_lex has aggregate function in determining pre call

TODO: improve commit message
Brandon Nesterenko
MDEV-38716: Server crashes in TABLE::evaluate_update_default_function

This patch fixes two issues:

Problem (1)
A regression from MDEV-36290 would cause the server to crash when one
uses TIMESTAMP or DATETIME types with option ON UPDATE. The root cause
of this is actually from problem (2) described below, MDEV-36290 simply
forced the bug to be revealed.

Problem (2)
After a table is rebuilt (i.e. via ALTER TABLE .. FORCE), any fields
which use ON UPDATE lose the update trigger after the ALTER. This is
because recreated fields (i.e those in alter_info->create_list) had no
checks to ensure their representation in the target table's
default_fields list. The intent is provided by the following code
comment:

/*
  Update the set of auto-update fields to contain only the new fields
  added to the table. Only these fields should be updated automatically.
  Old fields keep their current values, and therefore should not be
  present in the set of autoupdate fields.
*/

However, fields with trigger-based default values were left
unconsidered.

Linking back to problem (1), the crash happened because of the
following MDEV-36290 changes:

diff --git a/sql/sql_table.cc b/sql/sql_table.cc
index 13ed810cac2..5cc74785ef7 100644
--- a/sql/sql_table.cc
+++ b/sql/sql_table.cc
@@ -12737,7 +12737,12 @@ copy_data_between_tables(THD *thd, TABLE *from, TABLE *to,
    }
  }
  if (dfield_ptr)
-    *dfield_ptr= NULL;
+  {
+    if (dfield_ptr == to->default_field)
+      to->default_field= 0; // No default fields left
+    else
+      *dfield_ptr= NULL; // Mark end of default field pointers
+  }

That is, the destination table's default_field becomes nullified when
no default_field changes exist. However, because ON UPDATE
default_fields were not moved to the destination table, the server
would crash with a segfault when updating the records in the
post-ALTER table (in TABLE::evaluate_update_default_function()) because
it would de-reference the nullified pointer in default_field.

The fix for these problems is the same. The default_field information
is copied from these recreated fields when default values are used
later by ON UPDATE options.

Reviewed-by: TODO
Signed-off-by: Brandon Nesterenko <[email protected]>
Thirunarayanan Balathandayuthapani
MDEV-19574: innodb_stats_method ignored with persistent statistics

Problem:
========
When persistent statistics are enabled (innodb_stats_persistent=ON),
the innodb_stats_method setting was not being properly passed
through the statistics calculation chain. This caused NULL handling
to always use the default behavior, regardless of the
configured stats method.

The issue affected query optimization for queries involving
NULL values, as the statistics collection wasn't respecting
the user's preference for NULL value treatment
(NULLS_EQUAL, NULLS_UNEQUAL, or NULLS_IGNORED).

Solution:
========
1. Passed innodb_stats_method parameter through the
statistics calculation chain:
- dict_stats_scan_page()
- dict_stats_analyze_index_below_cur()
- dict_stats_analyze_index_for_n_prefix()
- dict_stats_analyze_index_level
- dict_stats_analyze_index
- dict_stats_update_persistent
- dict_stats_save

2. Conditionally set n_non_null_key_vals based on the NULLS_IGNORED
method.

3. Introduced IndexLevelStats which is to collect statistics
at a specific B-tree level during index analysis

4. Introduced LeafPageStats which is to collect statistics
for leaf page analysis

5. IndexLevelStats, LeafPageStats replaces multiple individual
parameters in function signatures.

6. Add stats method name to stat_description in case of non
default innodb_stats_method variable value

7. When InnoDB scan the leaf page directly, assign leaf page
count as number of pages scanned in case of multi-level index.
For single page indexes, use 1. This change leads to multiple
changes in existing test case.
Kristian Nielsen
Binlog-in-engine: Fix sporadic test failure of binlog_in_engine.purge_locking

The test was for some reason incorrectly doing SHOW BINLOG EVENTS when the
binlogging of the prior event is deliberately non-deterministic in which
binlog file it will appear in, causing test to depend on timing.

Signed-off-by: Kristian Nielsen <[email protected]>
Sergei Golubchik
Merge branch '10.11' into 11.4