Console View
|
Categories: connectors experimental galera main |
|
| connectors | experimental | galera | main | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41385 wsrep_sync_wait has no effect after COM_CHANGE_USER Also reported as: https://github.com/mariadb-corporation/galera/issues/613. THD::cleanup() cleared THD::wsrep_client_thread. Besides connection close, THD::cleanup() is also called from THD::change_user() when handling COM_CHANGE_USER and COM_RESET_CONNECTION. THD::init() does not set the flag back, and the only place that sets it is thd_prepare_connection(). After either command the session was no longer treated as a wsrep client connection: WSREP_CLIENT() returned false for the rest of the session. As a result wsrep_sync_wait was silently ignored, so causal reads could return stale data. Such connections were also skipped by wsrep_close_client_connections() and wsrep_wait_committing_connections_close(). Connection pools issue COM_RESET_CONNECTION whenever a connection is reused, so in practice most pooled sessions were affected. Fix by moving the reset of wsrep_client_thread from THD::cleanup() to end_connection(), after wsrep_close(). The galera.galera_change_user test is extended to do a causal read on a fresh connection, after COM_CHANGE_USER and after COM_RESET_CONNECTION. Each read must increase the provider's wsrep_causal_reads status counter. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40598: Capture sequences used only in column DEFAULT expressions Problem: A sequence referenced only in a column's DEFAULT expression (e.g. "a INT DEFAULT NEXTVAL(s1)") is opened only when a statement evaluates DEFAULT values (INSERT, LOAD DATA, etc). A plain SELECT never opens it, so the sequence never appears in thd->lex->query_tables, and Optimizer_context_recorder:: dump_sql_script() had no way to see it. The dependent table's definition was then captured without the sequence it depends on, making the captured context unusable on replay. TABLE::internal_tables lists these sequences, but its entries' TABLE_LIST::table pointer is set by open_table() and never reset when the statement ends, so it can point to a TABLE that has since been closed, reused or freed. It cannot be trusted as "open now". Fix: dump_sql_script() walks TABLE::internal_tables for each dumped table and calls the new resolve_default_sequence() for each entry. That function never reads the cached TABLE_LIST::table; it uses only the entry's db/table_name to resolve the sequence independently: - Fast path: if the sequence is in thd->open_tables and was used by this statement (TABLE::query_id match, which matters under LOCK TABLES), dump its CREATE SEQUENCE and current value, as for a sequence used directly in the query. - Slow path: otherwise open it in an isolated LEX, statement arena and Open_tables_state (as Table_ident::resolve_table_rowtype_ref() and fill_schema_table_by_open() do), so the running statement's LEX, sroutines list, locks and a prepared statement's permanent arena are not disturbed, and a name that has become a view is handled. The MDL request is non-blocking when the thread already holds locks. Only the CREATE SEQUENCE is dumped here, since the statement never used the sequence. If the open fails (dropped sequence, name now a view, MDL conflict), push the new warning ER_OPT_CONTEXT_SEQUENCE_CAPTURE_FAILED and carry on instead of failing the user's statement. OOM or a killed connection is still returned as an error. dump_sequence_context() now reports through a "newly_dumped" out-parameter whether it emitted the DDL, so the caller adds SETVAL exactly once without repeating the dedup lookup. Sequences are dumped before the dependent table's CREATE TABLE, so replay can recreate both in order. Tests (opt_context_store_ddls.test): INSERT-opened sequence; plain SELECT; SELECT after FLUSH TABLES; LOCK TABLES with the sequence locked but not used (no SETVAL); two sequences in one table; dropped sequence (warning, table still captured); stored function in the query (separate LEX); name replaced by a view; and re-execution as a prepared statement. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40462 Move innodb.innodb-virtual-columns-debug into gcol.innodb_virtual_debug innodb.innodb-virtual-columns-debug contained only the MDEV-17005 test case. It is about InnoDB virtual column templates, so it belongs with the other InnoDB virtual column debug tests in gcol.innodb_virtual_debug. Rationale Adding a separate test file for every single bugfix tends to grow the number of MTR tests by tens of thousands (as there are already 40k MDEV tickets). Each test file carries a fixed overhead that does not depend on its size: before and after every test MTR runs check-testcase to compare the server state, checks the error log for warnings, and may restart the server when the options differ from the previous test. For tiny test files this overhead dominates the time of the test itself, so the total time of MTR runs on buildbot and in local testing grows considerably. Besides that, every test is at least two files (.test and .result, often also .opt or .combinations). Tens of thousands of extra small files bloat the test directories, which slows down MTR test collection and any filesystem traversal of the source tree, and they enlarge the git index and tree objects, which makes git status, checkout, clone and merges between branches slower. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alessandro Vetere
iminelink@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Fix the build with cmake -DWITH_INNODB_AHI=OFF Without BTR_CUR_HASH_ADAPT, buf_block_t::index, btr_search and innodb_adaptive_hash_index_cells do not exist, so the assertion in buf_LRU_truncate_temp() and the derivation of btr_search.n_cells in innodb_init_params() are compiled only with it. mysql-test/collections/broken-no-innodb-ahi.list names the tests that fail in such a build, for --skip-test-list. They need the adaptive hash index variables or counters, or the IS_HASHED column of INFORMATION_SCHEMA.INNODB_BUFFER_PAGE and INNODB_BUFFER_PAGE_LRU, which the build omits together with the sys schema views that read it. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
mariadb-PranavTiwari
pranav.tiwari@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-39343- Added a system variable Grant_tables_intact that tells if tables are intact or not after dump restore. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38758 vcol.vcol_partition_innodb fails on buildbot TABLE_ROWS in INFORMATION_SCHEMA.PARTITIONS for InnoDB is dict_table_t::stat_n_rows. Each INSERT puts the record into the clustered index and only afterwards increments stat_n_rows without any latch. The second INSERT into a fresh partition schedules a background persistent stats recalc, which is not throttled because stats_last_recalc is still 0. If the recalc scans the leaf page after the next INSERT has added its record but before that INSERT increments stat_n_rows, the recalc stores the exact count and the increment then adds one more, so p2 shows 4 rows instead of 3. The fix disables innodb_stats_auto_recalc for the duration of inc/vcol_partition.inc in the InnoDB variant of the test, so stat_n_rows is maintained only by the per-row increments and is exact. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
do_save_blob(): don't store the previous row's BLOB on OOM do_save_blob() ignored the result of copy->tmp.copy(). When the reallocation fails, copy->tmp keeps its old buffer, so the previous row's BLOB value was stored into the destination field. The error is already raised by my_malloc(MY_WME), but end_write() does not check thd->is_error() after copy_fields(), so the stale row was still written to the temporary table. The fix resets the destination field and returns when the copy fails. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Pekka Lampio
pekka.lampio@galeracluster.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-38999 Add results file for the MTR test galera.galera_MDEV-38999 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-29483 Heap-use-after-free (Binary_string::copy()) with window functions JOIN::make_aggr_tables_info(): a query with a window function buffers join rows into a postjoin-aggr temporary table via copy_fields(). BLOB/TEXT fields were copied by Copy_field::do_field_eq(), a raw memcpy of the packed record representation (length + pointer), leaving the tmp table's field pointing at whatever storage backed the source row. Once the underlying handler (e.g. InnoDB) frees or reuses that storage on a later row fetch, any later read of the buffered blob (e.g. Item_field::str_result() via result_field, referenced from a WHERE/HAVING subquery predicate) dereferences a dangling pointer. The fix forces BLOB/TEXT fields to be deep-copied (do_save_blob()) for any query with window functions, the same way simple GROUP BY already does via save_sum_fields. save_sum_fields controls an unrelated decision (whether to defer Item_sum computation instead of wiring an incremental result_field, see the SUM_FUNC_ITEM handling in Create_tmp_table::add_fields()) and must not be conflated with blob-copy safety. Introduce a separate save_blobs flag of JOIN::create_postjoin_aggr_table() and Create_tmp_table that independently controls the Copy_field::set() deep-copy decision. Create_tmp_table: add a save_blobs constructor parameter. The old constructor is kept and sets save_blobs same as save_sum_fields. create_tmp_table(): split into a variant taking a prepared Create_tmp_table, used by JOIN::create_postjoin_aggr_table(). The other create_tmp_table() callers are unchanged and keep save_blobs same as save_sum_fields. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
ParadoxV5
paradox.ver5@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-4632 multi_source.status_vars test fails sporadically in buildbot `Slave_received_heartbeats`’s test for independence between connections relied on timing with seconds-level precision, which is not consistent on a loaded CI. To avoid reliance on timing accuracy, this commit changes the test strategy to instead measure a live result and use that to show the independence. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-29483 Test for wrong result with window function and IN subquery Add a test case to main.win that still returns a wrong result with the previous commit applied. The result file records the current (wrong) output: an empty set. The expected result is one row 'b'. CREATE TABLE t1 (c0 TEXT, c1 INT); INSERT INTO t1 VALUES ('a', 1), ('b', 2); CREATE TABLE t2 (c0 TEXT); INSERT INTO t2 VALUES ('b'); SELECT COALESCE(c0, LAST_VALUE(c0) OVER (ORDER BY c1)) FROM (SELECT c0, c1 FROM t1) dt WHERE c1 = 0 OR c0 IN (SELECT c0 FROM t2); Findings so far: 1. The derived table dt is merged. The outer references to c0 in the WHERE clause and in the select list become Item_direct_view_ref objects pointing to one shared Item_field. 2. The window function forces a post-join aggregation temporary table (JOIN::make_aggr_tables_info() -> create_tmp_table()). create_tmp_field() with modify_item redirects result_field of that shared Item_field to the temporary table field. 3. The WHERE clause is evaluated in evaluate_join_record() for each join row, before the row is written to the temporary table. The IN predicate caches its left expression: Item_in_optimizer::val_bool() -> Item_cache_str::cache_value() -> Item_direct_view_ref::str_result() -> Item_field::str_result(), which reads result_field, not field. 4. So the WHERE clause compares the value copied by copy_fields() for the last row that passed the WHERE clause, not the current row. In the test, row 'a' sees an empty temporary table field and is filtered out, so nothing is copied, and row 'b' sees the same empty field and is filtered out too. 5. The heap-use-after-free from the original report is the same bug with a BLOB column on InnoDB. copy_fields() copied the BLOB with do_field_eq(), which keeps a pointer into the InnoDB row blob heap. InnoDB frees that heap on the next row fetch (row_mysql_prebuilt_free_blob_heap()), and the next WHERE evaluation dereferences it in Binary_string::copy(). Recorded with rr: the only free of the crashing block happens in the t1 scan on the row1 -> row2 transition, and at the crash result_field belongs to the temporary table while field holds row2's own value. 6. The previous commit deep-copies BLOBs for window function queries, which removes the use-after-free but not the stale read. The wrong result is not BLOB-specific and does not depend on InnoDB. 7. With a plain c0 in the select list instead of the COALESCE() expression the result is correct, so the shape of the select list item affects whether result_field is redirected. The real fix belongs to the optimizer layer: either the merged derived table must not share an item between a WHERE condition and a select list item that gets a temporary table result_field, or the temporary table setup must not redirect such an item, or the IN predicate cache must read field when evaluated before the temporary table is filled. MDEV-39868 (commit c0c4b5573f9) is a similar case of merged derived table columns sharing one Item_field. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40598: Capture sequences used only in column DEFAULT expressions Problem: A sequence referenced only in a column's DEFAULT expression (e.g. "a INT DEFAULT NEXTVAL(s1)") is opened only when a statement evaluates DEFAULT values (INSERT, LOAD DATA, etc). A plain SELECT never opens it, so the sequence never appears in thd->lex->query_tables, and Optimizer_context_recorder:: dump_sql_script() had no way to see it. The dependent table's definition was then captured without the sequence it depends on, making the captured context unusable on replay. TABLE::internal_tables lists these sequences, but its entries' TABLE_LIST::table pointer is set by open_table() and never reset when the statement ends, so it can point to a TABLE that has since been closed, reused or freed. It cannot be trusted as "open now". Fix: dump_sql_script() walks TABLE::internal_tables for each dumped table and calls the new resolve_default_sequence() for each entry. That function never reads the cached TABLE_LIST::table; it uses only the entry's db and table name to resolve the sequence independently, skipping sequences that were already dumped: - Fast path: if the sequence is in thd->open_tables and was used by this statement (TABLE::query_id match, which matters under LOCK TABLES), dump its CREATE SEQUENCE and current value, as for a sequence used directly in the query. - Slow path: otherwise open it in an isolated LEX, statement arena and Open_tables_state (arena swap as in fill_schema_table_by_open(), the rest as in Table_ident::resolve_table_rowtype_ref()), so the running statement's LEX, sroutines list, locks and a prepared statement's permanent arena are not disturbed. The MDL request is non-blocking when the thread already holds locks. As the statement never used the sequence, the user's SELECT privilege on it is checked first. Only the CREATE SEQUENCE is dumped, never SETVAL. If the sequence cannot be opened, is not accessible, or the name no longer resolves to a sequence (it was dropped, replaced by a view, ...), the new warning ER_OPT_CONTEXT_SEQUENCE_CAPTURE_FAILED is pushed, in addition to the underlying error turned into a warning, and nothing is captured for it. A kill is left as an error, and OOM makes the capture fail. To share code with the existing dump loop, the "register the name in the dedup hash and emit CREATE DATABASE" step is factored out into register_table_for_dump(), and the SETVAL logic of the loop into dump_sequence_current_value(). Sequences are dumped before the dependent table's CREATE TABLE, so replay can recreate both in order. Tests (opt_context_store_ddls.test): INSERT-opened sequence; plain SELECT; LOCK TABLES with the sequence locked but not used (no SETVAL); a user without privilege on the sequence; two sequences in one table; dropped sequence (warning, table still captured); stored function in the query (separate LEX); name replaced by a view; and re-execution as a prepared statement. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Khaled Riyad
khaled57.dev@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38861 heap-use-after-free in Prepared_statement::execute() DROP PROCEDURE and CREATE OR REPLACE PROCEDURE executed from inside the routine itself removed it from the SP cache. sp_head::destroy() then freed the memory root that the running sp_head, its LEX and its instructions live in, and the caller kept using them. Skip the removal while the routine is being executed. sp_cache_invalidate() above has already bumped the cache version, so the stale entry is removed by the next lookup, after IS_INVOKED has been cleared. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
mariadb-PranavTiwari
pranav.tiwari@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-39343- Added a system variable Grant_tables_intact that tells if tables are intact or not after dump restore. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40462 innodb.innodb-virtual-columns-debug failed in buildbot with wrong result The test sporadically fails because DELETE does not reach the ib_open_after_dict_open sync point: SET debug_sync= "now WAIT_FOR delete_open"; +Warnings: +Warning 1639 debug sync point wait timed out SELECT a FROM t1; a -NULL -NULL The sync point is in ha_innobase::open(), which is called only when a new TABLE is created from the share. After ALTER TABLE ... ALGORITHM=COPY no TABLE for t1 is cached, but t1 has indexed virtual columns, so the purge thread opens it via open_purge_table() and on close_thread_tables() leaves the TABLE in the table cache. If that happens before DELETE, the DELETE reuses the cached TABLE, does not call ha_innobase::open() and completes without signalling delete_open. The fix waits for purge with innodb_max_purge_lag_wait=0 and then does FLUSH TABLES t1 before arming the sync point, so DELETE always opens a new TABLE. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40553 Print GIS ranges in optimizer trace and context Ranges built over GIS (geometry) columns could not be printed in the optimizer trace or the recorded optimizer context: Field_geom printed every key value as the placeholder "unprintable_geometry_value", regardless of whether the index stored the column's raw value (or a prefix of it) or, for a SPATIAL index, its MBR (Minimum Bounding Rectangle). Field::print_key_part_value() now takes an image_type argument (see Field::image_type()) that says which of the two the key holds. For a SPATIAL index (image_type itMBR), Field_geom::print_key_part_value() decodes the four doubles the key stores and prints them as a WKT POLYGON. For every other index (image_type itRAW), the key holds the value or a prefix of it, and it now prints in binary form, the same way Field_blob does, instead of the placeholder. print_mbr_range_operator() prints the spatial relation a GEOM range carries (MBRWITHIN, MBRCONTAINS, MBRINTERSECTS, MBRDISJOINT, MBREQUALS), inverted where needed so the indexed column reads on the left. print_range() and print_key_value() thread the new image_type argument through to Field_geom::print_key_part_value(). Note: this commit deliberately also carries a fix that is not specific to GIS. Writing a test for this surfaced another bug in the code that records/replays the optimizer context, fixed here since a test would otherwise fail for reasons unrelated to GIS: the context literal that dump_sql_script() writes into the recorded replay script (opt_context_store_replay.cc) escaped backslashes SQL-style. That does not round-trip through INFORMATION_SCHEMA.OPTIMIZER_CONTEXT's regexp-based extraction the same way running the recorded script does, so a context extracted that way no longer matched the ranges the optimizer prints. The literal is now written with NO_BACKSLASH_ESCAPES in effect instead, so its text is identical to the JSON it carries; a single quote is written as its JSON escape \u0027, since NO_BACKSLASH_ESCAPES leaves it as the only character that could still end the literal early. The tests that list ranges now read num_rows as bigint unsigned, because MBRDISJOINT records HA_POS_ERROR (18446744073709551615). Tested with main.opt_trace, main.opt_context_store_stats and main.opt_context_replay_basic. The first two share the new GIS range fixture (include/opt_gis_range_print.inc); main.opt_context_replay_basic has its own GIS record/replay cases. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Pekka Lampio
pekka.lampio@galeracluster.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38999: Galera - stale reads in explicit transactions wsrep_must_sync_wait() exempted every statement from its causal wait once a transaction was open, regardless of which wsrep_sync_wait bit it needed - so writes inside BEGIN...COMMIT could run against a stale, not-yet-applied view. Track per-bit which parts of wsrep_sync_wait have already been satisfied earlier in the current transaction, and only skip a statement's wait once its own bit has actually been honoured. Assisted-by Claude AI |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Khaled Riyad
khaled57.dev@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38004 Double free on re-execution of prepared aggregate function Between two executions of a prepared statement, Item_sp::cleanup() freed the stored function's memory root but left the arena's free list pointing into it. That list is filled when the function call returns and the active arena is restored. On the next execution Item_sum_sp::clear() walked the stale list and destroyed already freed items. Free the items before the memory they live in. The sequence is now in Item_sp::free_call_ctx(), so the three places that repeat it cannot drift apart again. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
ParadoxV5
paradox.ver5@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-4632 multi_source.status_vars test fails sporadically in buildbot `Slave_received_heartbeats`’s test for independence between connections relied on timing with second-level precision, which is inconsistent on a loaded CI. To avoid relying on timing accuracy, this commit changes the test strategy to measure a live result and use that to show independence. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
mariadb-PranavTiwari
pranav.tiwari@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-39343 mysql_upgrade tool now uses mariadb_info table and grant_tables_intact variable to decide whether upgrade is needed or not. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40553 Print GIS ranges in optimizer trace and context Ranges built over GIS (geometry) columns could not be printed in the optimizer trace or the recorded optimizer context: Field_geom printed every key value as the placeholder "unprintable_geometry_value", regardless of whether the index stored the column's raw value (or a prefix of it) or, for a SPATIAL index, its MBR (Minimum Bounding Rectangle). Field::print_key_part_value() now takes an image_type argument (see Field::image_type()) that says which of the two the key holds. For a SPATIAL index (image_type itMBR), Field_geom::print_key_part_value() decodes the four doubles the key stores and prints them as a WKT POLYGON. For every other index (image_type itRAW), the key holds the value or a prefix of it, and it now prints in binary form, the same way Field_blob does, instead of the placeholder. print_mbr_range_operator() prints the spatial relation a GEOM range carries (MBRWITHIN, MBRCONTAINS, MBRINTERSECTS, MBRDISJOINT, MBREQUALS), inverted where needed so the indexed column reads on the left. print_range() and print_key_value() thread the new image_type argument through to Field_geom::print_key_part_value(). Writing a test for this surfaced another bug in the code that records/replays the optimizer context, fixed here since a test would otherwise fail for reasons unrelated to GIS: the context literal that dump_sql_script() writes into the recorded replay script (opt_context_store_replay.cc) escaped backslashes SQL-style. That does not round-trip through INFORMATION_SCHEMA.OPTIMIZER_CONTEXT's regexp-based extraction the same way running the recorded script does, so a context extracted that way no longer matched the ranges the optimizer prints. The literal is now written with NO_BACKSLASH_ESCAPES in effect instead, so its text is identical to the JSON it carries; a single quote is written as its JSON escape ', since NO_BACKSLASH_ESCAPES leaves it as the only character that could still end the literal early. The tests that list ranges now read num_rows as bigint unsigned, because MBRDISJOINT records HA_POS_ERROR (18446744073709551615). Tested with main.opt_trace, main.opt_context_store_stats and main.opt_context_replay_basic, which share the new GIS range fixture (include/opt_gis_range_print.inc). Co-Authored-By: Claude Sonnet 5.5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
mariadb-PranavTiwari
pranav.tiwari@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-39343- Added a system variable Grant_tables_intact that tells if tables are intact or not after dump restore. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
mariadb-PranavTiwari
pranav.tiwari@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-39343- Added a system variable Grant_tables_intact that tells if tables are intact or not after dump restore. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
mariadb-PranavTiwari
pranav.tiwari@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-39343- Added a system variable Grant_tables_intact that tells if tables are intact or not after dump restore. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39868 Wrong result with a window fn over merged derived table column Problem: ======== A query with a window function over a column of a merged derived table returns an empty set when another table is joined on a condition over the same column and is accessed with "Range checked for each record": SELECT AVG(subq.c2) OVER (), t2.c1 FROM t1 LEFT JOIN (SELECT * FROM t3) AS subq ON t1.c1 = subq.c1 STRAIGHT_JOIN t2 ON subq.c2 > t2.c1; With derived_merge=on all the references to subq.c2 are Item_direct_view_ref objects sharing one underlying Item_field, because their ref pointers all point into the derived table's field_translation. Item::split_sum_func2() calls real_item() and puts that shared Item_field into the list of the window function's temporary table fields, so create_tmp_field_from_item_field() sets its result_field to a column of the temporary table. Item_field::val_int() reads field, but Item_field::save_in_field() reads result_field, so the two now return different values. The join condition is evaluated through the same Item_field, and the runtime range analysis in Field::get_mm_leaf_int() uses save_in_field_no_warnings(). It reads the still empty temporary table column instead of the value of t3.c2, treats the value as NULL, and builds a SEL_TREE::IMPOSSIBLE. Table t2 then produces no rows. Solution: ========= Do not unwrap Item_direct_view_ref in Item::split_sum_func2(). The wrapper is created per reference and is not shared, so the temporary table field is attached to the wrapper alone and the conditions that refer to the same view column keep reading the base table field. Item_ref::create_tmp_field_ex() already creates the same temporary table field for a view ref over a column, and change_to_use_tmp_fields() already handles REF_ITEM, so no other change is needed. Ref access was never affected: get_store_key() takes real_item()->field explicitly. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alessandro Vetere
iminelink@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-32286 Reuse remembered clustered leaves in secondary-index scans Row_sel_get_clust_rec_for_mysql::operator() descends the clustered B-tree from the root for every row whose clustered-index record a secondary-index scan must read, although consecutive rows land on the same clustered leaf page wherever the secondary order tracks the clustered one. A non-covering scan needs one for every row, a locking read needs one whatever the secondary index holds, because an exclusive select lock type makes ha_innobase::build_template() build its template against the clustered index, and a covering scan needs one for every row of a secondary leaf whose PAGE_MAX_TRX_ID its read view cannot see. ANALYZE FORMAT=JSON charges each descent its full height, and those descents are nearly the whole cost: the secondary index is charged its own descent and one page for each further leaf, and nothing per row, because the position that its cursor holds between two rows is restored optimistically, which latches the leaf again without counting an access. So pages_accessed is the row count times the height of the clustered index, plus a handful: 1000 rows over a 2-level clustered index cost 2006 and 750 rows over a 3-level one cost 2291, where a full table scan of the same data costs 23 and 110. Let a handle remember the clustered leaves that the lookups of one statement reached, and let the next lookup try them before it descends again. The reasoning behind each value and each rejection is in the comments beside it. row0mysql.h defines clust_leaf_hint_slot, which names one leaf: its page number, copies of its first and last user record truncated to the key fields, which bound the key range that the leaf held when it was remembered, the rec_get_offsets() of both, and the dict_index_t::n_core_fields that the copies were made under. A slot of a leaf that had no right sibling names no last key, because every key above the last record of the rightmost leaf still belongs to it. CLUST_LEAF_HINT_SLOTS (4) slots hang off the new row_prebuilt_t::clust_leaf_hint, beside clust_leaf_hint_mru, the most recently used order held as slot numbers, and clust_leaf_hint_n and clust_leaf_hint_miss, the used-slot count and the miss counter. row0sel.cc holds the policy. row_sel_clust_leaf_hint_covers() compares a key against the remembered ranges, so a lookup that no slot can answer costs no buffer pool access and no pages_accessed. row_sel_clust_leaf_hint_search() probes the first slot that covers the key and moves it to the front of the order. row_sel_clust_leaf_hint_remember() records the leaf that a descent landed on, and refreshes the slot of a leaf that is remembered already rather than spend a second one on the same page. Two descents fill no slot: the lookups of a statement before lookup CLUST_LEAF_HINT_MIN_LOOKUPS (4), and a leaf that is the root. row_sel_clust_leaf_hint_armed() stands a scan down once the slots stop paying for themselves: a miss adds CLUST_LEAF_HINT_MISS_WEIGHT (2) to the miss counter and a hit takes one away, so a scan gives the slots up where it answers too little of its lookups to pay for them, CLUST_LEAF_HINT_MAX_MISSES (8) misses with no hit between them still reach the threshold, and one lookup in CLUST_LEAF_HINT_RETRY (1024) starts the count again, so a scan whose order becomes correlated only later recovers. Both halves of the cost stop there, the test of the slots and the copies that refresh them. Row_sel_get_clust_rec_for_mysql::operator() calls all of this in place of its btr_pcur_open_with_no_init(), and only where the adaptive hash index is disabled. A guess of that index lands on the record with no page-local search and no page access to charge, but neither of the two is faster for every shape of index and scan, and a hint that runs first stops that index from adding entries. The adaptive hash index is off by default, so the hints are active in a default configuration. btr0cur.h and btr0cur.cc add btr_cur_t::try_leaf_hint(), a PAGE_CUR_LE, BTR_SEARCH_LEAF search on one named leaf, which starts from the record where the last search of that leaf landed while that record is still valid, and from a binary search of the page otherwise. It acquires the page with buf_page_try_get(): a hint is never derived from a latched parent page, so by the time it is tried it can precede the caller's already-latched secondary-index leaf in the latching order, where a blocking wait can deadlock. It then rejects the page unless the checks that it makes on the latched frame put the match on it. Those checks are the sole authority on the result, so a stale range costs a wasted probe or a needless descent, never a wrong result, and the ranges need no invalidation protocol. A slot also holds a btr_leaf_step, where the last search of its leaf landed: the block, its buf_block_t::modify_clock and the record. A record pointer stays valid while the block holds the same page and the clock has not moved, because eviction, deletion and reorganization advance the clock and an insertion moves no record. While the step is valid, btr_cur_t::try_leaf_hint() starts from that record instead of a binary search of the page. page0cur.cc adds page_cur_search_near(), which compares the key with the record and with one neighbour, and positions the cursor on the record, on the record after it or on the record before it. It refuses any other answer and leaves it to page_cur_search_with_match(). So a scan whose consecutive rows are neighbours in the clustered index finds each row with two comparisons. A scan that reads the clustered leaf in falling key order does the same: a secondary key that falls as the primary key rises, or a secondary index read backward. Where the secondary order tracks the clustered one with gaps between the rows, the step fails, and btr_leaf_step::expect (below) leaves the leaf to the binary search. The step refuses the metadata pseudo-record, which compares below every key on 0 fields, and a record before it that is the infimum. A debug build checks every successful step against the binary search. A failed step costs comparisons whose outcome no branch predictor can guess, so btr_leaf_step::expect gates it. The next search tries the step only where this one landed where a step would have: on the remembered record, or right before or after it. The successor link of the landing record tells the record right before, because finding a predecessor takes a walk of the page directory. So a scan in random order over a few leaves, which the slots answer in full, keeps the binary search and pays only the tests that set expect. row_sel_clust_leaf_hint_remember() sets the step on the record where a descent landed, with expect set. ha_innodb.cc: ha_innobase::reset() zeroes the used-slot count and the miss counter per statement, matching autoinc_last_value. row0mysql.cc: row_prebuilt_free() frees the key buffers that the slots own. innodb.clust_leaf_hint measures pages_accessed over key orders that differ in how closely the secondary order tracks the clustered one, and further tables check query results over the record formats and key shapes that a clustered-index lookup has to read, down to the metadata pseudo-record of instant ALTER TABLE, to leaves that split and merge while a locking read walks them, and to a record that a remembered leaf supplies for a scan that must then rebuild an older version of it. Two of the tables scan a covering index, which reads a clustered record under an exclusive select lock type and under a PAGE_MAX_TRX_ID that the read view cannot see. clust_leaf_hint_off_debug runs the same body with the hints turned off, through a debug switch that returns before a lookup tests or refreshes the slots, so a diff of the two .result files is what the hints save: 2006 to 1031 (2-level clustered index), 2291 to 1011 (3-level), 4006 to 2015 (two interleaved key ranges), 12016 to 9078 (locality in the second half alone) and 20020 to 10045 for a covering scan that FOR UPDATE makes non-covering, where the same scan without FOR UPDATE costs 20 in both files. Two orders with too little locality to pay for the slots give them up early and end within a hundred accesses of the unhinted count: 20020 to 19966 (decorrelated) and 20020 to 19999 (shuffled). Two further tables in innodb.clust_leaf_hint cover the step back. t16 scans a secondary index whose keys fall as the primary key rises, and reads an ascending secondary index backward: 20020 to 10047 for both. t17 runs the inverse scan on ROW_FORMAT=REDUNDANT with an instantly added column. innodb.clust_leaf_hint_backward_debug reaches the two refusals of the step back that no count shows, at the infimum and at the metadata pseudo-record. A lookup reaches either only for a key that left the clustered index while the slot still covers it. The rollback of an insert over a delete-marked record removes the key while purge leaves the secondary entry, and a PAGE_MAX_TRX_ID that the read view cannot see makes the reader look that entry up. A debug switch writes each refusal to the error log, and the case reads both back. innodb.clust_leaf_hint_instant_alter covers the one rejection that no count reaches, of a slot whose keys were copied under another dict_index_t::n_core_fields than the index reports. dict_index_t::clear_instant_alter() is the only writer of that value that a shared metadata lock allows, and it needs the clustered index to lose the last user record of its root page, while no leaf that is the root fills a slot, so the tree has to shrink between the two, which purge does there. The reader therefore reads uncommitted rows at READ UNCOMMITTED and waits in a stored function while a rollback and purge take them away, and one row that arrives above the position it stopped at is the lookup that tests the slots. The rejection leaves nothing that a query can read, so that branch writes the two counts to the error log under a debug switch and the case reads them back with search_pattern_in_file.inc. They are printed and not named in the pattern, so that a run which reaches the branch with other counts, or in the direction where the clear lowers them, is a difference to look at and not a pass. main.rowid_filter_innodb: 90 to 88, and its ahi combination unchanged. A static_assert in Row_sel_get_clust_rec_for_mysql::operator() fails to compile if btr_search.enabled stops being a Boolean that applies to every index, because the test !btr_search.enabled that stands the hints down would then have to become !btr_search.is_enabled(clust_index). |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
mariadb-PranavTiwari
pranav.tiwari@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-39343- Added a system variable Grant_tables_intact that tells if tables are intact or not after dump restore. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41366 Check prefix key match in partition unordered index scan The idea of Case 2 in can_skip_merging_scans is that when key prefix is fixed, and the infix that is also the "partition by range" column, we can scan each partition in order. This relies on accurately returning EOF when a partition has no (more) matching rows. It is possible to have no more matching rows at the first index read of a partition in a LAST_OR_PREV access, in which case we need to check the prefix match to rule out false positives and return EOF correctly. TODO: more test coverage? |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40553 Print GIS ranges in optimizer trace and context Ranges built over GIS (geometry) columns could not be printed in the optimizer trace or the recorded optimizer context: Field_geom printed every key value as the placeholder "unprintable_geometry_value", regardless of whether the index stored the column's raw value (or a prefix of it) or, for a SPATIAL index, its MBR (Minimum Bounding Rectangle). Field::print_key_part_value() now takes an image_type argument (see Field::image_type()) that says which of the two the key holds. For a SPATIAL index (image_type itMBR), Field_geom::print_key_part_value() decodes the four doubles the key stores and prints them as a WKT POLYGON. For every other index (image_type itRAW), the key holds the value or a prefix of it, and it now prints in binary form, the same way Field_blob does, instead of the placeholder. print_mbr_range_operator() prints the spatial relation a GEOM range carries (MBRWITHIN, MBRCONTAINS, MBRINTERSECTS, MBRDISJOINT, MBREQUALS), inverted where needed so the indexed column reads on the left. print_range() and print_key_value() thread the new image_type argument through to Field_geom::print_key_part_value(). Writing a test for this surfaced another bug in the code that records/replays the optimizer context, fixed here since a test would otherwise fail for reasons unrelated to GIS: the context literal that dump_sql_script() writes into the recorded replay script (opt_context_store_replay.cc) escaped backslashes SQL-style. That does not round-trip through INFORMATION_SCHEMA.OPTIMIZER_CONTEXT's regexp-based extraction the same way running the recorded script does, so a context extracted that way no longer matched the ranges the optimizer prints. The literal is now written with NO_BACKSLASH_ESCAPES in effect instead, so its text is identical to the JSON it carries; a single quote is written as its JSON escape ', since NO_BACKSLASH_ESCAPES leaves it as the only character that could still end the literal early. The tests that list ranges now read num_rows as bigint unsigned, because MBRDISJOINT records HA_POS_ERROR (18446744073709551615). Tested with main.opt_trace, main.opt_context_store_stats and main.opt_context_replay_basic, which share the new GIS range fixture (include/opt_gis_range_print.inc). |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Khaled Riyad
khaled57.dev@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38861 heap-use-after-free in Prepared_statement::execute() DROP PROCEDURE and CREATE OR REPLACE PROCEDURE executed from inside the routine itself removed it from the SP cache. sp_head::destroy() then freed the memory root that the running sp_head, its LEX and its instructions live in, and the caller kept using them. Skip the removal while the routine is being executed. sp_cache_invalidate() above has already bumped the cache version, so the stale entry is removed by the next lookup, after IS_INVOKED has been cleared. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41243, MDEV-41273 Don't freeze mem_root after engine pushdown When a statement is pushed down to an engine, the once-per-statement part of the optimization (e.g. the first_cond_optimization part of JOIN::optimize(), or the whole of JOIN::optimize() for a pushed down UNION) is skipped. Re-executing the same prepared statement or stored routine without pushdown then allocated from a mem_root already marked ROOT_FLAG_READ_ONLY, hitting an assertion in alloc_root(). Add LEX::dont_freeze_mem_root, set by the pushdown_handler and derived_handler constructors so it covers every engine implementing them. Prepared_statement::execute_loop(), sp_head::execute() (through sp_head::dont_freeze_mem_root) and the SP instruction reparse path no longer freeze the mem_root of such statements. The flag is only declared and used in PROTECT_STATEMENT_MEMROOT builds. Note this disables the mem_root protection for the statement for good, even if later executions are not pushed down. For a stored routine one pushed down instruction disables it for the whole routine. Add tests for prepared SELECT, UNION, UPDATE and DELETE, prepared EXPLAIN of a pushed UNION, derived tables, stored procedures, functions, triggers, cursors and metadata-invalidation reparse, with pushdown switched on and off in either order. The routine and prepared statement tests check the statements received by the remote server to verify that pushdown actually happened. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Khaled Riyad
khaled57.dev@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38004 Double free on re-execution of prepared aggregate function Between two executions of a prepared statement, Item_sp::cleanup() freed the stored function's memory root but left the arena's free list pointing into it. That list is filled when the function call returns and the active arena is restored. On the next execution Item_sum_sp::clear() walked the stale list and destroyed already freed items. Free the items before the memory they live in. The sequence is now in Item_sp::free_call_ctx(), so the three places that repeat it cannot drift apart again. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41243, MDEV-41273: fedx pushdown assert for UPDATE/DELETE prepared stmts When a statement is pushed down to an engine, JOIN::optimize() skips its once-per-statement work. Re-executing the same prepared statement or stored routine without pushdown then allocated from a mem_root already marked ROOT_FLAG_READ_ONLY, hitting an assertion in alloc_root(). Add LEX::dont_freeze_mem_root, set by the pushdown_handler and derived_handler constructors so it covers every engine implementing them. Prepared_statement::execute_loop(), sp_head::execute() (through sp_head::dont_freeze_mem_root, PROTECT_STATEMENT_MEMROOT builds only) and the SP instruction reparse path no longer freeze the mem_root of such statements. Note this disables the mem_root protection for them. Add tests for prepared statements, stored procedures, functions, triggers, cursors and metadata-invalidation reparse, with pushdown switched on and off in either order. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Georgi (Joro) Kodinov
joro@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| more doxygen formatting added. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
mariadb-PranavTiwari
pranav.tiwari@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-39343- Added a system variable Grant_tables_intact that tells if tables are intact or not after dump restore. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41385 wsrep_sync_wait has no effect after COM_CHANGE_USER Also reported as: https://github.com/mariadb-corporation/galera/issues/613. THD::cleanup() cleared THD::wsrep_client_thread. Besides connection close, THD::cleanup() is also called from THD::change_user() when handling COM_CHANGE_USER and COM_RESET_CONNECTION. THD::init() does not set the flag back, and the only place that sets it is thd_prepare_connection(). After either command the session was no longer treated as a wsrep client connection: WSREP_CLIENT() returned false for the rest of the session. As a result wsrep_sync_wait was silently ignored, so causal reads could return stale data. Such connections were also skipped by wsrep_close_client_connections() and wsrep_wait_committing_connections_close(). Connection pools issue COM_RESET_CONNECTION whenever a connection is reused, so in practice most pooled sessions were affected. Fix by moving the reset of wsrep_client_thread from THD::cleanup() to end_connection(), after wsrep_close(). The galera.galera_change_user test is extended to do a causal read on a fresh connection, after COM_CHANGE_USER and after COM_RESET_CONNECTION. Each read must increase the provider's wsrep_causal_reads status counter. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexander Barkov
bar@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41246 "Illegal mix of collations" on the mysql.user view This SQL script failed: SET NAMES latin1 COLLATE latin1_swedish_ci; CREATE OR REPLACE VIEW v1 AS SELECT 'Y' AS c1; SET NAMES big5 COLLATE big5_chinese_ci; SELECT * FROM v1 WHERE c1='y'; with the following error: ERROR 1267 (HY000): Illegal mix of collations (latin1_swedish_ci,COERCIBLE) and (big5_chinese_ci,COERCIBLE) for operation '=' Note, latin1_swedish_ci and big5_chinese_ci are used here as examples. The error also happened with different collation combinations. Fix main idea: If two collations have equal comparison rules (known as "tailoring") on a given character repertoire, like latin1_swedish_ci and big5_chinese_ci on ASCII letters, then the "Illegal mix of collations" error can be avoided in a comparison operator. We can choose any of the sides as the operation effective collation - the result will be equal. This optimization is not applied when at least one side has an explicit COLLATE clause. Two explicit COLLATE clauses in one comparison are already illegal when the character sets are the same, so for consistency this stays illegal when the character sets differ too. Most important details: - Splitting enum_repertoire_t into smaller subsets, for better repertoire granularity. A variable holding a repertoire value can now have multiple MY_REPERTOIRE_XXX flags set. This patch implements detecting tailoring equality on these repertoires: * MY_REPERTOIRE_ASCII_ALNUM - [A..Z,a..z,0..9]. * MY_REPERTOIRE_ASCII_IDENT - ALNUM + underscore * MY_REPERTOIRE_ASCII - the entire range U+0000..U+007F - Adding a new virtual function "tailoring" in my_collation_handler_st. It returns the tailoring on the given repertoire for the given collation. If cs1->coll->tailoring(cs1, some_repertoire) returns {0,0}, it means the illegal mix optimization cannot be used for this collation on the given repertoire. If these calls: tr1= cs1->coll->tailoring(cs1, some_repertoire); tr2= cs2->coll->tailoring(cs2, some_repertoire); return both non-NULL results and tr1.str==tr2.str, then these collations are equal on the given repertoire and are mutually replaceable for a comparison operator, so "Illegal mix of collations" can be avoided. - Adding a new method DTCollation::aggregate_by_tailoring(). Like the other successful branches of DTCollation::aggregate(), it makes the resulting repertoire cover both sides. - Adding a new flag MY_COLL_ALLOW_BY_TAILORING. It indicates to DTCollation::aggregate() that the illegal mix optimization by repertoire can be used in the given context. MY_COLL_CMP_CONV now includes MY_COLL_ALLOW_BY_TAILORING. Note, only comparison operators pass this flag. Functions returning a string result do not pass this flag, because in operations like CONCAT(a,b) we still need to evaluate precisely the collation of the result - we cannot just choose a collation of one of the sides (even if they are compatible on the given repertoire). This also applies to ExtractValue() and UpdateXML(), which now aggregate their arguments without this flag. - As in my_repertoire_t the value MY_REPERTOIRE_ASCII is now a set of bits rather than a single bit, the way to detect "is only ASCII" repertoires has changed in the code. For example: // repertoire *IS* ascii if (repertoire == MY_REPERTOIRE_ASCII) has changed in multiple places in the code to // repertoire *HAS* only ascii characters if (my_repertoire_is_subset_of(repertoire, MY_REPERTOIRE_ASCII)) The new function my_repertoire_is_subset_of() in m_ctype.h is used for this purpose in C code. In C++ code DTCollation::repertoire_is_subset_of() is used, e.g.: if (collation.repertoire_is_subset_of(MY_REPERTOIRE_ASCII)) - Repertoire of some expressions was adjusted to the new meaning: * Item_null now has MY_REPERTOIRE_NONE (was ASCII). * MY_LOCALE::repertoire() now returns MY_REPERTOIRE_UNICODE30 (was EXTENDED). * Lex_string_with_metadata_st::repertoire(cs) now scans the string contents to detect the actual repertoire. * HEX() now has MY_REPERTOIRE_ASCII_ALNUM. * QUOTE, MAKE_SET, EXPORT_SET, LPAD, RPAD, GROUP_CONCAT, JSON_ARRAY, JSON_OBJECT and JSON_OBJECTAGG add the repertoire of the extra characters they put into the result (quotes, separators, padding, brackets). * LOWER() and UPPER() add the letters of both cases to the repertoire, because the result can contain letters of the opposite case. - The tis620 collations now set CHARSET_INFO::tab_to_uni (was NULL), so my_charset_is_ascii_based() is true for tis620. This changes the result of subselect_extra_no_semijoin from an error to success. - New flags were added for CHARSET_INFO::state * MY_CS_ASCII_BINARY_CI - for simple 8bit case insensitive collations. It means that this collation has no irregularities on the ASCII repertoire. * MY_CS_IDENT_BINARY_CI - for simple 8bit case insensitive collations. It means that this collation has no irregularities on the IDENT repertoire (but can have irregularities say on punctuation). * MY_CS_ASCII_STD_UCA - for UCA collations. It means that a UCA collation does not reorder ASCII characters. - strings/conf_to_src.c was modified to detect and print MY_CS_ASCII_BINARY_CI and MY_CS_IDENT_BINARY_CI flags. - strings/ctype-extra.c was regenerated with new flags. - Adding a number of MTR tests in plugin/func_test/mysql-test/func_test/. They display a tailoring by collation name and repertoire as returned by: cs->coll->tailoring(cs, some_repertoire) A dynamically linked plugin function collation_tailoring() was added for the purpose of these tests. - Adding the test plugin/func_test/mysql-test/func_test/ctype_ldml.test. It displays the tailorings of collations loaded at runtime from Index.xml: simple 8bit collations (flags detected from the weights) and Unicode collations (LDML rules which do or do not reorder ASCII). - Adding a number of MTR tests mysql-test/main/ctype_xxx_tailoring.test They display two-dimensional charts showing which collations are compatible on which repertoires. - Adding a number of MTR tests in the form of the originally reported script for various collations: SELECT Insert_priv FROM mysql.user WHERE Insert_priv='...'; The repeated blocks of queries are shared through mysql-test/include/ ctype_tailoring_0{1,2,3}_mysql_user_Insert_priv*.inc. - Adding tests to ctype_cp932.test checking that ExtractValue() and UpdateXML() still raise "Illegal mix of collations". |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
mariadb-PranavTiwari
pranav.tiwari@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-39343- Added a system variable Grant_tables_intact that tells if tables are intact or not after dump restore. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41366 Check prefix key match in partition unordered index scan The idea of Case 2 in can_skip_merging_scans is that when key prefix is fixed, and the infix that is also the "partition by range" column, we can scan each partition in order. This relies on accurately returning EOF when a partition has no (more) matching rows. It is possible to have no more matching rows at the first index read of a partition in a non-exct access, in which case we need to check the prefix match to rule out false positives and return EOF correctly. Also strengthen the checks in ha_partition::can_skip_merging_scans() so that AFTER_KEY and BEFORE_KEY reads require both prefix and the partitioned column in the keypart_map to ensure correctness. For example, this check disqualifies the index_read_map call in an existing loose index scan testcase, causing the index scan to use both unordered (in read_range_first) and ordered. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39566 crash on a sorted scan of performance_schema.status_by_* The bitmap of materialized rows in PFS_table_context was sized by the live row count of the buffer container, which grows while a scan runs: a sorted scan sizes it twice and asserted when the two sizes differed, and set_item() could write past its end. status_by_account used the thread local key of status_by_host. The allocation used the number of bits in a word as its size in bytes. m_last_item started at 0, so set_item(0) did nothing and rnd_pos() dropped the first row. Size the bitmap by the container maximum, which is fixed at startup; give status_by_account its own key THR_PFS_SBA; allocate sizeof(ulong) per word; start m_last_item at a sentinel. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||