Console View
|
Categories: connectors experimental galera main |
|
| connectors | experimental | galera | main | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Georgi (Joro) Kodinov
joro@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39572: Fix small typos in COMMUNITY_CONTRIBUTIONS.md Fixed some minor header typos in the document. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Daniel Black
daniel@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-40949 Missing space after how-to-produce-a-full-stack-trace-for-mariadb link | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Daniel Bartholomew
db@dbart.us |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge branch 'bb-10.11-bumpversion' of github.com:MariaDB/server into bb-10.11-bumpversion | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40410: Tight innodb_buffer_pool_size_max on ThreadSanitizer bur_pool_t::size_in_bytes_max_default: Define as 0 also on ThreadSanitizer. The symbol __SANITIZE_THREAD__ is predefined starting with Clang 22 or GCC 7 when building with -fsanitize=thread. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hemant Dangi
hemant.dangi@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40501: Assertion `info->type == READ_CACHE || info->type == WRITE_CACHE' failed in reinit_io_cache upon CHANGE MASTER Issue: CHANGE MASTER ... FOR CHANNEL with a channel name within MAX_CONNECTION_NAME can still overflow the OS file name limit once escaped into the relay log file name. Relay_log_info::init() then fails to open the relay log, leaving its index file unopened, but Master_info_index::remove_master_info() unconditionally calls reset_logs() on it during CHANGE MASTER's error cleanup, which hits the assertion in reinit_io_cache(). Solution: Guard the reset_logs() call in remove_master_info() with is_open(), so a relay log that was never opened is never passed to it. Also use MY_SAFE_PATH in open_index_file() so an over-length name fails deterministically instead of silently falling back to a mangled one. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39868 Wrong result with a window fn over merged derived table column Problem: ======== A query with a window function over a column of a merged derived table returns an empty set when another table is joined on a condition over the same column and is accessed with "Range checked for each record": SELECT AVG(subq.c2) OVER (), t2.c1 FROM t1 LEFT JOIN (SELECT * FROM t3) AS subq ON t1.c1 = subq.c1 STRAIGHT_JOIN t2 ON subq.c2 > t2.c1; With derived_merge=on all the references to subq.c2 are Item_direct_view_ref objects sharing one underlying Item_field, because their ref pointers all point into the derived table's field_translation. Item::split_sum_func2() calls real_item() and puts that shared Item_field into the list of the window function's temporary table fields, so create_tmp_field_from_item_field() sets its result_field to a column of the temporary table. Item_field::val_int() reads field, but Item_field::save_in_field() reads result_field, so the two now return different values. The join condition is evaluated through the same Item_field, and the runtime range analysis in Field::get_mm_leaf_int() uses save_in_field_no_warnings(). It reads the still empty temporary table column instead of the value of t3.c2, treats the value as NULL, and builds a SEL_TREE::IMPOSSIBLE. Table t2 then produces no rows. Solution: ========= Do not unwrap Item_direct_view_ref in Item::split_sum_func2(). The wrapper is created per reference and is not shared, so the temporary table field is attached to the wrapper alone and the conditions that refer to the same view column keep reading the base table field. Item_ref::create_tmp_field_ex() already creates the same temporary table field for a view ref over a column, and change_to_use_tmp_fields() already handles REF_ITEM, so no other change is needed. Ref access was never affected: get_store_key() takes real_item()->field explicitly. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Jaeheon Shim
git@jaeheonshim.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40698 Fix ROLLUP query results with empty result set ROLLUP is defined as the UNION of grouping by every prefix of fields in the original GROUP BY list. A ROLLUP query with an empty result set returned zero rows when it should return a summary NULL, since grouping by the empty prefix returns a single summary row. The empty row case is handled in two separate locations. First, if the optimizer is able to determine that no rows will be produced, e.g. due to the table being empty or the WHERE condition resolving to false, JOIN::send_row_on_empty_set is used to determine whether or not to send an empty result row. Therefore, send_row_on_empty_set is modified to include select_lex->olap == ROLLUP_TYPE. Second, it may be the case that the absence of rows is not confirmed until the execution phase. For instance when the WHERE condition is not constant, or in the case of InnoDB where an empty table is not detected during optimization. This is handled in both end_send_group and end_write_group by this expression join->first_record || (end_of_records && !join->group && !join->group_optimized_away) The condition is extracted into need_empty_set_row and a second variable empty_set_send_rollup_total is recorded to prevent running the default rollup_send_data/rollup_write_data on the empty row case. This is because the null summary row is already handled by send_data_with_check. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
yuchen.pei@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Refuse a write a multi-valued index cannot hold the keys of The engine drops a key outside its fulltext token size limits as the row is written, not as it is searched for, so such a row goes missing from the index for good: once the settings are wide again the optimizer uses the index and does not find it. A corrupted index, and the one thing a multi-valued index must never be. The write used to be refused where the internal column that held the keys was computed. There is no such column now, so refuse it at the handler instead -- ha_write_row() and ha_update_row(), before the row reaches the engine, where every writer goes past: DML, LOAD DATA, the row-based replication appliers and the copy an ALTER TABLE makes. Which indexes are affected is settled when the table is opened, because the limits are settings the server reads at startup: TABLE::mvi_unfit_keys, which is empty for all but a handful of tables and is what the write path tests before anything else. What is not settled until the write is whether the statement writes the document such an index reads -- an UPDATE that leaves the base column alone gives the engine no reason to re-read it, so no entry is rewritten and none can go missing. A DELETE only removes entries and is not asked at all. The refusal says why, so what ha_write_row() returns is HA_ERR_GENERIC: "something went wrong", which is also what it means elsewhere for a caller that has already said what. print_error() no longer puts ER_GET_ERRNO on top of a message that is already there. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
KhaledR57
khaled57.dev@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-37167 Nested BEGINs (4600+) cause a segmentation fault Each nested BEGIN adds one sp_pcontext. Two walks over the finished tree recursed once per nesting level and overran the thread stack. ~sp_pcontext() freed its children recursively. sp_head now threads a single linked list through the contexts and frees them iteratively, so teardown depth is constant and a context no longer frees its children. retrieve_field_definitions() descends the children to build the run-time frame. It emits them in run-time offset order, so it stays recursive, but it now checks the stack and returns an error instead of crashing. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Jan Lindström
jan.lindstrom@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41017 : Galera test failure galera.galera_toi_alter_auto_increment New warning was added in MDEV-33660 commit bd127faa. Re-record test result file. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
VasuBhakt
cpswastik31@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40174 Remove unnecessary double parsing JSON document in `json_normalize` `json_normalize`/`json_equals` unnecessary double parsing JSON document Removed the redundant `json_valid_engine` pre-check to eliminate double-parsing, and integrated error handling directly into the normalization engine: * Catch Empty Strings: Return an error directly in `json_normalize_engine` if the root type is `JSON_VALUE_UNINITIALIZED`. * Catch Trailing Garbage on Scalars: Updated `json_norm_build` to enforce a full scan to the end of the document for scalar values. This prevents edge cases (e.g., raw date strings like 2026-07-17...) from being falsely accepted as a valid JSON number without checking the remainder of the string for syntax errors. * Propagate Syntax Errors: Updated `json_normalize_engine` to explicitly check the engine's error flag after the build phase. * Add Edge Case Tests: Added tests in `json_normalize.test` for empty strings, whitespace, and trailing garbage on scalars to ensure correct error generation. Testing: Verified locally using MTR (`main.json_normalize` and `main.json_equals`). Signed-off-by: VasuBhakt <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
yuchen.pei@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Index the base column, not a copy of its keys Patch 5/N of Approach 1TB. WIP note(ycp): this is a squash of five commits A multi-valued index no longer materialises anything. Until now the DDL built a hidden stored column computed by MVI_ENCODE(), holding the encoded elements of the array as a space-separated document, and put a fulltext index over that column: the keys of every row were on disk twice, once in the document the array came from and once in the column built out of it. Now the key part is the base column itself, and the engine tokenizes it with the mvi fulltext parser, which finds the array inside the document and encodes its elements as it goes. Nothing is stored that was not already there. What the index was declared with -- the array, and the datatype its elements are cast to -- has nowhere to live on the key any more, so it lives in the FRM's EXTRA2_MVI_SPEC section, which was written and read back but unused until now. A Key carries it into that section on the way in, and every TABLE parses it back out into a TABLE::mvi_spec of its own on the way out, the way a virtual column expression is parsed, because it is an expression over one column of one TABLE. That is now the only thing that says a key is a multi-valued index: the key definition itself says only that it is a fulltext key over a column, which it has in common with a plain fulltext key over the very same column. Which is why the parser becomes reserved. A key that named it without a declaration behind it would be indistinguishable from a multi-valued index whose declaration went missing from the FRM, and the server could no longer say which of the two it was looking at. So no table definition may name it, and a key of that shape with no entry in the section is an FRM to refuse -- which is what keeps the section checkable at all. The other thing the key parts stopped saying is which array an index is over. Two multi-valued indexes over one column have the same key part whatever they index, so the declaration decides both whether two of them are duplicates of each other, which check_duplicate_key() already asks, and whether an ALTER TABLE leaves an existing one alone, which compare_keys_but_name() asks: an index declared over another array holds other keys and has to be rebuilt. MATCH ... AGAINST against such an index parses now that the key part is a column the user can name. A boolean-mode search for an encoded key is the same search the optimizer builds. A natural-language one reaches the parser in the mode a document being indexed does, so it is read as a document, finds no array in itself and matches nothing. TODO: a write has to be refused when the index has keys the engine's fulltext token size settings drop, or the row goes missing from the index for good. That hung off the computation of the hidden column and there is no such column now; see the TODO in TABLE::update_virtual_fields() and the testcases it takes out of multi_valued_index_token_size.test. MDEV-40168 Rename the base column in the declaration too RENAME COLUMN rewrites the references to the old name in every expression of the table -- the virtual columns, the check constraints, the defaults -- and a multi-valued index used to be renamed along with them, because its declaration was the expression of the internal column that held its keys. That column is gone and the declaration belongs to the key now, so nothing reached it: the new table's FRM kept saying the old name, and opening it failed to resolve the column the index reads. Rename it in the same place and the same way, on the open table, before the key loop hands these Items to the new table's keys. Dropping the base column now drops the index with it, the way dropping a column drops any other index over it, where before the internal column's dependency on it made the DROP an error. There is nothing of the index left to keep either way: the array it indexes is inside that column. MDEV-40168 Adding a multi-valued index in place Adding one used to mean adding a stored column to hold its keys, so it rebuilt the table and ALGORITHM=INPLACE was refused. There is no column to add any more: the keys are read out of one that is already there, so the engine builds the index itself and INPLACE is enough, on a table with rows and on one that already has a fulltext index alike. Adding a column to a table that has one is still not instant, but for another reason now. It is not the stored column that was in the way, it is the fulltext index the multi-valued index is: InnoDB refuses to add a stored column instantly to a table with one of those, see instant_alter_column_possible() and MDEV-17459. Record all three, and take the remarks about the internal column out of the comments that still described it. MDEV-40168 Do not write into the document, and build the index with the declaration Two bugs, both showing up as wrong results in multi_valued_index.test. encode_mvi_key() was writing into the document it was reading. The binary branch points `sorted' at the element instead of copying it, and step 2 then cuts that down to the longest key image an index can hold, which leaves the String shorter than the buffer it was given. String::c_ptr() NUL-terminates in place when there is room past the string, so it put a NUL over the first byte of the element the image does not cover -- any element longer than MVI_KEY_IMAGE_MAX_LEN. That has always been true, and used to be invisible: the document was either the Item's own copy or the literal of a JSON predicate, which is where the warning 4036 that multi_valued_index.test asked about in a TODO came from. It is not invisible any more. The fulltext parser now reads the document out of the row InnoDB is about to store, so the NUL went into the row and stayed there, and the JSON in it no longer parsed. Use ptr(). The other one is that an index built by ALTER TABLE held the words of the document rather than the keys of the array, so it found nothing. The declaration reaches the engine's build through the key definition being built, but what the fulltext parser is handed was being looked up through the temporary TABLE of the altered definition instead. Make one where the declaration itself is made, in mysql_prepare_create_table_finalize(), and let InnoDB take it off the key like the rest of the definition. An index built without it cannot hold the right entries at all, so assert on that rather than let it happen quietly. MDEV-40168 Tell the sort index what it is parsing for The index the fulltext sort builds from is not the index being created: row_merge_create_fts_sort_index() makes one of its own, and row_merge_fts_doc_tokenize() reads the parser out of that. It was given the parser and not what the parser is parsing for, so a multi-valued index built by ALTER TABLE was tokenized by the built-in parser and ended up holding the words of the document instead of the keys of the array -- an index that finds nothing, while the same index populated by DML finds everything. It is the last of the places that carry the parser across without the argument beside it; the others were done when the argument was added to the interface, and this one was not, because nothing set the argument then. Also stop multi_valued_index_parser.test asking which of two fulltext keys over one column a MATCH() searches. Both keys are over that column now and the statement says no more than that, so the answer is not the test's to assert. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Jan Lindström
jan.lindstrom@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40281 : galera.galera_wsrep_new_cluster test failure Rejoining with an emptied datadir requires a full SST, which on a loaded machine does not finish within the 60 seconds galera_wait_ready.inc allowed, aborting the test while the SST was still running. Let the readiness wait take an optional $galera_wait_ready_timeout and give that restart 300 seconds. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-37936 Followup to MDEV-36321: out_rows for GROUP BY: use of item names? MDEV-36321 compared item name strings when looking for key use in items of the select list within a derived table: item_index_in_key() matched a GROUP BY item to a key part by comparing item->name with the key part's field_name. This fails for aliased columns (grp_id_2 AS grp_id), positional GROUP BY on expressions (GROUP BY 2), and selects in a UNION whose columns are named differently, and it can match on a coincidental alias. Here we remove this type of comparison and compare the underlying fields and their position: the GROUP BY item is located in the select list with Item::eq(), and its position there is its field_index in the derived table, which is what the generated key's key_parts refer to. Two further changes to infer_derived_key_statistics(): - The SELECT DISTINCT check required key_parts == item_list.elements. It now requires that every non-const select list item is a key part, which also accepts keys that include a constant column (5 AS c) and rejects keys that omit a varying one. - The GROUP BY check now runs only when the DISTINCT check did not cover the select, so both cannot increment rec_per_key for the same select, while a select that is both DISTINCT and grouped can still be recognised through its GROUP BY list. Not handled: a select list with the same column twice (SELECT a, a AS a2 ... GROUP BY a) matches the first occurrence, so a key on a2 is not recognised. The estimate errs high, which is safe. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40985 row_end access out of bounds in ALTER TABLE ha_innobase_inplace_ctx::create_key_defs(): In the assignment that had been introduced in commit e056efdd6cfa62cc4c978fce5730af0b8d4c3c6b (MDEV-25004), account for virtual columns. Until MDEV-22363 hopefully lands some day, InnoDB maintains two arrays of columns, which complicates the mapping between TABLE_SHARE::fields and dict_table_t::cols. This complication was not accounted for here. Reviewed by: Thirunarayanan Balathandayuthapani |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Jan Lindström
jan.lindstrom@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41028 : Galera appliers deadlock on a foreign key referencing a CHAR column in a multi-byte character set The write set key of a row is built from the MySQL record by wsrep_store_key_val_for_row(), and the key of a foreign key parent row from the InnoDB record by wsrep_rec_get_foreign_key(). A CHAR is not padded the same way in the two formats: the MySQL record pads it to n_chars * mbmaxlen bytes, while InnoDB strips that padding down to, but not below, n_chars bytes. That compares a byte count with a character count, so a value holding a multi byte character and shorter than the column was left with a different number of characters on the two paths, and the keys differed. A child INSERT then had no dependency on its parent row and the appliers ran it in parallel with a change of that very row. The two paths did not agree on the strnxfrm buffer length either, 3072 on one and 3500 on the other. Both now go through wsrep_store_string_key_val(), which brings a CHAR to exactly the number of characters the column holds and always normalizes with WSREP_MAX_SUPPORTED_KEY_LENGTH, so that the key of a column does not depend on how much room the columns before it happened to leave. Only the copy into the caller's buffer is bounded by the space that is left, which also stops wsrep_rec_get_foreign_key() from writing past its key buffer. This changes the write set keys, so it is done from protocol version 5 on and the old encoding is kept below that. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Kristian Nielsen
knielsen@knielsen-hq.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-22848: SET GLOBAL gtid_slave_pos leaves dangling partial transaction When AUTOCOMMIT=0, SET GLOBAL gtid_slave_pos did not properly commit the (full) transaction, leaving the InnoDB hton registrered in the ha_list. This could then later assert when InnoDB was called upon to eg. prepare() a transaction that it does not participate in. This patch makes rpl_slave_state::load() properly commit the (full) transaction to solve the issue. Reviewed-by: Brandon Nesterenko <[email protected]> Reviewed-by: Monty <[email protected]> Signed-off-by: Kristian Nielsen <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Index the base column, not a copy of its keys Patch 5/N of Approach 1TB. WIP note(ycp): this is a squash of five commits A multi-valued index no longer materialises anything. Until now the DDL built a hidden stored column computed by MVI_ENCODE(), holding the encoded elements of the array as a space-separated document, and put a fulltext index over that column: the keys of every row were on disk twice, once in the document the array came from and once in the column built out of it. Now the key part is the base column itself, and the engine tokenizes it with the mvi fulltext parser, which finds the array inside the document and encodes its elements as it goes. Nothing is stored that was not already there. What the index was declared with -- the array, and the datatype its elements are cast to -- has nowhere to live on the key any more, so it lives in the FRM's EXTRA2_MVI_SPEC section, which was written and read back but unused until now. A Key carries it into that section on the way in, and every TABLE parses it back out into a TABLE::mvi_spec of its own on the way out, the way a virtual column expression is parsed, because it is an expression over one column of one TABLE. That is now the only thing that says a key is a multi-valued index: the key definition itself says only that it is a fulltext key over a column, which it has in common with a plain fulltext key over the very same column. Which is why the parser becomes reserved. A key that named it without a declaration behind it would be indistinguishable from a multi-valued index whose declaration went missing from the FRM, and the server could no longer say which of the two it was looking at. So no table definition may name it, and a key of that shape with no entry in the section is an FRM to refuse -- which is what keeps the section checkable at all. The other thing the key parts stopped saying is which array an index is over. Two multi-valued indexes over one column have the same key part whatever they index, so the declaration decides both whether two of them are duplicates of each other, which check_duplicate_key() already asks, and whether an ALTER TABLE leaves an existing one alone, which compare_keys_but_name() asks: an index declared over another array holds other keys and has to be rebuilt. MATCH ... AGAINST against such an index parses now that the key part is a column the user can name. A boolean-mode search for an encoded key is the same search the optimizer builds. A natural-language one reaches the parser in the mode a document being indexed does, so it is read as a document, finds no array in itself and matches nothing. TODO: a write has to be refused when the index has keys the engine's fulltext token size settings drop, or the row goes missing from the index for good. That hung off the computation of the hidden column and there is no such column now; see the TODO in TABLE::update_virtual_fields() and the testcases it takes out of multi_valued_index_token_size.test. MDEV-40168 Rename the base column in the declaration too RENAME COLUMN rewrites the references to the old name in every expression of the table -- the virtual columns, the check constraints, the defaults -- and a multi-valued index used to be renamed along with them, because its declaration was the expression of the internal column that held its keys. That column is gone and the declaration belongs to the key now, so nothing reached it: the new table's FRM kept saying the old name, and opening it failed to resolve the column the index reads. Rename it in the same place and the same way, on the open table, before the key loop hands these Items to the new table's keys. Dropping the base column now drops the index with it, the way dropping a column drops any other index over it, where before the internal column's dependency on it made the DROP an error. There is nothing of the index left to keep either way: the array it indexes is inside that column. MDEV-40168 Adding a multi-valued index in place Adding one used to mean adding a stored column to hold its keys, so it rebuilt the table and ALGORITHM=INPLACE was refused. There is no column to add any more: the keys are read out of one that is already there, so the engine builds the index itself and INPLACE is enough, on a table with rows and on one that already has a fulltext index alike. Adding a column to a table that has one is still not instant, but for another reason now. It is not the stored column that was in the way, it is the fulltext index the multi-valued index is: InnoDB refuses to add a stored column instantly to a table with one of those, see instant_alter_column_possible() and MDEV-17459. Record all three, and take the remarks about the internal column out of the comments that still described it. MDEV-40168 Do not write into the document, and build the index with the declaration Two bugs, both showing up as wrong results in multi_valued_index.test. encode_mvi_key() was writing into the document it was reading. The binary branch points `sorted' at the element instead of copying it, and step 2 then cuts that down to the longest key image an index can hold, which leaves the String shorter than the buffer it was given. String::c_ptr() NUL-terminates in place when there is room past the string, so it put a NUL over the first byte of the element the image does not cover -- any element longer than MVI_KEY_IMAGE_MAX_LEN. That has always been true, and used to be invisible: the document was either the Item's own copy or the literal of a JSON predicate, which is where the warning 4036 that multi_valued_index.test asked about in a TODO came from. It is not invisible any more. The fulltext parser now reads the document out of the row InnoDB is about to store, so the NUL went into the row and stayed there, and the JSON in it no longer parsed. Use ptr(). The other one is that an index built by ALTER TABLE held the words of the document rather than the keys of the array, so it found nothing. The declaration reaches the engine's build through the key definition being built, but what the fulltext parser is handed was being looked up through the temporary TABLE of the altered definition instead. Make one where the declaration itself is made, in mysql_prepare_create_table_finalize(), and let InnoDB take it off the key like the rest of the definition. An index built without it cannot hold the right entries at all, so assert on that rather than let it happen quietly. MDEV-40168 Tell the sort index what it is parsing for The index the fulltext sort builds from is not the index being created: row_merge_create_fts_sort_index() makes one of its own, and row_merge_fts_doc_tokenize() reads the parser out of that. It was given the parser and not what the parser is parsing for, so a multi-valued index built by ALTER TABLE was tokenized by the built-in parser and ended up holding the words of the document instead of the keys of the array -- an index that finds nothing, while the same index populated by DML finds everything. It is the last of the places that carry the parser across without the argument beside it; the others were done when the argument was added to the interface, and this one was not, because nothing set the argument then. Also stop multi_valued_index_parser.test asking which of two fulltext keys over one column a MATCH() searches. Both keys are over that column now and the statement says no more than that, so the answer is not the test's to assert. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Tell two multi-valued indexes apart in check_duplicate_key() WIP note(ycp): this is more of a fix for Approach 1V. A multi-valued index is over an internal column the DDL made up to hold its keys. That column is named after nothing the user wrote and every such index gets one of its own, so the key parts of two multi-valued indexes never match -- not when the two are different indexes, and not when they are the same one either. Two keys over the very same array were not being reported as duplicates, and each of them materialised a column of its own to hold the same keys. What decides whether two of them hold the same keys is what they were declared with, the array and what its elements are encoded as, and nothing else does. So look that up, in the columns of the statement for a key the statement declares and in the FRM's for one the table already had, and compare the declarations in place of the key parts. A key that is not a multi-valued index is compared as before, and one of each is never a duplicate of the other. This is also where the comparison has to happen once the internal column goes away: two multi-valued indexes over one column would then have the same key part as well as the same column, and the declaration would be the only thing left that tells them apart. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thirunarayanan Balathandayuthapani
thiru@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41022 Wildcard term returns no rows when combined with a phrase in boolean mode search Problem: ======== MATCH(..) AGAINST('"zzzz" aut*' IN BOOLEAN MODE) fails to return the rows which match the wildcard term when the matching word is still in the FTS cache. fts_query_phrase_search() sets query->flags to FTS_PHRASE or FTS_PROXIMITY, but never resets it before returning. The subsequent wildcard term of the same query then finds the stale flag in fts_query_cache() and fts_query_difference(), takes the exact word lookup path instead of fts_cache_find_wildcard() and misses the words present in the cache. Solution: ========= - Reset query->flags in fts_query_phrase_search() before returning, so that the phrase or proximity state doesn't leak into the terms evaluated later in the same query. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Disable rpl_parallel_multi_domain_xa MDEV-34104 describes why this test fails. It was filed 2 years ago, but the fix is complex, and we keep this failing test around hurting all other devs. The fix is planned, once finished, we can re-enable this test. Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38918 Make large pages an explicit per-caller opt-in my_large_malloc() attempted large pages whenever --large-pages was enabled, silently rounding the size up and reporting it back via an in/out parameter. ut_malloc_dontdump() never passed that adjusted size on to its own callers (the InnoDB redo log buffer and recv_sys_t::tmp_buf), so freeing later used the original, smaller size, causing the reported "faux memory leak". Only the buffer pool and the MyISAM/Aria key caches are documented to benefit from large pages. Everything else that ended up calling my_large_malloc() only wanted its "do not dump to core" property and picked up large pages as an undocumented side effect; those buffers are also small and sequentially accessed, so they would have gained little from large pages anyway. Add MY_TRY_LARGE_PAGES: my_large_malloc() and my_large_virtual_alloc() now only attempt large pages when a caller passes this flag, instead of always trying whenever the global option is set. Only the buffer pool and the key caches pass it. The redo log buffer, tmp_buf, and row0log.cc's crypt buffers no longer request large pages at all, which removes the size-rounding bug for them without touching that code. my_large_virtual_alloc()'s fallback (no usable large page size) must also return read-write memory right away, like the Windows large-pages fallback already does, since my_virtual_mem_commit() is a no-op for MY_TRY_LARGE_PAGES. my_large_pages_flag is now set once, in my_init_large_pages(), and never changed thereafter, on any platform. Both my_virtual_mem_commit() and my_virtual_mem_decommit() are now no-ops, aside from accounting, whenever large pages are requested. Also fix a broken mtr suppression regex in main.large_pages that would fail the test on Windows. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Add the fulltext parser of a multi-valued index WIP note(ycp): this is a squash of two commits. Patch 4/N of Approach 1TB A multi-valued index is a fulltext index whose tokens are the encoded elements of a JSON array. Today the server encodes them into a hidden column and the engine tokenizes that column's text with the built-in parser. 1TB moves that work into a parser of its own, so that the elements can be read out of the base column and nothing has to be materialised. This is the parser, doing nothing of its own yet. Every mode hands the text to the built-in parser through param->mysql_parse(), which is what the engine would have done with no parser at all, so a fulltext index naming this one behaves exactly as before -- the same rows and the same tokens in the index, which is what multi_valued_index_parser.test checks, on all three engines that honour a parser. What the modes will become is worth saying now, because they are not the same text: MYSQL_FTPARSER_SIMPLE_MODE is a document, the value of the column the index is over, and this is where the array will be walked and encoded once ftparser_arg carries the path and the datatype. Until then the column already holds the keys separated by spaces, which is precisely what the built-in parser splits. The other two modes are a query -- the boolean string the optimizer built, or a phrase being matched -- and those are keys already. There will be nothing to encode there even later: the encoding happens where the query is built, so that the two sides cannot disagree about what a key is. The plugin is MANDATORY, and named "mvi" because that name goes into the FRM of every table using it and cannot change afterwards. Its maturity is STABLE because plugin_add() asserts a mandatory plugin is at least as mature as the server. MDEV-40168 Read the array in the mvi fulltext parser The parser now turns a document into the keys of the index rather than handing it to the built-in parser: it finds the array the index was declared over, walks it with Mvi_array_iterator -- the same iterator MVI_ENCODE and the optimizer use, so that the three cannot disagree about what a key is -- and hands each encoded element to mysql_add_word(). What an index was declared with arrives in MYSQL_FTPARSER_PARAM::ftparser_arg as an Mvi_parser_arg: the path to the array, parsed once, and the datatype its elements are cast to. The parser cannot allocate -- it runs on the engine's threads during a commit or an index build -- and it cannot have per-index state of its own either, so the argument is read-only and everything that changes while a document is read is on the stack of the parse. A query stays with the built-in parser. The optimizer encodes the keys where it builds the query, and boolean mode is the only form it builds, so MYSQL_FTPARSER_FULL_BOOLEAN_INFO is what tells a query from a document. The one case that cannot be told apart is a natural-language query, which comes in the same mode a document being indexed does; it is read as a document, finds no array, and matches nothing. MATCH ... AGAINST on a multi-valued index is not something the optimizer writes, and an empty result is the conservative way for it to go wrong. A key with nothing in ftparser_arg is not a multi-valued index, only a fulltext key that happens to name the parser, and keeps behaving exactly as before. The argument also has to reach an index that ALTER TABLE builds, which does not go through the table the server opens afterwards: carry it on index_def_t next to the parser it belongs with. No DDL produces such a declaration yet, so a debug keyword plants a fixed one -- the array at $.tags, encoded as CHAR -- on any fulltext key that names the parser. That is what the new testcase drives, cross-checking the keys the index ends up holding against what MVI_ENCODE() makes of the same array. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Kristian Nielsen
knielsen@knielsen-hq.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-35691: Invalid access, use-after-free, on rli->description_event_for_exec This commit rewrites the rpl_master_has_bug() mechanism to solve a problem with invalid memory access. The rpl_master_has_bug() mechanism detects certain bugs depending on the master version, and uses that to enable specific work-arounds on the slave. The problem was that rpl_master_has_bug() accessed Relay_log_info::description_event_for_exec that is not valid to access from concurrent parallel replication worker threads, only from the SQL driver thread. Thus it could use the wrong event or access invalid/freed memory. This patch instead computes a bitmask of detected bugs when the SQL driver thread processes the format description event, and reads that bitmask with an atomic load from the worker threads. The bitmask of bugs can only change when the master restarts with a new version, and we do not replicate events concurrently across a format description event from a master restart. Thus, the bitmask is safe to read concurrently from the Relay_log_info object without locking. This also avoids an expensive match of each entry in the bug list against the master server version done for every single call to rpl_master_has_bug(), which could be quite expensive when done eg. per field in row events as in Field_string::compatible_field_size(). Also remove redundant conditional in table_def::compatible_with(). Thanks to Andrei Elkin for the idea to safely read the bitmask concurrently from the Relay_log_info. Reviewed-by: Andrei Elkin <[email protected]> Signed-off-by: Kristian Nielsen <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40454 UBSAN: maria.aria_pack_mdev invalid-shift-exponent Shifting with 64 is no-op in the the code (no ill effects). Added a test to not do anything if shift with 64 would happen. Tested with ma_test_all that test aria_pack. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Daniel Bartholomew
db@dbart.us |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| bump the VERSION | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
ParadoxV5
paradox.ver5@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40996 Support `--sync_with_master 0, $variable` in mysqltest `--sync_with_master` uses `get_string()`, which has `$variable` support, but it only uses the read buffer, which is written with the unexpanded string and not the variable value. Reviewed-by: KhaledR57 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41012 Galera appliers hang with foreign key of types UUID, INET4, INET6 For direct row writes, the certification key for MYSQL_TYPE_STRING and MYSQL_TYPE_VARSTRING is built by collating the column value and taking the collation from Field::charset(). The data types implemented on Field_fbt - UUID, INET6 and INET4 report MYSQL_TYPE_STRING, and their charset() is my_charset_numeric, which is latin1. Their values are however plain binary and accordingly innodb maps them to DATA_FIXBINARY. Their keys were therefore run through latin1_swedish_ci, which folds them. That corrupts the key in two ways: 1. A key mismatch for the same row. The FK constraint's referenced key that is appended for the parent of a child INSERT is built from the InnoDB record and is not collated, and it does not match the primary key appended due to the parent row's direct write. Certification saw no dependency between a child INSERT and a concurrent parent UPDATE, and two appliers could apply them in parallel causing a hang or crash. 2. A key collision between distinct rows. The folding is many to one, so different values collapse onto one key, Certification compares keys byte for byte, so unrelated rows were treated as the same row. Concurrent transactions on them certified as a conflict and one was aborted with ER_LOCK_DEADLOCK. Fix is for wsrep_store_key_val_for_row() to skip the collation for fields that InnoDB stores as binary, using the same condition as get_innobase_type_from_mysql_type(). This is a no-op for the types that worked before. This change requires to bump the application protocol version to level 5. The commit has also two mtr tests for regression testing. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Daniel Bartholomew
db@dbart.us |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| bump the VERSION | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
yuchen.pei@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Index the base column, not a copy of its keys Patch 5/N of Approach 1TB. WIP note(ycp): this is a squash of three commits A multi-valued index no longer materialises anything. Until now the DDL built a hidden stored column computed by MVI_ENCODE(), holding the encoded elements of the array as a space-separated document, and put a fulltext index over that column: the keys of every row were on disk twice, once in the document the array came from and once in the column built out of it. Now the key part is the base column itself, and the engine tokenizes it with the mvi fulltext parser, which finds the array inside the document and encodes its elements as it goes. Nothing is stored that was not already there. What the index was declared with -- the array, and the datatype its elements are cast to -- has nowhere to live on the key any more, so it lives in the FRM's EXTRA2_MVI_SPEC section, which was written and read back but unused until now. A Key carries it into that section on the way in, and every TABLE parses it back out into a TABLE::mvi_spec of its own on the way out, the way a virtual column expression is parsed, because it is an expression over one column of one TABLE. That is now the only thing that says a key is a multi-valued index: the key definition itself says only that it is a fulltext key over a column, which it has in common with a plain fulltext key over the very same column. Which is why the parser becomes reserved. A key that named it without a declaration behind it would be indistinguishable from a multi-valued index whose declaration went missing from the FRM, and the server could no longer say which of the two it was looking at. So no table definition may name it, and a key of that shape with no entry in the section is an FRM to refuse -- which is what keeps the section checkable at all. The other thing the key parts stopped saying is which array an index is over. Two multi-valued indexes over one column have the same key part whatever they index, so the declaration decides both whether two of them are duplicates of each other, which check_duplicate_key() already asks, and whether an ALTER TABLE leaves an existing one alone, which compare_keys_but_name() asks: an index declared over another array holds other keys and has to be rebuilt. MATCH ... AGAINST against such an index parses now that the key part is a column the user can name. A boolean-mode search for an encoded key is the same search the optimizer builds. A natural-language one reaches the parser in the mode a document being indexed does, so it is read as a document, finds no array in itself and matches nothing. TODO: a write has to be refused when the index has keys the engine's fulltext token size settings drop, or the row goes missing from the index for good. That hung off the computation of the hidden column and there is no such column now; see the TODO in TABLE::update_virtual_fields() and the testcases it takes out of multi_valued_index_token_size.test. Co-Authored-By: Claude Opus 5 <[email protected]> MDEV-40168 Rename the base column in the declaration too RENAME COLUMN rewrites the references to the old name in every expression of the table -- the virtual columns, the check constraints, the defaults -- and a multi-valued index used to be renamed along with them, because its declaration was the expression of the internal column that held its keys. That column is gone and the declaration belongs to the key now, so nothing reached it: the new table's FRM kept saying the old name, and opening it failed to resolve the column the index reads. Rename it in the same place and the same way, on the open table, before the key loop hands these Items to the new table's keys. Dropping the base column now drops the index with it, the way dropping a column drops any other index over it, where before the internal column's dependency on it made the DROP an error. There is nothing of the index left to keep either way: the array it indexes is inside that column. Co-Authored-By: Claude Opus 5 <[email protected]> MDEV-40168 Adding a multi-valued index in place Adding one used to mean adding a stored column to hold its keys, so it rebuilt the table and ALGORITHM=INPLACE was refused. There is no column to add any more: the keys are read out of one that is already there, so the engine builds the index itself and INPLACE is enough, on a table with rows and on one that already has a fulltext index alike. Adding a column to a table that has one is still not instant, but for another reason now. It is not the stored column that was in the way, it is the fulltext index the multi-valued index is: InnoDB refuses to add a stored column instantly to a table with one of those, see instant_alter_column_possible() and MDEV-17459. Record all three, and take the remarks about the internal column out of the comments that still described it. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Jacob Williams
jacob@iakob.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38757 Fix EXCHANGE PARTITION with generated columns containing AND/OR conditions EXCHANGE PARTITION fails with ERROR 1736 (Tables have different definitions) when tables contain generated columns with AND/OR conditions, even when the expressions are logically equivalent. This occurs because when expressions are re-parsed (e.g., via CREATE TABLE ... LIKE), the order of arguments in AND/OR conditions may change, but the comparison was order-sensitive. The Item_cond::eq() method was implemented to perform set-based comparison for commutative AND/OR operations. The set-based comparison algorithm ensures that two Item_cond expressions are considered equal if they contain the same set of equivalent arguments, regardless of order. Added comprehensive test case covering: - Generated columns with OR conditions - Generated columns with AND conditions - Multiple generated columns with different AND/OR combinations - Nested AND/OR conditions The fix allows EXCHANGE PARTITION to succeed when expressions are logically equivalent but have different argument ordering, which is correct behavior since AND/OR operations are commutative. MDEV-38757 Limit unordered vcol condition comparison to EXCHANGE PARTITION Review follow-up to the previous commit, which made Item_cond::eq() compare AND/OR argument lists as sets for every caller. That changed equality semantics globally and broke main.derived_cond_pushdown, where conditions that eq() started reporting as equal were dropped from attached_condition. Reordering AND/OR operands also changes evaluation order, which is observable when operands are functions, so the relaxed comparison must not be the default. Item::Eq_config gains an unordered_conditions flag, defaulting to false, next to the existing binary_cmp and omit_table_names flags. Item_cond::eq() compares its argument lists as sets only when that flag is set, and otherwise reports two distinct Item_cond objects as unequal, as it did before this patch series. The flag is threaded through Virtual_column_info::is_equal() and a new mysql_compare_tables() parameter, and only Sql_cmd_alter_table_exchange_partition passes it as true, so the relaxed comparison stays confined to the EXCHANGE PARTITION metadata check. The set comparison tests containment in both directions rather than comparing element counts, so an expression also matches a form that repeats one of its terms, for example col1 > 10 and col2 < 100 or col3 > 50 col3 > 50 or col1 > 10 and col2 < 100 or col3 > 50 Test 5 of parts.partition_exchange_generated_columns covers that case. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hemant Dangi
hemant.dangi@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40944: Galera test failure on galera_sst_mariabackup_ssl_role_certs Issue: mariadb-backup SST unconditionally passes socat's "commonname=" option; some socat builds don't register it, so parseopts() rejects it as unknown regardless of value, breaking all SSL-encrypted SST. Solution: probe the socat binary once for commonname support and drop the option when unsupported. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Otto Kekäläinen
otto@kekalainen.net |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Promote getting GitHub stars in client prompt Ask users to give MariaDB a star by having an extra line in the MariaDB client prompt: Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is X Server version: Y Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Help others discover MariaDB. Star it on GitHub: https://github.com/MariaDB/server Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> Test file updated with: nano --noconvert --nonewlines mysql-test/main/mysql-interactive.result This change is similar to 346c7afe9b7071ce9c47892a83d69944b608b3da applied on 'main', but without the SERVER_MATURITY_LEVEL check in order to have this text visible in stable releases and actually help get the GitHub star count up, and without the server log entry. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
yuchen.pei@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Index the base column, not a copy of its keys Patch 5/N of Approach 1TB. WIP note(ycp): this is a squash of four commits A multi-valued index no longer materialises anything. Until now the DDL built a hidden stored column computed by MVI_ENCODE(), holding the encoded elements of the array as a space-separated document, and put a fulltext index over that column: the keys of every row were on disk twice, once in the document the array came from and once in the column built out of it. Now the key part is the base column itself, and the engine tokenizes it with the mvi fulltext parser, which finds the array inside the document and encodes its elements as it goes. Nothing is stored that was not already there. What the index was declared with -- the array, and the datatype its elements are cast to -- has nowhere to live on the key any more, so it lives in the FRM's EXTRA2_MVI_SPEC section, which was written and read back but unused until now. A Key carries it into that section on the way in, and every TABLE parses it back out into a TABLE::mvi_spec of its own on the way out, the way a virtual column expression is parsed, because it is an expression over one column of one TABLE. That is now the only thing that says a key is a multi-valued index: the key definition itself says only that it is a fulltext key over a column, which it has in common with a plain fulltext key over the very same column. Which is why the parser becomes reserved. A key that named it without a declaration behind it would be indistinguishable from a multi-valued index whose declaration went missing from the FRM, and the server could no longer say which of the two it was looking at. So no table definition may name it, and a key of that shape with no entry in the section is an FRM to refuse -- which is what keeps the section checkable at all. The other thing the key parts stopped saying is which array an index is over. Two multi-valued indexes over one column have the same key part whatever they index, so the declaration decides both whether two of them are duplicates of each other, which check_duplicate_key() already asks, and whether an ALTER TABLE leaves an existing one alone, which compare_keys_but_name() asks: an index declared over another array holds other keys and has to be rebuilt. MATCH ... AGAINST against such an index parses now that the key part is a column the user can name. A boolean-mode search for an encoded key is the same search the optimizer builds. A natural-language one reaches the parser in the mode a document being indexed does, so it is read as a document, finds no array in itself and matches nothing. TODO: a write has to be refused when the index has keys the engine's fulltext token size settings drop, or the row goes missing from the index for good. That hung off the computation of the hidden column and there is no such column now; see the TODO in TABLE::update_virtual_fields() and the testcases it takes out of multi_valued_index_token_size.test. MDEV-40168 Rename the base column in the declaration too RENAME COLUMN rewrites the references to the old name in every expression of the table -- the virtual columns, the check constraints, the defaults -- and a multi-valued index used to be renamed along with them, because its declaration was the expression of the internal column that held its keys. That column is gone and the declaration belongs to the key now, so nothing reached it: the new table's FRM kept saying the old name, and opening it failed to resolve the column the index reads. Rename it in the same place and the same way, on the open table, before the key loop hands these Items to the new table's keys. Dropping the base column now drops the index with it, the way dropping a column drops any other index over it, where before the internal column's dependency on it made the DROP an error. There is nothing of the index left to keep either way: the array it indexes is inside that column. MDEV-40168 Adding a multi-valued index in place Adding one used to mean adding a stored column to hold its keys, so it rebuilt the table and ALGORITHM=INPLACE was refused. There is no column to add any more: the keys are read out of one that is already there, so the engine builds the index itself and INPLACE is enough, on a table with rows and on one that already has a fulltext index alike. Adding a column to a table that has one is still not instant, but for another reason now. It is not the stored column that was in the way, it is the fulltext index the multi-valued index is: InnoDB refuses to add a stored column instantly to a table with one of those, see instant_alter_column_possible() and MDEV-17459. Record all three, and take the remarks about the internal column out of the comments that still described it. MDEV-40168 Do not write into the document, and build the index with the declaration Two bugs, both showing up as wrong results in multi_valued_index.test. encode_mvi_key() was writing into the document it was reading. The binary branch points `sorted' at the element instead of copying it, and step 2 then cuts that down to the longest key image an index can hold, which leaves the String shorter than the buffer it was given. String::c_ptr() NUL-terminates in place when there is room past the string, so it put a NUL over the first byte of the element the image does not cover -- any element longer than MVI_KEY_IMAGE_MAX_LEN. That has always been true, and used to be invisible: the document was either the Item's own copy or the literal of a JSON predicate, which is where the warning 4036 that multi_valued_index.test asked about in a TODO came from. It is not invisible any more. The fulltext parser now reads the document out of the row InnoDB is about to store, so the NUL went into the row and stayed there, and the JSON in it no longer parsed. Use ptr(). The other one is that an index built by ALTER TABLE held the words of the document rather than the keys of the array, so it found nothing. The declaration reaches the engine's build through the key definition being built, but what the fulltext parser is handed was being looked up through the temporary TABLE of the altered definition instead. Make one where the declaration itself is made, in mysql_prepare_create_table_finalize(), and let InnoDB take it off the key like the rest of the definition. An index built without it cannot hold the right entries at all, so assert on that rather than let it happen quietly. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Daniel Black
daniel@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39285: dtrace to check for sys/sdt.h on Linux Buildbot SRPM builders show it is possible to have dtrace (the program) installed without the sys/sdt.h. The dtrace program using -h generated source code that includes the sys/sdt.h header so lets make sure it exists. To make sure that SRPM picks up all include headers, we check that in cmake/build_depends.cmake and let the discovery of all the include headers contribute to the build dependencies. Where there is multiple assume that all header files original from the same package and search for the last one. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Jan Lindström
jan.lindstrom@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40929 : CREATE TABLE ... AS SELECT is not replicated in Galera when the table is partitioned Problem was that check for partitioned tables was missing because then partition implementing handlerton should be used in condition instead. Thanks to Roel Van de Paar <[email protected]> for providing test case and fix candidate. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-33959 mysqldump: dump sequences before tables across databases mariadb-dump --all-databases (or --databases with several databases) dumps databases in SHOW DATABASES order. If a table in one database defaults a column to nextval() of a sequence living in a different database, and that database sorts later, the resulting dump fails to reload with "Table 'db.seq' doesn't exist" -- the table gets created before the sequence it depends on. MDEV-21785 already dumps sequences before tables within a single database, but that alone doesn't help when the dependency crosses a database boundary. Fix by adding a sequences-first pre-pass (dump_all_sequences_in_db) that runs over every database being dumped before any of them reaches the existing per-database table-dump pass, mirroring the "dump all tables, then all views" two-pass shape already used in this file for views. To keep output byte-for-byte unchanged for the common case of a database with no sequences, the pre-pass only creates a database if it turns out to actually own a sequence, and threads that fact through to the table-dump pass so it skips re-creating the database and re-dumping the sequences it already handled. The pre-pass leaves --xml and the "mysql" system database to the unchanged single-pass path: get_sequence_structure() isn't XML-aware, and "mysql" never owns user sequences in practice but has a LOG_OUTPUT save/restore that only closes in the table pass. Co-Authored-By: Claude Sonnet 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-37936 Followup to MDEV-36321: out_rows for GROUP BY: use of item names? MDEV-36321 compared item name strings when looking for key use in items of the select list within a derived table. Here we remove this type of comparison and compare the underlying fields and their position. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alessandro Vetere
iminelink@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38056 Assertion 'bpage->state() >= buf_page_t::UNFIXED' in buf_page_get_zip() Three callers reconstruct old versions of a clustered index record, derive secondary index entries from them, and dereference the BLOB pointers of externally stored columns: row_vers_impl_x_locked_low() for the implicit lock check, row_undo_mod_sec_is_unsafe() for rollback, and row_check_index() for CHECK TABLE ... EXTENDED. purge_sys.view is what keeps those pages allocated, but purge_sys_t::view_guard froze it only for the duration of trx_undo_prev_version_build(), and the dereference happens after. On ROW_FORMAT=COMPRESSED this trips the assertion above; in a release build the freed page is read anyway, which is silent corruption. All three now hold purge_sys.latch across the dereference, and only where the version has externally stored columns, since row_build() dereferences nothing otherwise. In row_check_index() the freeze spans the whole comparison, which fetches one field at a time and may evaluate a virtual column expression; for the latest version it is also confined to a delete-marked record, the only kind that need not own what it points at. Freezing that late means the view may have advanced since the walk decided a version was reachable, so each caller re-establishes that under the freeze; trx_undo_prev_version_build() states the condition and why testing the oldest writer applied suffices. The first two callers can treat that test as an invariant and end the walk where it fails. row_check_index() cannot, because it decides reachability from the lagging purge_sys.end_view on purpose, and that lag is how it finds orphan secondary index records: where the test fails it stops and reports nothing rather than treating the record as an orphan, and no report is lost for good, end_view eventually reaching the same point. Its two purge_sys.is_purgeable() tests now read the frozen view wherever a fetch follows, which makes them atomic with the fetch they guard. trx_undo_prev_version_build(): remove the gate that was meant to stop CHECK TABLE ... EXTENDED from fetching BLOBs it may no longer own, and view_guard::is_extended() with it, which no guard mode could satisfy. That decision belongs to the caller, the only one that knows whether it will dereference anything. row_log_table_get_pk(): document why the online ALTER path may dereference without a freeze. Debug-only keywords. purge_hold_cleanup parks a purge batch between its last purged record and purge_sys_t::batch_cleanup(), the window in which a reader that goes by purge_sys.end_view can still reach history the batch has removed; a batch opens and closes it without ever returning to the test. purge_no_blob_freeze sends a version that does have externally stored columns down the path one without any takes, which is what all four call sites did before this change, and reproduces the assertion above. row_vers_impl_x_locked_purgeable, row_undo_mod_sec_is_unsafe_purgeable and row_check_index_purgeable force the re-validation to fail, reaching exits that purge_sys.view advancing mid-walk otherwise produces. The first reports no implicit lock for a row that a live transaction still holds, so a test may only check that nothing breaks; the other two make the server more cautious. Tests. old_blob and old_blob_updel cover the implicit lock check, old_blob_rollback the rollback, old_blob_check CHECK TABLE ... EXTENDED. Each parks a walk at a dereference, makes the BLOB freeable, and asserts that the counter of purged update records, which is what would free it, stays at zero while parked and advances once the walk is over. old_blob_updel covers an undo log record that stores only the 20-byte reference and needs purge_hold_cleanup to reach it; old_blob_rollback parks at a reference that the version merely inherited. All four fail with the original assertion under debug_dbug=+d,purge_no_blob_freeze, and skip above a 16k page size, which ROW_FORMAT=COMPRESSED requires. old_blob_purgeable drives the three re-validation exits, needs no synchronisation, and runs at every page size. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39368: Code cleanup and make it more maintainable 1. Remove hard codings and instead use MACROS 2. Introduce pre and post query hooks 3. Remove duplicate code and instead use functions |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||