Console View
|
Categories: connectors experimental galera main |
|
| connectors | experimental | galera | main | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Mohammad Tafzeel Shams
tafzeel.shams@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40404 : Assertion '!lock.is_locked_or_waiting()' failed in buf_page_t::init() A locking read on a SPATIAL INDEX collects the matching records into a shadow block matched_rec_t::block. sel_set_rtr_rec_lock() holds page.lock on that block while it locks the records, because lock_rec_lock() asserts that the block of the record being locked is latched. If a lock request has to wait, the records are collected again after the wait, and rtr_init_match() re-initializes the block by buf_page_t::init(), which asserts that the block is not latched. The assertion is about a latch that would be a leftover from the previous identity of the block. It does not apply to rtr_init_match(), where the block is in buf_page_t::MEMORY state, private to the current thread, which is holding page.lock on it on purpose. Changes: - buf_page_t::init(): Tolerate a latch that the current thread is holding on a block that is being initialized in buf_page_t::MEMORY state. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Khaled Riyad
khaled57.dev@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41150 mariadb client: Ctrl-C does not interrupt tab separated result output Ctrl-C sets interrupted_query and sends KILL QUERY to the server, but the result is already fetched into the client by then, so killing the query cannot stop the output. Only the printing loop can, by checking interrupted_query. print_table_data(), print_table_data_html(), print_table_data_xml() and print_table_data_vertically() check it, print_tab_data() did not, so the output of -s, -N and -B could not be interrupted. Add the same check to print_tab_data(). No test case: the suite drives the client through a pty with socat, where 0x03 is not turned into a signal, and there is no other way in the suite to signal a client while it is printing. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
take old_mode and deprecation.h changes from 11.8-13.0 cherry-pick commits 9ecec1f7302, 24fd8c7856e, 3e971059757 * make it easier to add more old-mode values * update deprecation.h to match changes in versioning |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thirunarayanan Balathandayuthapani
thiru@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41550 MTR doesn't produce core dump or stack trace when a server hangs during startup If mariadbd does not finish starting up, for example it hangs in a restart done by a test, mtr only killed it, so there was no way to see where it was stuck. When a test fails and a server is still running but has not created its pid file, send it SIGABRT before killing it. The server then writes a stack trace to its error log and dumps core, and the log is added to the failure report. MTR_DUMP_CORE_TIMEOUT (default 120 seconds) sets how long mtr waits for the core dump. Nothing is done when the server runs under a debugger. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Andrei Elkin
andrei.elkin@pp.inet.fi |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41214 PURGE BINARY LOGS does not delete non-active binlog files PURGE BINARY LOGS could end with a warning instead of deleting files, as the test below shows when run on the base of this commit. The warning names a wrong reason. It follows an automatic purge that was refused by the slave guard, slave_connections_needed_for_purge: the automatic purge does not delete a binlog while it might still be needed by a slave, not necessarily a connected one, that is while fewer than the variable's number of slaves are connected. Since MDEV-34504 the manual PURGE is meant to ignore the guard. Yet it still protects a binlog that a slave is reading. Technically this was possible because the automatic and the manual purge share a code path, can_purge_log(), which cached the refusal and replayed it, under the reason of the first arm below, to any later purge: if (is_active(name) || (!is_relay_log && waiting_for_slave_to_change_binlog && purge_sending_new_binlog_file == sending_new_binlog_file && !strcmp(name, purge_binlog_name))) reason= "it is the current active binlog"; So "active" had to be read as "active or busy". The fix removes this duality too: the two arms are now separate and have their own reasons. The main issue is fixed by narrowing the busy arm with an additional `!interactive &&` conjunct, so that a manual PURGE never consults the cached refusal. Running the test below on an unfixed 11.4 server (source tree at commit 42038e1145d) fails with "Result length mismatch" and, in part 2, this diff against the recorded result: PURGE BINARY LOGS TO 'master-bin.000006'; +Warnings: +Note 1375 Binary log 'master-bin.000003' is not purged because it is the current active binlog SHOW WARNINGS; Level Code Message +Note 1375 Binary log 'master-bin.000003' is not purged because it is the current active binlog show binary logs; Log_name File_size +master-bin.000003 # +master-bin.000004 # +master-bin.000005 # master-bin.000006 # master-bin.000007 # master-bin.000008 # Test binlog.binlog_purge_stale_refusal_cache: the test sets slave_connections_needed_for_purge=1, binlog_expire_logs_seconds=0 and max_binlog_total_size=0 first and restores them at the end. It starts with RESET MASTER so that the binlog names, 000003 and so on, do not depend on the tests run before it in the same mtr environment. Part 1 is a control with nothing cached; part 2 makes an automatic purge refuse 000003 (the files must be older than binlog_expire_logs_seconds, hence --real_sleep 2 before the rotation) and then runs PURGE BINARY LOGS TO 'master-bin.000006', which must remove 000003..000005. As a workaround, a PURGE could work if the following settings are made before it, in this order: SET GLOBAL binlog_expire_logs_seconds=0, slave_connections_needed_for_purge=0; The second setting invalidates the cached refusal. The first one keeps the purge that this setting triggers from refusing and caching again. Note that both settings change the server's policy, not only this one PURGE: the time-based expiry and the slave guard are off until they are restored. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| backup_config_append(): Report error on openat(2) failure | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hemant Dangi
hemant.dangi@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40501: Assertion `info->type == READ_CACHE || info->type == WRITE_CACHE' failed in reinit_io_cache upon CHANGE MASTER Issue: CHANGE MASTER ... FOR CHANNEL with a channel name within MAX_CONNECTION_NAME can still overflow the OS file name limit once escaped into the relay log file name. Relay_log_info::init() then fails to open the relay log, leaving its index file unopened, but Master_info_index::remove_master_info() unconditionally calls reset_logs() on it during CHANGE MASTER's error cleanup, which hits the assertion in reinit_io_cache(). Solution: Guard the reset_logs() call in remove_master_info() with is_open(), so a relay log that was never opened is never passed to it. Also use MY_SAFE_PATH in open_index_file() so an over-length name fails deterministically instead of silently falling back to a mangled one. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
iff-sal
iffathdev@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40296: SELECT @@GLOBAL.replicate_do_db ignores default_master_connection When querying replication filter system variables without specifying a connection name qualifier (e.g., SELECT @@GLOBAL.replicate_do_db), Sys_var_rpl_filter::global_value_ptr() previously passed the unspecified base_name directly to get_master_info(), which resolved to the default connection (empty_clex_str) rather than respecting @@SESSION.default_master_connection. Fix by falling back to thd->variables.default_master_connection when base_name->str is NULL in Sys_var_rpl_filter::global_value_ptr(). Add regression test in suite/multi_source/rpl_filter_default_master.test covering explicit named connection, unspecified connection falling back to default, and explicit unnamed connection with empty backticks. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
KhaledR57
khaled57.dev@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-37167 Nested BEGINs (4600+) cause a segmentation fault Each nested BEGIN adds one sp_pcontext. Two walks over the finished tree recursed once per nesting level and overran the thread stack. ~sp_pcontext() freed its children recursively. sp_head now threads a single linked list through the contexts and frees them iteratively, so teardown depth is constant and a context no longer frees its children. retrieve_field_definitions() descends the children to build the run-time frame. It emits them in run-time offset order, so it stays recursive, but it now checks the stack and returns an error instead of crashing. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-32412 Pushdown from HAVING: Item_func is immutable while arguments are not Fix-up for commit e97560eac03 (MDEV-28958), which made set_extraction_flag() ignore basic constants so that the read-only Item_true/Item_false singletons are never written to. Because of that, the callers that mark a subtree with MARKER_IMMUTABLE before pushing a condition from HAVING into WHERE cannot mark basic constants inside it. Item::cleanup_excluding_immutables_processor() did not know about this exception and cleaned such items up, unfixing them while their marked parents stayed fixed. fix_fields() does not descend into fixed items, so the constant was left unfixed and Item_direct_view_ref::used_tables() later dereferenced a NULL null_ref_table. Skip basic constants there too. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alessandro Vetere
iminelink@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41557 Test encryption.innochecksum is flaky The test checks that sys.innodb_buffer_stats_by_table reports old pages (pages_old) after a server restart. InnoDB creates the old sublist of the buffer pool LRU list only when the list holds at least BUF_LRU_OLD_MIN_LEN (512) pages. After the restart, most of the pages in the list come from the buffer pool load at startup. This load runs asynchronously, and the server accepts connections before it completes. If the test runs the check first, the list can be shorter than 512 pages, no page is old, and the result shows 0 instead of 1. The option --innodb-buffer-pool-load-at-startup=0 disables the load. This is the extreme case of the race: the check always runs before any page of the dump is loaded. With this option, the test fails every time with innodb_page_size=4k and 16k. Wait for the buffer pool load to complete before the check, as other tests do. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Jan Lindström
jan.lindstrom@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41028 : Galera appliers deadlock on a foreign key referencing a CHAR column in a multi-byte character set The write set key of a row is built from the MySQL record by wsrep_store_key_val_for_row(), and the key of a foreign key parent row from the InnoDB record by wsrep_rec_get_foreign_key(). A CHAR is not padded the same way in the two formats: the MySQL record pads it to n_chars * mbmaxlen bytes, while InnoDB strips that padding down to, but not below, n_chars bytes. That compares a byte count with a character count, so a value holding a multi byte character and shorter than the column was left with a different number of characters on the two paths, and the keys differed. A child INSERT then had no dependency on its parent row and the appliers ran it in parallel with a change of that very row. The two paths did not agree on the strnxfrm buffer length either, 3072 on one and 3500 on the other. Both now go through wsrep_store_string_key_val(), which brings a CHAR to exactly the number of characters the column holds and always normalizes with WSREP_MAX_SUPPORTED_KEY_LENGTH, so that the key of a column does not depend on how much room the columns before it happened to leave. Only the copy into the caller's buffer is bounded by the space that is left, which also stops wsrep_rec_get_foreign_key() from writing past its key buffer. This changes the write set keys, so it is done from protocol version 5 on and the old encoding is kept below that. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge 11.4 into 11.8 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge 10.11 into 11.4 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thirunarayanan Balathandayuthapani
thiru@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41548 InnoDB: Cannot replay rename … because the target file exists Problem: ======= recv_rename_files() replayed the renames in tablespace id order, because renamed_spaces was a std::map keyed by space id that kept only the latest file name per tablespace. A multi-table RENAME such as RENAME TABLE t2 TO t3, t1 TO t2 frees a file name for a tablespace that may have a lower id, and replaying in id order then fails with "Cannot replay rename of tablespace … because the target file exists". Solution: ======== renamed_spaces: Change to a std::vector of (id, from, to), retaining every FILE_RENAME record in log order. The records are appended only while recv_sys.file_checkpoint is set, hence exactly once and in ascending LSN. recv_rename_files(): Replay the records in that order, applying one only when the tablespace currently uses the file name that the record renames from. The order in which the server executed the renames is always replayable, because each rename required its target to be unoccupied. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Allow log file extension during backup | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thirunarayanan Balathandayuthapani
thiru@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41022 Wildcard term returns no rows when combined with a phrase in boolean mode search Problem: ======== MATCH(..) AGAINST('"zzzz" aut*' IN BOOLEAN MODE) fails to return the rows which match the wildcard term when the matching word is still in the FTS cache. fts_query_phrase_search() sets query->flags to FTS_PHRASE or FTS_PROXIMITY, but never resets it before returning. The subsequent wildcard term of the same query then finds the stale flag in fts_query_cache() and fts_query_difference(), takes the exact word lookup path instead of fts_cache_find_wildcard() and misses the words present in the cache. Solution: ========= - Reset query->flags in fts_query_phrase_search() before returning, so that the phrase or proximity state doesn't leak into the terms evaluated later in the same query. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Disable rpl_parallel_multi_domain_xa MDEV-34104 describes why this test fails. It was filed 2 years ago, but the fix is complex, and we keep this failing test around hurting all other devs. The fix is planned, once finished, we can re-enable this test. Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexander Barkov
bar@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41246 "Illegal mix of collations" on the mysql.user view This SQL script failed: SET NAMES latin1 COLLATE latin1_swedish_ci; CREATE OR REPLACE VIEW v1 AS SELECT 'Y' AS c1; SET NAMES big5 COLLATE big5_chinese_ci; SELECT * FROM v1 WHERE c1='y'; with the following error: ERROR 1267 (HY000): Illegal mix of collations (latin1_swedish_ci,COERCIBLE) and (big5_chinese_ci,COERCIBLE) for operation '=' Note, latin1_swedish_ci and big5_chinese_ci are used here as examples. The error also happened with different collation combinations. Fix main idea: If two collations have equal comparison rules (known as "tailoring") on a given character repertoire, like latin1_swedish_ci and big5_chinese_ci on ASCII letters, then the "Illegal mix of collations" error can be avoided in a comparison operator. We can choose any of the sides as the operation effective collation - the result will be equal. This optimization is not applied when at least one side has an explicit COLLATE clause. Two explicit COLLATE clauses in one comparison are already illegal when the character sets are the same, so for consistency this stays illegal when the character sets differ too. Most important details: - Splitting enum_repertoire_t into smaller subsets, for better repertoire granularity. A variable holding a repertoire value can now have multiple MY_REPERTOIRE_XXX flags set. This patch implements detecting tailoring equality on these repertoires: * MY_REPERTOIRE_ASCII_DIGITS - [0..9] * MY_REPERTOIRE_ASCII_MINUS_DOT_DIGITS - [-.0..9], e.g. '-10.5'. It consists of the new bits MY_REPERTOIRE_ASCII_MINUS and MY_REPERTOIRE_ASCII_DOT. They were a part of MY_REPERTOIRE_ASCII_NOT_IDENT before. As a result, the numeric values of MY_REPERTOIRE_ASCII and MY_REPERTOIRE_UNICODE30 changed (223 and 255). * MY_REPERTOIRE_ASCII_ALNUM - [A..Z,a..z,0..9]. * MY_REPERTOIRE_ASCII_IDENT - ALNUM + underscore * MY_REPERTOIRE_ASCII - the entire range U+0000..U+007F - Adding a new virtual function "tailoring" in my_collation_handler_st. It returns the tailoring on the given repertoire for the given collation. If cs1->coll->tailoring(cs1, some_repertoire) returns {0,0}, it means the illegal mix optimization cannot be used for this collation on the given repertoire. If these calls: tr1= cs1->coll->tailoring(cs1, some_repertoire); tr2= cs2->coll->tailoring(cs2, some_repertoire); return both non-NULL results and tr1.str==tr2.str, then these collations are equal on the given repertoire and are mutually replaceable for a comparison operator, so "Illegal mix of collations" can be avoided. - Tailoring strings are shared constants my_tailoring_str_* (strings/strings_def.h). Tailorings are compared by the string pointer, so tailoring() implementations return strings only from these constants. The strings do not depend on PAD/NOPAD. - my_string_repertoire() for ucs2, utf16, utf32 and NONASCII 8bit character sets no longer stops at the first bad byte sequence with the repertoire found so far, which could be MY_REPERTOIRE_NONE (compatible with everything). It returns MY_REPERTOIRE_EXTENDED for such strings. A unit test was added into unittest/strings. - Adding CHARSET_INFO::is_ascii_superset(). It tells if a character set can store all ASCII characters (it is false for swe7). It is used in Item_func_conv_charset to calculate "safe", and in left_is_algorithmically_simpler() to prefer other character sets to those where ASCII strings cannot be converted safely. - DBUG_ASSERTs were added into mysys/charset.c, to check that the "tailoring" method is set in collation handlers when a collation is added: compiled-in, from ctype-extra.c, or loaded from Index.xml. - Adding a new function my_collations_equal_on_repertoire(cs1, cs2, repertoire) in strings/. It tells if two collations are equal on the repertoire, taking tailoring() results and PAD/NOPAD and the UCA version into account. Callers do not compare tailoring() results directly, so the way tailorings are compared (currently by the pointer to the shared constant string) is private to strings/ and can be extended later, e.g. for UCA collations on MY_REPERTOIRE_UNICODE30. - Adding a new method DTCollation::aggregate_by_tailoring(). Collations with different MY_CS_NOPAD are never compatible. UCA collations of different versions are not compatible on the repertoire with ASCII punctuation, as the UCA version affects its order (e.g. UCA-6.2.0 moved GRAVE ACCENT and CIRCUMFLEX ACCENT after PERCENT SIGN). - DTCollation::aggregate() now makes the repertoire of the result cover the repertoires of both sides in all successful branches. Before, set(dt) replaced the repertoire with the repertoire of the winner side only, so the result could have a too narrow repertoire, e.g. IF(1, 1.5, HEX(255)) had MY_REPERTOIRE_ASCII_ALNUM although the value '1.5' has a punctuation character. With the more granular repertoires this could wrongly allow mixing collations which are equal on ALNUM but different on punctuation. - Adding a new flag MY_COLL_ALLOW_BY_TAILORING. It indicates to DTCollation::aggregate() that the illegal mix optimization by repertoire can be used in the given context. MY_COLL_CMP_CONV now includes MY_COLL_ALLOW_BY_TAILORING. Note, only comparison operators pass this flag. Functions returning a string result do not pass this flag, because in operations like CONCAT(a,b) we still need to evaluate precisely the collation of the result - we cannot just choose a collation of one of the sides (even if they are compatible on the given repertoire). This also applies to ExtractValue() and UpdateXML(), which now aggregate their arguments without this flag. - As in my_repertoire_t the value MY_REPERTOIRE_ASCII is now a set of bits rather than a single bit, the way to detect "is only ASCII" repertoires has changed in the code. For example: // repertoire *IS* ascii if (repertoire == MY_REPERTOIRE_ASCII) has changed in multiple places in the code to // repertoire *HAS* only ascii characters if (my_repertoire_is_subset_of(repertoire, MY_REPERTOIRE_ASCII)) The new function my_repertoire_is_subset_of() in m_ctype.h is used for this purpose in C code. In C++ code DTCollation::repertoire_is_subset_of() is used, e.g.: if (collation.repertoire_is_subset_of(MY_REPERTOIRE_ASCII)) - Repertoire of some expressions was adjusted to the new meaning: * Item_null now has MY_REPERTOIRE_NONE (was ASCII). * MY_LOCALE::repertoire() now returns MY_REPERTOIRE_UNICODE30 (was EXTENDED). * Lex_string_with_metadata_st::repertoire(cs) now scans the string contents to detect the actual repertoire. * HEX() now has MY_REPERTOIRE_ASCII_ALNUM. * DATE_FORMAT() decides on MY_REPERTOIRE_EXTENDED using the locale which is actually used: the explicit third argument, or @@lc_time_names. It was always @@lc_time_names before. A non-constant locale argument is assumed to be non-ASCII. * QUOTE, MAKE_SET, EXPORT_SET, LPAD, RPAD, GROUP_CONCAT, JSON_ARRAY, JSON_OBJECT and JSON_OBJECTAGG add the repertoire of the extra characters they put into the result (quotes, separators, padding, brackets). * LOWER() and UPPER() add the letters of both cases to the repertoire, because the result can contain letters of the opposite case. In Turkish collations they also add MY_REPERTOIRE_EXTENDED, as an ASCII letter can be converted to a non-ASCII one (I -> dotless i). Unicode collations are detected by the new member casefold_info_st::can_convert_to_non_ascii_on_casefolding, simple 8bit collations by to_lower['I'] and to_upper['i']. - Lex_string_with_metadata_st::repertoire(cs) now scans a string literal for any byte oriented character set (mbminlen==1), not only for ASCII based ones. It was assumed to have the full repertoire for swe7, which is not ASCII based. The scan is done by Unicode code points, so swe7 strings with digits and letters get a narrow repertoire, while e.g. 0x5B (LATIN CAPITAL LETTER A WITH DIAERESIS in swe7) is non-ASCII. Therefore swe7_swedish_ci can be mixed with other collations now, in the same way as other collations with MY_CS_IDENT_CASEUP_CI. The test func_time was adjusted: a swe7 literal ' ull' with ASCII characters is now safe to convert, the error is still tested for '[ull'. - cp1250_czech_cs sorts -, . and 0..9 in the code point order, so it gets the MINUS_DOT_DIGITS tailoring. latin2_czech_cs gets only DIGITS, as it sorts the minus and the dot differently. Both were verified with ctype_digits_verify. - The tis620 collations now set CHARSET_INFO::tab_to_uni (was NULL), so my_charset_is_ascii_based() is true for tis620. It was false for tis620, although tis620 is ASCII compatible. Therefore tis620 string literals were always considered to have a non-ASCII repertoire, and the repertoire optimizations did not work for them. Now ASCII-only tis620 literals are detected as ASCII. This changes the result of subselect_extra_no_semijoin from an error to success. - tis620_thai_ci and tis620_thai_nopad_ci fold letters to lower case and then compare by the code point on the entire ASCII range (the Thai specific rules affect only non-ASCII characters). A new function my_tailoring_ascii_casedn_ci() returns tailorings for such collations on ALNUM, IDENT and ASCII. For example, the underscore sorts before the letters, unlike in the caseup tailorings. - New flags were added for CHARSET_INFO::state * MY_CS_ASCII_CASEUP_CI - for simple 8bit case insensitive collations. It means that this collation has no irregularities on the ASCII repertoire, and converts lower case letters to upper case ones, so the underscore sorts after the letters (with an upper to lower case conversion it would sort before the letters). * MY_CS_IDENT_CASEUP_CI - for simple 8bit case insensitive collations. It means that this collation has no irregularities on the IDENT repertoire (but can have irregularities say on punctuation). * MY_CS_ASCII_STD_UCA - for UCA collations. It means that a UCA collation does not reorder ASCII characters. * MY_CS_DIGITS_STD - for any collation. It means that the ASCII digits 0..9 are sorted in the code point order and are not tailored, and, for PAD collations, the space sorts before the digits. Strings of digits are compared in the same way in all such collations, even if they have irregularities on letters (latin7, cp866, czech, danish, turkish UCA, etc.), or differ by case sensitivity. It is detected from sort_order for 8bit collations, and from the tailoring rules for UCA collations. Collations with MY_CS_DIGITS_STD still must have the same PAD/NOPAD attribute. * MY_CS_ASCII_MINUS_DOT_DIGITS - for any collation. Means MY_CS_DIGITS_STD, and additionally the minus sorts before the dot, which sorts before the digits (and the space sorts before the minus in PAD collations). Strings like '-10.5' are compared in the same way in all such collations. It is detected from sort_order for simple 8bit collations (my_coll_init_simple()), and from the tailoring rules for UCA collations. For the multi-byte collations big5_chinese_ci, gbk_chinese_ci, gb2312_chinese_ci and sjis_japanese_ci it is declared by my_tailoring_ident_caseup_ci_generic(). The flags MY_CS_DIGITS_STD and MY_CS_ASCII_MINUS_DOT_DIGITS are stored in ctype-extra.c for compiled collations (conf_to_src.c prints them), and the detection is skipped for a collation which already has any of the repertoire flags (MY_CS_REPERTOIRE_FLAGS). - strings/conf_to_src.c was modified to detect and print MY_CS_ASCII_CASEUP_CI and MY_CS_IDENT_CASEUP_CI flags. - strings/ctype-extra.c was regenerated with new flags. - 8bit collations are not given MY_CS_ASCII_CASEUP_CI and MY_CS_IDENT_CASEUP_CI if they are PAD and the space does not sort before the digits. A shorter string is padded with spaces for comparison, so the position of the space matters even for strings without spaces ("ab" is compared to "abc" as "ab " to "abc"). A test collation latin1_space_after_letters_ci was added to mysql-test/std_data/ldml/ for this. ctype-extra.c does not change. - The initializer of my_collation_cs_handler in ctype-utf8.c (utf8mb3_general_cs, under HAVE_UTF8_GENERAL_CS) was fixed to match my_collation_handler_st: the missing get_id and get_collation_name members were added, so that eq_collation was not initialized in their place, and the new member "tailoring" is now set to my_tailoring_none. - Results of the existing tests func_str, func_test, ps, view and subselect_sj changed from "Illegal mix of collations" to success. In these tests ASCII-only strings were compared using collations with equal tailoring on ASCII, e.g. latin1_swedish_ci with latin2_general_ci (func_str, ps), koi8r_general_ci with latin1_swedish_ci (func_test), latin1_general_ci with latin1_swedish_ci on letters and digits (view), cp932_japanese_ci with latin1_swedish_ci (subselect_sj). Such comparisons are now allowed. Cases with collations that are still incompatible on ASCII, e.g. latin7_general_ci, were added to the tests and still fail. - Adding a number of MTR tests in plugin/func_test/mysql-test/func_test/. They display a tailoring by collation name and repertoire as returned by: cs->coll->tailoring(cs, some_repertoire) A dynamically linked plugin function collation_tailoring() was added for the purpose of these tests. Tailorings do not depend on PAD/NOPAD and on the UCA version, so the function shows "[nopad]" for NOPAD collations, and "[version X.Y.Z]" for UCA collations on the ASCII repertoire. The version is formatted by a new class UCAVersion, which uses a new method CharBuffer::append_uint8(). - Adding the test plugin/func_test/mysql-test/func_test/ ctype_caseup_ci_verify.test. It checks the order of ASCII characters for all collations declared "caseup CI" on the ASCII or IDENT range. - Adding the test plugin/func_test/mysql-test/func_test/ collation_tailoring_args.test. It checks the arguments of the plugin function collation_tailoring(): the repertoire passed by name or by number, and the NULL results for unknown names. - Adding the test plugin/func_test/mysql-test/func_test/ctype_ldml.test. It displays the tailorings of collations loaded at runtime from Index.xml: simple 8bit collations (flags detected from the weights) and Unicode collations (LDML rules which do or do not reorder ASCII). - Adding a number of MTR tests mysql-test/main/ctype_xxx_tailoring.test They display two-dimensional charts showing which collations are compatible on which repertoires. Tests for DATE_FORMAT() with an explicit locale were added to ctype_big5_tailoring.test, ctype_cp932_tailoring.test and ctype_latin5_tailoring.test, which also has the chart for latin5. - Adding a number of MTR tests in the form of the originally reported script for various collations: SELECT Insert_priv FROM mysql.user WHERE Insert_priv='...'; The repeated blocks of queries are shared through mysql-test/include/ ctype_tailoring_0{1,2,3}_mysql_user_Insert_priv*.inc. - Adding tests to ctype_cp932.test checking that ExtractValue() and UpdateXML() still raise "Illegal mix of collations". |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-35867 mariadb-check --repair does not repair sequences, only changes the status to OK REPAIR of a sequence table with more or fewer than one row reported OK. Now ha_sequence::repair() checks the row count after the engine repair and reports the sequence as corrupt. Also a failed table recreate in REPAIR was reported as "Table is already up to date" (bool result taken as HA_ADMIN_ALREADY_DONE). Co-Authored-By: Claude Opus 5.5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexander Barkov
bar@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41246 "Illegal mix of collations" on the mysql.user view This SQL script failed: SET NAMES latin1 COLLATE latin1_swedish_ci; CREATE OR REPLACE VIEW v1 AS SELECT 'Y' AS c1; SET NAMES big5 COLLATE big5_chinese_ci; SELECT * FROM v1 WHERE c1='y'; with the following error: ERROR 1267 (HY000): Illegal mix of collations (latin1_swedish_ci,COERCIBLE) and (big5_chinese_ci,COERCIBLE) for operation '=' Note, latin1_swedish_ci and big5_chinese_ci are used here as examples. The error also happened with different collation combinations. Fix main idea: If two collations have equal comparison rules (known as "tailoring") on a given character repertoire, like latin1_swedish_ci and big5_chinese_ci on ASCII letters, then the "Illegal mix of collations" error can be avoided in a comparison operator. We can choose any of the sides as the operation effective collation - the result will be equal. This optimization is not applied when at least one side has an explicit COLLATE clause. Two explicit COLLATE clauses in one comparison are already illegal when the character sets are the same, so for consistency this stays illegal when the character sets differ too. Most important details: - Splitting enum_repertoire_t into smaller subsets, for better repertoire granularity. A variable holding a repertoire value can now have multiple MY_REPERTOIRE_XXX flags set. This patch implements detecting tailoring equality on these repertoires: * MY_REPERTOIRE_ASCII_DIGITS - [0..9] * MY_REPERTOIRE_ASCII_MINUS_DOT_DIGITS - [-.0..9], e.g. '-10.5'. It consists of the new bits MY_REPERTOIRE_ASCII_MINUS and MY_REPERTOIRE_ASCII_DOT. They were a part of MY_REPERTOIRE_ASCII_NOT_IDENT before. As a result, the numeric values of MY_REPERTOIRE_ASCII and MY_REPERTOIRE_UNICODE30 changed (223 and 255). * MY_REPERTOIRE_ASCII_ALNUM - [A..Z,a..z,0..9]. * MY_REPERTOIRE_ASCII_IDENT - ALNUM + underscore * MY_REPERTOIRE_ASCII - the entire range U+0000..U+007F - Adding a new virtual function "tailoring" in my_collation_handler_st. It returns the tailoring on the given repertoire for the given collation. If cs1->coll->tailoring(cs1, some_repertoire) returns {0,0}, it means the illegal mix optimization cannot be used for this collation on the given repertoire. If these calls: tr1= cs1->coll->tailoring(cs1, some_repertoire); tr2= cs2->coll->tailoring(cs2, some_repertoire); return both non-NULL results and tr1.str==tr2.str, then these collations are equal on the given repertoire and are mutually replaceable for a comparison operator, so "Illegal mix of collations" can be avoided. - Tailoring strings are shared constants my_tailoring_str_* (strings/strings_def.h). Tailorings are compared by the string pointer, so tailoring() implementations return strings only from these constants. The strings do not depend on PAD/NOPAD. - my_string_repertoire() for ucs2, utf16, utf32 and NONASCII 8bit character sets no longer stops at the first bad byte sequence with the repertoire found so far, which could be MY_REPERTOIRE_NONE (compatible with everything). It returns MY_REPERTOIRE_EXTENDED for such strings. A unit test was added into unittest/strings. - Adding CHARSET_INFO::is_ascii_superset(). It tells if a character set can store all ASCII characters (it is false for swe7). It is used in Item_func_conv_charset to calculate "safe", and in left_is_algorithmically_simpler() to prefer other character sets to those where ASCII strings cannot be converted safely. - DBUG_ASSERTs were added into mysys/charset.c, to check that the "tailoring" method is set in collation handlers when a collation is added: compiled-in, from ctype-extra.c, or loaded from Index.xml. - Adding a new function my_collations_equal_on_repertoire(cs1, cs2, repertoire) in strings/. It tells if two collations are equal on the repertoire, taking tailoring() results and PAD/NOPAD and the UCA version into account. Callers do not compare tailoring() results directly, so the way tailorings are compared (currently by the pointer to the shared constant string) is private to strings/ and can be extended later, e.g. for UCA collations on MY_REPERTOIRE_UNICODE30. - Adding a new method DTCollation::aggregate_by_tailoring(). Collations with different MY_CS_NOPAD are never compatible. UCA collations of different versions are not compatible on the repertoire with ASCII punctuation, as the UCA version affects its order (e.g. UCA-6.2.0 moved GRAVE ACCENT and CIRCUMFLEX ACCENT after PERCENT SIGN). - DTCollation::aggregate() now makes the repertoire of the result cover the repertoires of both sides in all successful branches. Before, set(dt) replaced the repertoire with the repertoire of the winner side only, so the result could have a too narrow repertoire, e.g. IF(1, 1.5, HEX(255)) had MY_REPERTOIRE_ASCII_ALNUM although the value '1.5' has a punctuation character. With the more granular repertoires this could wrongly allow mixing collations which are equal on ALNUM but different on punctuation. - Adding a new flag MY_COLL_ALLOW_BY_TAILORING. It indicates to DTCollation::aggregate() that the illegal mix optimization by repertoire can be used in the given context. MY_COLL_CMP_CONV now includes MY_COLL_ALLOW_BY_TAILORING. Note, only comparison operators pass this flag. Functions returning a string result do not pass this flag, because in operations like CONCAT(a,b) we still need to evaluate precisely the collation of the result - we cannot just choose a collation of one of the sides (even if they are compatible on the given repertoire). This also applies to ExtractValue() and UpdateXML(), which now aggregate their arguments without this flag. - As in my_repertoire_t the value MY_REPERTOIRE_ASCII is now a set of bits rather than a single bit, the way to detect "is only ASCII" repertoires has changed in the code. For example: // repertoire *IS* ascii if (repertoire == MY_REPERTOIRE_ASCII) has changed in multiple places in the code to // repertoire *HAS* only ascii characters if (my_repertoire_is_subset_of(repertoire, MY_REPERTOIRE_ASCII)) The new function my_repertoire_is_subset_of() in m_ctype.h is used for this purpose in C code. In C++ code DTCollation::repertoire_is_subset_of() is used, e.g.: if (collation.repertoire_is_subset_of(MY_REPERTOIRE_ASCII)) - Repertoire of some expressions was adjusted to the new meaning: * Item_null now has MY_REPERTOIRE_NONE (was ASCII). * MY_LOCALE::repertoire() now returns MY_REPERTOIRE_UNICODE30 (was EXTENDED). * Lex_string_with_metadata_st::repertoire(cs) now scans the string contents to detect the actual repertoire. * HEX() now has MY_REPERTOIRE_ASCII_ALNUM. * DATE_FORMAT() decides on MY_REPERTOIRE_EXTENDED using the locale which is actually used: the explicit third argument, or @@lc_time_names. It was always @@lc_time_names before. A non-constant locale argument is assumed to be non-ASCII. * QUOTE, MAKE_SET, EXPORT_SET, LPAD, RPAD, GROUP_CONCAT, JSON_ARRAY, JSON_OBJECT and JSON_OBJECTAGG add the repertoire of the extra characters they put into the result (quotes, separators, padding, brackets). * LOWER() and UPPER() add the letters of both cases to the repertoire, because the result can contain letters of the opposite case. In Turkish collations they also add MY_REPERTOIRE_EXTENDED, as an ASCII letter can be converted to a non-ASCII one (I -> dotless i). Unicode collations are detected by the new member casefold_info_st::can_convert_to_non_ascii_on_casefolding, simple 8bit collations by to_lower['I'] and to_upper['i']. - Lex_string_with_metadata_st::repertoire(cs) now scans a string literal for any byte oriented character set (mbminlen==1), not only for ASCII based ones. It was assumed to have the full repertoire for swe7, which is not ASCII based. The scan is done by Unicode code points, so swe7 strings with digits and letters get a narrow repertoire, while e.g. 0x5B (LATIN CAPITAL LETTER A WITH DIAERESIS in swe7) is non-ASCII. Therefore swe7_swedish_ci can be mixed with other collations now, in the same way as other collations with MY_CS_IDENT_CASEUP_CI. The test func_time was adjusted: a swe7 literal ' ull' with ASCII characters is now safe to convert, the error is still tested for '[ull'. - cp1250_czech_cs sorts -, . and 0..9 in the code point order, so it gets the MINUS_DOT_DIGITS tailoring. latin2_czech_cs gets only DIGITS, as it sorts the minus and the dot differently. Both were verified with ctype_digits_verify. - The tis620 collations now set CHARSET_INFO::tab_to_uni (was NULL), so my_charset_is_ascii_based() is true for tis620. It was false for tis620, although tis620 is ASCII compatible. Therefore tis620 string literals were always considered to have a non-ASCII repertoire, and the repertoire optimizations did not work for them. Now ASCII-only tis620 literals are detected as ASCII. This changes the result of subselect_extra_no_semijoin from an error to success. - tis620_thai_ci and tis620_thai_nopad_ci fold letters to lower case and then compare by the code point on the entire ASCII range (the Thai specific rules affect only non-ASCII characters). A new function my_tailoring_ascii_casedn_ci() returns tailorings for such collations on ALNUM, IDENT and ASCII. For example, the underscore sorts before the letters, unlike in the caseup tailorings. - New flags were added for CHARSET_INFO::state * MY_CS_ASCII_CASEUP_CI - for simple 8bit case insensitive collations. It means that this collation has no irregularities on the ASCII repertoire, and converts lower case letters to upper case ones, so the underscore sorts after the letters (with an upper to lower case conversion it would sort before the letters). * MY_CS_IDENT_CASEUP_CI - for simple 8bit case insensitive collations. It means that this collation has no irregularities on the IDENT repertoire (but can have irregularities say on punctuation). * MY_CS_ASCII_STD_UCA - for UCA collations. It means that a UCA collation does not reorder ASCII characters. * MY_CS_DIGITS_STD - for any collation. It means that the ASCII digits 0..9 are sorted in the code point order and are not tailored, and, for PAD collations, the space sorts before the digits. Strings of digits are compared in the same way in all such collations, even if they have irregularities on letters (latin7, cp866, czech, danish, turkish UCA, etc.), or differ by case sensitivity. It is detected from sort_order for 8bit collations, and from the tailoring rules for UCA collations. Collations with MY_CS_DIGITS_STD still must have the same PAD/NOPAD attribute. * MY_CS_ASCII_MINUS_DOT_DIGITS - for any collation. Means MY_CS_DIGITS_STD, and additionally the minus sorts before the dot, which sorts before the digits (and the space sorts before the minus in PAD collations). Strings like '-10.5' are compared in the same way in all such collations. It is detected from sort_order for simple 8bit collations (my_coll_init_simple()), and from the tailoring rules for UCA collations. For the multi-byte collations big5_chinese_ci, gbk_chinese_ci, gb2312_chinese_ci and sjis_japanese_ci it is declared by my_tailoring_ident_caseup_ci_generic(). The flags MY_CS_DIGITS_STD and MY_CS_ASCII_MINUS_DOT_DIGITS are stored in ctype-extra.c for compiled collations (conf_to_src.c prints them), and the detection is skipped for a collation which already has any of the repertoire flags (MY_CS_REPERTOIRE_FLAGS). - strings/conf_to_src.c was modified to detect and print MY_CS_ASCII_CASEUP_CI and MY_CS_IDENT_CASEUP_CI flags. - strings/ctype-extra.c was regenerated with new flags. - 8bit collations are not given MY_CS_ASCII_CASEUP_CI and MY_CS_IDENT_CASEUP_CI if they are PAD and the space does not sort before the digits. A shorter string is padded with spaces for comparison, so the position of the space matters even for strings without spaces ("ab" is compared to "abc" as "ab " to "abc"). A test collation latin1_space_after_letters_ci was added to mysql-test/std_data/ldml/ for this. ctype-extra.c does not change. - The initializer of my_collation_cs_handler in ctype-utf8.c (utf8mb3_general_cs, under HAVE_UTF8_GENERAL_CS) was fixed to match my_collation_handler_st: the missing get_id and get_collation_name members were added, so that eq_collation was not initialized in their place, and the new member "tailoring" is now set to my_tailoring_none. - Results of the existing tests func_str, func_test, ps, view and subselect_sj changed from "Illegal mix of collations" to success. In these tests ASCII-only strings were compared using collations with equal tailoring on ASCII, e.g. latin1_swedish_ci with latin2_general_ci (func_str, ps), koi8r_general_ci with latin1_swedish_ci (func_test), latin1_general_ci with latin1_swedish_ci on letters and digits (view), cp932_japanese_ci with latin1_swedish_ci (subselect_sj). Such comparisons are now allowed. Cases with collations that are still incompatible on ASCII, e.g. latin7_general_ci, were added to the tests and still fail. - Adding a number of MTR tests in plugin/func_test/mysql-test/func_test/. They display a tailoring by collation name and repertoire as returned by: cs->coll->tailoring(cs, some_repertoire) A dynamically linked plugin function collation_tailoring() was added for the purpose of these tests. Tailorings do not depend on PAD/NOPAD and on the UCA version, so the function shows "[nopad]" for NOPAD collations, and "[version X.Y.Z]" for UCA collations on the ASCII repertoire. The version is formatted by a new class UCAVersion, which uses a new method CharBuffer::append_uint8(). - Adding the test plugin/func_test/mysql-test/func_test/ ctype_caseup_ci_verify.test. It checks the order of ASCII characters for all collations declared "caseup CI" on the ASCII or IDENT range. - Adding the test plugin/func_test/mysql-test/func_test/ collation_tailoring_args.test. It checks the arguments of the plugin function collation_tailoring(): the repertoire passed by name or by number, and the NULL results for unknown names. - Adding the test plugin/func_test/mysql-test/func_test/ctype_ldml.test. It displays the tailorings of collations loaded at runtime from Index.xml: simple 8bit collations (flags detected from the weights) and Unicode collations (LDML rules which do or do not reorder ASCII). - Adding a number of MTR tests mysql-test/main/ctype_xxx_tailoring.test They display two-dimensional charts showing which collations are compatible on which repertoires. Tests for DATE_FORMAT() with an explicit locale were added to ctype_big5_tailoring.test, ctype_cp932_tailoring.test and ctype_latin5_tailoring.test, which also has the chart for latin5. - Adding a number of MTR tests in the form of the originally reported script for various collations: SELECT Insert_priv FROM mysql.user WHERE Insert_priv='...'; The repeated blocks of queries are shared through mysql-test/include/ ctype_tailoring_0{1,2,3}_mysql_user_Insert_priv*.inc. - Adding tests to ctype_cp932.test checking that ExtractValue() and UpdateXML() still raise "Illegal mix of collations". |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexander Barkov
bar@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41246 "Illegal mix of collations" on the mysql.user view This SQL script failed: SET NAMES latin1 COLLATE latin1_swedish_ci; CREATE OR REPLACE VIEW v1 AS SELECT 'Y' AS c1; SET NAMES big5 COLLATE big5_chinese_ci; SELECT * FROM v1 WHERE c1='y'; with the following error: ERROR 1267 (HY000): Illegal mix of collations (latin1_swedish_ci,COERCIBLE) and (big5_chinese_ci,COERCIBLE) for operation '=' Note, latin1_swedish_ci and big5_chinese_ci are used here as examples. The error also happened with different collation combinations. Fix main idea: If two collations have equal comparison rules (known as "tailoring") on a given character repertoire, like latin1_swedish_ci and big5_chinese_ci on ASCII letters, then the "Illegal mix of collations" error can be avoided in a comparison operator. We can choose any of the sides as the operation effective collation - the result will be equal. This optimization is not applied when at least one side has an explicit COLLATE clause. Two explicit COLLATE clauses in one comparison are already illegal when the character sets are the same, so for consistency this stays illegal when the character sets differ too. Most important details: - Splitting enum_repertoire_t into smaller subsets, for better repertoire granularity. A variable holding a repertoire value can now have multiple MY_REPERTOIRE_XXX flags set. This patch implements detecting tailoring equality on these repertoires: * MY_REPERTOIRE_ASCII_DIGITS - [0..9] * MY_REPERTOIRE_ASCII_MINUS_DOT_DIGITS - [-.0..9], e.g. '-10.5'. It consists of the new bits MY_REPERTOIRE_ASCII_MINUS and MY_REPERTOIRE_ASCII_DOT. They were a part of MY_REPERTOIRE_ASCII_NOT_IDENT before. As a result, the numeric values of MY_REPERTOIRE_ASCII and MY_REPERTOIRE_UNICODE30 changed (223 and 255). * MY_REPERTOIRE_ASCII_ALNUM - [A..Z,a..z,0..9]. * MY_REPERTOIRE_ASCII_IDENT - ALNUM + underscore * MY_REPERTOIRE_ASCII - the entire range U+0000..U+007F - Adding a new virtual function "tailoring" in my_collation_handler_st. It returns the tailoring on the given repertoire for the given collation. If cs1->coll->tailoring(cs1, some_repertoire) returns {0,0}, it means the illegal mix optimization cannot be used for this collation on the given repertoire. If these calls: tr1= cs1->coll->tailoring(cs1, some_repertoire); tr2= cs2->coll->tailoring(cs2, some_repertoire); return both non-NULL results and tr1.str==tr2.str, then these collations are equal on the given repertoire and are mutually replaceable for a comparison operator, so "Illegal mix of collations" can be avoided. - Tailoring strings are shared constants my_tailoring_str_* (strings/strings_def.h). Tailorings are compared by the string pointer, so tailoring() implementations return strings only from these constants. The strings do not depend on PAD/NOPAD. - my_string_repertoire() for ucs2, utf16, utf32 and NONASCII 8bit character sets no longer stops at the first bad byte sequence with the repertoire found so far, which could be MY_REPERTOIRE_NONE (compatible with everything). It returns MY_REPERTOIRE_EXTENDED for such strings. A unit test was added into unittest/strings. - Adding CHARSET_INFO::is_ascii_superset(). It tells if a character set can store all ASCII characters (it is false for swe7). It is used in Item_func_conv_charset to calculate "safe", and in left_is_algorithmically_simpler() to prefer other character sets to those where ASCII strings cannot be converted safely. - DBUG_ASSERTs were added into mysys/charset.c, to check that the "tailoring" method is set in collation handlers when a collation is added: compiled-in, from ctype-extra.c, or loaded from Index.xml. - Adding a new function my_collations_equal_on_repertoire(cs1, cs2, repertoire) in strings/. It tells if two collations are equal on the repertoire, taking tailoring() results and PAD/NOPAD and the UCA version into account. Callers do not compare tailoring() results directly, so the way tailorings are compared (currently by the pointer to the shared constant string) is private to strings/ and can be extended later, e.g. for UCA collations on MY_REPERTOIRE_UNICODE30. - Adding a new method DTCollation::aggregate_by_tailoring(). Collations with different MY_CS_NOPAD are never compatible. UCA collations of different versions are not compatible on the repertoire with ASCII punctuation, as the UCA version affects its order (e.g. UCA-6.2.0 moved GRAVE ACCENT and CIRCUMFLEX ACCENT after PERCENT SIGN). - DTCollation::aggregate() now makes the repertoire of the result cover the repertoires of both sides in all successful branches. Before, set(dt) replaced the repertoire with the repertoire of the winner side only, so the result could have a too narrow repertoire, e.g. IF(1, 1.5, HEX(255)) had MY_REPERTOIRE_ASCII_ALNUM although the value '1.5' has a punctuation character. With the more granular repertoires this could wrongly allow mixing collations which are equal on ALNUM but different on punctuation. - Adding a new flag MY_COLL_ALLOW_BY_TAILORING. It indicates to DTCollation::aggregate() that the illegal mix optimization by repertoire can be used in the given context. MY_COLL_CMP_CONV now includes MY_COLL_ALLOW_BY_TAILORING. Note, only comparison operators pass this flag. Functions returning a string result do not pass this flag, because in operations like CONCAT(a,b) we still need to evaluate precisely the collation of the result - we cannot just choose a collation of one of the sides (even if they are compatible on the given repertoire). This also applies to ExtractValue() and UpdateXML(), which now aggregate their arguments without this flag. - As in my_repertoire_t the value MY_REPERTOIRE_ASCII is now a set of bits rather than a single bit, the way to detect "is only ASCII" repertoires has changed in the code. For example: // repertoire *IS* ascii if (repertoire == MY_REPERTOIRE_ASCII) has changed in multiple places in the code to // repertoire *HAS* only ascii characters if (my_repertoire_is_subset_of(repertoire, MY_REPERTOIRE_ASCII)) The new function my_repertoire_is_subset_of() in m_ctype.h is used for this purpose in C code. In C++ code DTCollation::repertoire_is_subset_of() is used, e.g.: if (collation.repertoire_is_subset_of(MY_REPERTOIRE_ASCII)) - Repertoire of some expressions was adjusted to the new meaning: * Item_null now has MY_REPERTOIRE_NONE (was ASCII). * MY_LOCALE::repertoire() now returns MY_REPERTOIRE_UNICODE30 (was EXTENDED). * Lex_string_with_metadata_st::repertoire(cs) now scans the string contents to detect the actual repertoire. * HEX() now has MY_REPERTOIRE_ASCII_ALNUM. * DATE_FORMAT() decides on MY_REPERTOIRE_EXTENDED using the locale which is actually used: the explicit third argument, or @@lc_time_names. It was always @@lc_time_names before. A non-constant locale argument is assumed to be non-ASCII. * QUOTE, MAKE_SET, EXPORT_SET, LPAD, RPAD, GROUP_CONCAT, JSON_ARRAY, JSON_OBJECT and JSON_OBJECTAGG add the repertoire of the extra characters they put into the result (quotes, separators, padding, brackets). * LOWER() and UPPER() add the letters of both cases to the repertoire, because the result can contain letters of the opposite case. In Turkish collations they also add MY_REPERTOIRE_EXTENDED, as an ASCII letter can be converted to a non-ASCII one (I -> dotless i). Unicode collations are detected by the new member casefold_info_st::can_convert_to_non_ascii_on_casefolding, simple 8bit collations by to_lower['I'] and to_upper['i']. - Lex_string_with_metadata_st::repertoire(cs) now scans a string literal for any byte oriented character set (mbminlen==1), not only for ASCII based ones. It was assumed to have the full repertoire for swe7, which is not ASCII based. The scan is done by Unicode code points, so swe7 strings with digits and letters get a narrow repertoire, while e.g. 0x5B (LATIN CAPITAL LETTER A WITH DIAERESIS in swe7) is non-ASCII. Therefore swe7_swedish_ci can be mixed with other collations now, in the same way as other collations with MY_CS_IDENT_CASEUP_CI. The test func_time was adjusted: a swe7 literal ' ull' with ASCII characters is now safe to convert, the error is still tested for '[ull'. - cp1250_czech_cs sorts -, . and 0..9 in the code point order, so it gets the MINUS_DOT_DIGITS tailoring. latin2_czech_cs gets only DIGITS, as it sorts the minus and the dot differently. Both were verified with ctype_digits_verify. - The tis620 collations now set CHARSET_INFO::tab_to_uni (was NULL), so my_charset_is_ascii_based() is true for tis620. It was false for tis620, although tis620 is ASCII compatible. Therefore tis620 string literals were always considered to have a non-ASCII repertoire, and the repertoire optimizations did not work for them. Now ASCII-only tis620 literals are detected as ASCII. This changes the result of subselect_extra_no_semijoin from an error to success. - tis620_thai_ci and tis620_thai_nopad_ci fold letters to lower case and then compare by the code point on the entire ASCII range (the Thai specific rules affect only non-ASCII characters). A new function my_tailoring_ascii_casedn_ci() returns tailorings for such collations on ALNUM, IDENT and ASCII. For example, the underscore sorts before the letters, unlike in the caseup tailorings. - New flags were added for CHARSET_INFO::state * MY_CS_ASCII_CASEUP_CI - for simple 8bit case insensitive collations. It means that this collation has no irregularities on the ASCII repertoire, and converts lower case letters to upper case ones, so the underscore sorts after the letters (with an upper to lower case conversion it would sort before the letters). * MY_CS_IDENT_CASEUP_CI - for simple 8bit case insensitive collations. It means that this collation has no irregularities on the IDENT repertoire (but can have irregularities say on punctuation). * MY_CS_ASCII_STD_UCA - for UCA collations. It means that a UCA collation does not reorder ASCII characters. * MY_CS_DIGITS_STD - for any collation. It means that the ASCII digits 0..9 are sorted in the code point order and are not tailored, and, for PAD collations, the space sorts before the digits. Strings of digits are compared in the same way in all such collations, even if they have irregularities on letters (latin7, cp866, czech, danish, turkish UCA, etc.), or differ by case sensitivity. It is detected from sort_order for 8bit collations, and from the tailoring rules for UCA collations. Collations with MY_CS_DIGITS_STD still must have the same PAD/NOPAD attribute. * MY_CS_ASCII_MINUS_DOT_DIGITS - for any collation. Means MY_CS_DIGITS_STD, and additionally the minus sorts before the dot, which sorts before the digits (and the space sorts before the minus in PAD collations). Strings like '-10.5' are compared in the same way in all such collations. It is detected from sort_order for simple 8bit collations (my_coll_init_simple()), and from the tailoring rules for UCA collations. For the multi-byte collations big5_chinese_ci, gbk_chinese_ci, gb2312_chinese_ci and sjis_japanese_ci it is declared by my_tailoring_ident_caseup_ci_generic(). The flags MY_CS_DIGITS_STD and MY_CS_ASCII_MINUS_DOT_DIGITS are stored in ctype-extra.c for compiled collations (conf_to_src.c prints them), and the detection is skipped for a collation which already has any of the repertoire flags (MY_CS_REPERTOIRE_FLAGS). - strings/conf_to_src.c was modified to detect and print MY_CS_ASCII_CASEUP_CI and MY_CS_IDENT_CASEUP_CI flags. - strings/ctype-extra.c was regenerated with new flags. - 8bit collations are not given MY_CS_ASCII_CASEUP_CI and MY_CS_IDENT_CASEUP_CI if they are PAD and the space does not sort before the digits. A shorter string is padded with spaces for comparison, so the position of the space matters even for strings without spaces ("ab" is compared to "abc" as "ab " to "abc"). A test collation latin1_space_after_letters_ci was added to mysql-test/std_data/ldml/ for this. ctype-extra.c does not change. - The initializer of my_collation_cs_handler in ctype-utf8.c (utf8mb3_general_cs, under HAVE_UTF8_GENERAL_CS) was fixed to match my_collation_handler_st: the missing get_id and get_collation_name members were added, so that eq_collation was not initialized in their place, and the new member "tailoring" is now set to my_tailoring_none. - Results of the existing tests func_str, func_test, ps, view and subselect_sj changed from "Illegal mix of collations" to success. In these tests ASCII-only strings were compared using collations with equal tailoring on ASCII, e.g. latin1_swedish_ci with latin2_general_ci (func_str, ps), koi8r_general_ci with latin1_swedish_ci (func_test), latin1_general_ci with latin1_swedish_ci on letters and digits (view), cp932_japanese_ci with latin1_swedish_ci (subselect_sj). Such comparisons are now allowed. Cases with collations that are still incompatible on ASCII, e.g. latin7_general_ci, were added to the tests and still fail. - Adding a number of MTR tests in plugin/func_test/mysql-test/func_test/. They display a tailoring by collation name and repertoire as returned by: cs->coll->tailoring(cs, some_repertoire) A dynamically linked plugin function collation_tailoring() was added for the purpose of these tests. Tailorings do not depend on PAD/NOPAD and on the UCA version, so the function shows "[nopad]" for NOPAD collations, and "[version X.Y.Z]" for UCA collations on the ASCII repertoire. The version is formatted by a new class UCAVersion, which uses a new method CharBuffer::append_uint8(). - Adding the test plugin/func_test/mysql-test/func_test/ ctype_caseup_ci_verify.test. It checks the order of ASCII characters for all collations declared "caseup CI" on the ASCII or IDENT range. - Adding the test plugin/func_test/mysql-test/func_test/ collation_tailoring_args.test. It checks the arguments of the plugin function collation_tailoring(): the repertoire passed by name or by number, and the NULL results for unknown names. - Adding the test plugin/func_test/mysql-test/func_test/ctype_ldml.test. It displays the tailorings of collations loaded at runtime from Index.xml: simple 8bit collations (flags detected from the weights) and Unicode collations (LDML rules which do or do not reorder ASCII). - Adding a number of MTR tests mysql-test/main/ctype_xxx_tailoring.test They display two-dimensional charts showing which collations are compatible on which repertoires. Tests for DATE_FORMAT() with an explicit locale were added to ctype_big5_tailoring.test, ctype_cp932_tailoring.test and ctype_latin5_tailoring.test, which also has the chart for latin5. - Adding a number of MTR tests in the form of the originally reported script for various collations: SELECT Insert_priv FROM mysql.user WHERE Insert_priv='...'; The repeated blocks of queries are shared through mysql-test/include/ ctype_tailoring_0{1,2,3}_mysql_user_Insert_priv*.inc. - Adding tests to ctype_cp932.test checking that ExtractValue() and UpdateXML() still raise "Illegal mix of collations". |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-35592 Duplicate entry 'NULL' for key 'key0' with subquery cache Expression cache lookup is null rejecting, so parameters with NULL never hit and were inserted into the cache table again and again. After conversion of the cache to an on-disk table (where NULLs are equal) such a repeated insert failed with ER_DUP_ENTRY. Do not store rows which can not be looked up. On 11.4 the cache is just switched off when the HEAP table is full, so the test fails only after MDEV-9101 (11.5+, ES 11.4 via MENT-2067). Co-Authored-By: Claude Opus 5.5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
ParadoxV5
paradox.ver5@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40996 Support `--sync_with_master 0, $variable` in mysqltest `--sync_with_master` uses `get_string()`, which has `$variable` support, but it only uses the read buffer, which is written with the unexpanded string and not the variable value. Reviewed-by: KhaledR57 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41252: partial join cost Assertion failure in recompute_join_cost_with_limit() recompute_join_cost_with_limit() computes the cost of the first table's partial join as best_read*fraction - pos->read_time*fraction. When the two costs are nearly equal and large (e.g. with a huge optimizer_scan_setup_cost, or when fraction is close to 1), the two products are rounded independently. The difference can then be a small negative number whose magnitude exceeds the absolute DBL_EPSILON tolerance used by the debug assertion, even though it is only a floating-point rounding artifact. Fix: scale the assertion tolerance with the magnitude of the operands (DBL_EPSILON * pos->read_time * fraction). Negative values are still clamped to 0.0 as before. Add test cases to optimizer_crash.test, one with a very large optimizer_scan_setup_cost and optimizer_join_limit_pref_ratio=1, and one with a large LIMIT on a 30000-row table. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge 10.11 into 11.4 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41012 Galera appliers hang with foreign key of types UUID, INET4, INET6 For direct row writes, the certification key for MYSQL_TYPE_STRING and MYSQL_TYPE_VARSTRING is built by collating the column value and taking the collation from Field::charset(). The data types implemented on Field_fbt - UUID, INET6 and INET4 report MYSQL_TYPE_STRING, and their charset() is my_charset_numeric, which is latin1. Their values are however plain binary and accordingly innodb maps them to DATA_FIXBINARY. Their keys were therefore run through latin1_swedish_ci, which folds them. That corrupts the key in two ways: 1. A key mismatch for the same row. The FK constraint's referenced key that is appended for the parent of a child INSERT is built from the InnoDB record and is not collated, and it does not match the primary key appended due to the parent row's direct write. Certification saw no dependency between a child INSERT and a concurrent parent UPDATE, and two appliers could apply them in parallel causing a hang or crash. 2. A key collision between distinct rows. The folding is many to one, so different values collapse onto one key, Certification compares keys byte for byte, so unrelated rows were treated as the same row. Concurrent transactions on them certified as a conflict and one was aborted with ER_LOCK_DEADLOCK. Fix is for wsrep_store_key_val_for_row() to skip the collation for fields that InnoDB stores as binary, using the same condition as get_innobase_type_from_mysql_type(). This is a no-op for the types that worked before. This change requires to bump the application protocol version to level 5. The commit has also two mtr tests for regression testing. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| extend copilot review insttructions with AI attribution policy | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Jacob Williams
jacob@iakob.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38757 Fix EXCHANGE PARTITION with generated columns containing AND/OR conditions EXCHANGE PARTITION fails with ERROR 1736 (Tables have different definitions) when tables contain generated columns with AND/OR conditions, even when the expressions are logically equivalent. This occurs because when expressions are re-parsed (e.g., via CREATE TABLE ... LIKE), the order of arguments in AND/OR conditions may change, but the comparison was order-sensitive. The Item_cond::eq() method was implemented to perform set-based comparison for commutative AND/OR operations. The set-based comparison algorithm ensures that two Item_cond expressions are considered equal if they contain the same set of equivalent arguments, regardless of order. Added comprehensive test case covering: - Generated columns with OR conditions - Generated columns with AND conditions - Multiple generated columns with different AND/OR combinations - Nested AND/OR conditions The fix allows EXCHANGE PARTITION to succeed when expressions are logically equivalent but have different argument ordering, which is correct behavior since AND/OR operations are commutative. MDEV-38757 Limit unordered vcol condition comparison to EXCHANGE PARTITION Review follow-up to the previous commit, which made Item_cond::eq() compare AND/OR argument lists as sets for every caller. That changed equality semantics globally and broke main.derived_cond_pushdown, where conditions that eq() started reporting as equal were dropped from attached_condition. Reordering AND/OR operands also changes evaluation order, which is observable when operands are functions, so the relaxed comparison must not be the default. Item::Eq_config gains an unordered_conditions flag, defaulting to false, next to the existing binary_cmp and omit_table_names flags. Item_cond::eq() compares its argument lists as sets only when that flag is set, and otherwise reports two distinct Item_cond objects as unequal, as it did before this patch series. The flag is threaded through Virtual_column_info::is_equal() and a new mysql_compare_tables() parameter, and only Sql_cmd_alter_table_exchange_partition passes it as true, so the relaxed comparison stays confined to the EXCHANGE PARTITION metadata check. The set comparison tests containment in both directions rather than comparing element counts, so an expression also matches a form that repeats one of its terms, for example col1 > 10 and col2 < 100 or col3 > 50 col3 > 50 or col1 > 10 and col2 < 100 or col3 > 50 Test 5 of parts.partition_exchange_generated_columns covers that case. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thirunarayanan Balathandayuthapani
thiru@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41548 InnoDB: Cannot replay rename … because the target file exists Problem: ======= recv_rename_files() replayed the renames in tablespace id order, because renamed_spaces was a std::map keyed by space id that kept only the latest file name per tablespace. A multi-table RENAME such as RENAME TABLE t2 TO t3, t1 TO t2 frees a file name for a tablespace that may have a lower id, and replaying in id order then fails with "Cannot replay rename of tablespace … because the target file exists". Solution: ======== renamed_spaces: Change to a std::vector of (id, from, to), retaining every FILE_RENAME record in log order. The records are appended only while recv_sys.file_checkpoint is set, hence exactly once and in ascending LSN. recv_rename_files(): Replay the records in that order, applying one only when the tablespace currently uses the file name that the record renames from. The order in which the server executed the renames is always replayable, because each rename required its target to be unoccupied. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Petrunia
sergey@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Add dbug_get_mem_root_alloc_size() debug helper | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexander Barkov
bar@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41246 "Illegal mix of collations" on the mysql.user view This SQL script failed: SET NAMES latin1 COLLATE latin1_swedish_ci; CREATE OR REPLACE VIEW v1 AS SELECT 'Y' AS c1; SET NAMES big5 COLLATE big5_chinese_ci; SELECT * FROM v1 WHERE c1='y'; with the following error: ERROR 1267 (HY000): Illegal mix of collations (latin1_swedish_ci,COERCIBLE) and (big5_chinese_ci,COERCIBLE) for operation '=' Note, latin1_swedish_ci and big5_chinese_ci are used here as examples. The error also happened with different collation combinations. Fix main idea: If two collations have equal comparison rules (known as "tailoring") on a given character repertoire, like latin1_swedish_ci and big5_chinese_ci on ASCII letters, then the "Illegal mix of collations" error can be avoided in a comparison operator. We can choose any of the sides as the operation effective collation - the result will be equal. This optimization is not applied when at least one side has an explicit COLLATE clause. Two explicit COLLATE clauses in one comparison are already illegal when the character sets are the same, so for consistency this stays illegal when the character sets differ too. Most important details: - Splitting enum_repertoire_t into smaller subsets, for better repertoire granularity. A variable holding a repertoire value can now have multiple MY_REPERTOIRE_XXX flags set. This patch implements detecting tailoring equality on these repertoires: * MY_REPERTOIRE_ASCII_DIGITS - [0..9] * MY_REPERTOIRE_ASCII_MINUS_DOT_DIGITS - [-.0..9], e.g. '-10.5'. It consists of the new bits MY_REPERTOIRE_ASCII_MINUS and MY_REPERTOIRE_ASCII_DOT. They were a part of MY_REPERTOIRE_ASCII_NOT_IDENT before. As a result, the numeric values of MY_REPERTOIRE_ASCII and MY_REPERTOIRE_UNICODE30 changed (223 and 255). * MY_REPERTOIRE_ASCII_ALNUM - [A..Z,a..z,0..9]. * MY_REPERTOIRE_ASCII_IDENT - ALNUM + underscore * MY_REPERTOIRE_ASCII - the entire range U+0000..U+007F - Adding a new virtual function "tailoring" in my_collation_handler_st. It returns the tailoring on the given repertoire for the given collation. If cs1->coll->tailoring(cs1, some_repertoire) returns {0,0}, it means the illegal mix optimization cannot be used for this collation on the given repertoire. If these calls: tr1= cs1->coll->tailoring(cs1, some_repertoire); tr2= cs2->coll->tailoring(cs2, some_repertoire); return both non-NULL results and tr1.str==tr2.str, then these collations are equal on the given repertoire and are mutually replaceable for a comparison operator, so "Illegal mix of collations" can be avoided. - Tailoring strings are shared constants my_tailoring_str_* (strings/strings_def.h). Tailorings are compared by the string pointer, so tailoring() implementations return strings only from these constants. The strings do not depend on PAD/NOPAD. - my_string_repertoire() for ucs2, utf16, utf32 and NONASCII 8bit character sets no longer stops at the first bad byte sequence with the repertoire found so far, which could be MY_REPERTOIRE_NONE (compatible with everything). It returns MY_REPERTOIRE_EXTENDED for such strings. A unit test was added into unittest/strings. - Adding CHARSET_INFO::is_ascii_superset(). It tells if a character set can store all ASCII characters (it is false for swe7). It is used in Item_func_conv_charset to calculate "safe", and in left_is_algorithmically_simpler() to prefer other character sets to those where ASCII strings cannot be converted safely. - DBUG_ASSERTs were added into mysys/charset.c, to check that the "tailoring" method is set in collation handlers when a collation is added: compiled-in, from ctype-extra.c, or loaded from Index.xml. - Adding a new function my_collations_equal_on_repertoire(cs1, cs2, repertoire) in strings/. It tells if two collations are equal on the repertoire, taking tailoring() results and PAD/NOPAD and the UCA version into account. Callers do not compare tailoring() results directly, so the way tailorings are compared (currently by the pointer to the shared constant string) is private to strings/ and can be extended later, e.g. for UCA collations on MY_REPERTOIRE_UNICODE30. - Adding a new method DTCollation::aggregate_by_tailoring(). Collations with different MY_CS_NOPAD are never compatible. UCA collations of different versions are not compatible on the repertoire with ASCII punctuation, as the UCA version affects its order (e.g. UCA-6.2.0 moved GRAVE ACCENT and CIRCUMFLEX ACCENT after PERCENT SIGN). - DTCollation::aggregate() now makes the repertoire of the result cover the repertoires of both sides in all successful branches. Before, set(dt) replaced the repertoire with the repertoire of the winner side only, so the result could have a too narrow repertoire, e.g. IF(1, 1.5, HEX(255)) had MY_REPERTOIRE_ASCII_ALNUM although the value '1.5' has a punctuation character. With the more granular repertoires this could wrongly allow mixing collations which are equal on ALNUM but different on punctuation. - Adding a new flag MY_COLL_ALLOW_BY_TAILORING. It indicates to DTCollation::aggregate() that the illegal mix optimization by repertoire can be used in the given context. MY_COLL_CMP_CONV now includes MY_COLL_ALLOW_BY_TAILORING. Note, only comparison operators pass this flag. Functions returning a string result do not pass this flag, because in operations like CONCAT(a,b) we still need to evaluate precisely the collation of the result - we cannot just choose a collation of one of the sides (even if they are compatible on the given repertoire). This also applies to ExtractValue() and UpdateXML(), which now aggregate their arguments without this flag. - As in my_repertoire_t the value MY_REPERTOIRE_ASCII is now a set of bits rather than a single bit, the way to detect "is only ASCII" repertoires has changed in the code. For example: // repertoire *IS* ascii if (repertoire == MY_REPERTOIRE_ASCII) has changed in multiple places in the code to // repertoire *HAS* only ascii characters if (my_repertoire_is_subset_of(repertoire, MY_REPERTOIRE_ASCII)) The new function my_repertoire_is_subset_of() in m_ctype.h is used for this purpose in C code. In C++ code DTCollation::repertoire_is_subset_of() is used, e.g.: if (collation.repertoire_is_subset_of(MY_REPERTOIRE_ASCII)) - Repertoire of some expressions was adjusted to the new meaning: * Item_null now has MY_REPERTOIRE_NONE (was ASCII). * MY_LOCALE::repertoire() now returns MY_REPERTOIRE_UNICODE30 (was EXTENDED). * Lex_string_with_metadata_st::repertoire(cs) now scans the string contents to detect the actual repertoire. * HEX() now has MY_REPERTOIRE_ASCII_ALNUM. * DATE_FORMAT() decides on MY_REPERTOIRE_EXTENDED using the locale which is actually used: the explicit third argument, or @@lc_time_names. It was always @@lc_time_names before. A non-constant locale argument is assumed to be non-ASCII. * QUOTE, MAKE_SET, EXPORT_SET, LPAD, RPAD, GROUP_CONCAT, JSON_ARRAY, JSON_OBJECT and JSON_OBJECTAGG add the repertoire of the extra characters they put into the result (quotes, separators, padding, brackets). * LOWER() and UPPER() add the letters of both cases to the repertoire, because the result can contain letters of the opposite case. In Turkish collations they also add MY_REPERTOIRE_EXTENDED, as an ASCII letter can be converted to a non-ASCII one (I -> dotless i). Unicode collations are detected by the new member casefold_info_st::can_convert_to_non_ascii_on_casefolding, simple 8bit collations by to_lower['I'] and to_upper['i']. - Lex_string_with_metadata_st::repertoire(cs) now scans a string literal for any byte oriented character set (mbminlen==1), not only for ASCII based ones. It was assumed to have the full repertoire for swe7, which is not ASCII based. The scan is done by Unicode code points, so swe7 strings with digits and letters get a narrow repertoire, while e.g. 0x5B (LATIN CAPITAL LETTER A WITH DIAERESIS in swe7) is non-ASCII. Therefore swe7_swedish_ci can be mixed with other collations now, in the same way as other collations with MY_CS_IDENT_CASEUP_CI. The test func_time was adjusted: a swe7 literal ' ull' with ASCII characters is now safe to convert, the error is still tested for '[ull'. - cp1250_czech_cs sorts -, . and 0..9 in the code point order, so it gets the MINUS_DOT_DIGITS tailoring. latin2_czech_cs gets only DIGITS, as it sorts the minus and the dot differently. Both were verified with ctype_digits_verify. - The tis620 collations now set CHARSET_INFO::tab_to_uni (was NULL), so my_charset_is_ascii_based() is true for tis620. It was false for tis620, although tis620 is ASCII compatible. Therefore tis620 string literals were always considered to have a non-ASCII repertoire, and the repertoire optimizations did not work for them. Now ASCII-only tis620 literals are detected as ASCII. This changes the result of subselect_extra_no_semijoin from an error to success. - tis620_thai_ci and tis620_thai_nopad_ci fold letters to lower case and then compare by the code point on the entire ASCII range (the Thai specific rules affect only non-ASCII characters). A new function my_tailoring_ascii_casedn_ci() returns tailorings for such collations on ALNUM, IDENT and ASCII. For example, the underscore sorts before the letters, unlike in the caseup tailorings. - New flags were added for CHARSET_INFO::state * MY_CS_ASCII_CASEUP_CI - for simple 8bit case insensitive collations. It means that this collation has no irregularities on the ASCII repertoire, and converts lower case letters to upper case ones, so the underscore sorts after the letters (with an upper to lower case conversion it would sort before the letters). * MY_CS_IDENT_CASEUP_CI - for simple 8bit case insensitive collations. It means that this collation has no irregularities on the IDENT repertoire (but can have irregularities say on punctuation). * MY_CS_ASCII_STD_UCA - for UCA collations. It means that a UCA collation does not reorder ASCII characters. * MY_CS_DIGITS_STD - for any collation. It means that the ASCII digits 0..9 are sorted in the code point order and are not tailored, and, for PAD collations, the space sorts before the digits. Strings of digits are compared in the same way in all such collations, even if they have irregularities on letters (latin7, cp866, czech, danish, turkish UCA, etc.), or differ by case sensitivity. It is detected from sort_order for 8bit collations, and from the tailoring rules for UCA collations. Collations with MY_CS_DIGITS_STD still must have the same PAD/NOPAD attribute. * MY_CS_ASCII_MINUS_DOT_DIGITS - for any collation. Means MY_CS_DIGITS_STD, and additionally the minus sorts before the dot, which sorts before the digits (and the space sorts before the minus in PAD collations). Strings like '-10.5' are compared in the same way in all such collations. It is detected from sort_order for simple 8bit collations (my_coll_init_simple()), and from the tailoring rules for UCA collations. For the multi-byte collations big5_chinese_ci, gbk_chinese_ci, gb2312_chinese_ci and sjis_japanese_ci it is declared by my_tailoring_ident_caseup_ci_generic(). The flags MY_CS_DIGITS_STD and MY_CS_ASCII_MINUS_DOT_DIGITS are stored in ctype-extra.c for compiled collations (conf_to_src.c prints them), and the detection is skipped for a collation which already has any of the repertoire flags (MY_CS_REPERTOIRE_FLAGS). - strings/conf_to_src.c was modified to detect and print MY_CS_ASCII_CASEUP_CI and MY_CS_IDENT_CASEUP_CI flags. - strings/ctype-extra.c was regenerated with new flags. - 8bit collations are not given MY_CS_ASCII_CASEUP_CI and MY_CS_IDENT_CASEUP_CI if they are PAD and the space does not sort before the digits. A shorter string is padded with spaces for comparison, so the position of the space matters even for strings without spaces ("ab" is compared to "abc" as "ab " to "abc"). A test collation latin1_space_after_letters_ci was added to mysql-test/std_data/ldml/ for this. ctype-extra.c does not change. - The initializer of my_collation_cs_handler in ctype-utf8.c (utf8mb3_general_cs, under HAVE_UTF8_GENERAL_CS) was fixed to match my_collation_handler_st: the missing get_id and get_collation_name members were added, so that eq_collation was not initialized in their place, and the new member "tailoring" is now set to my_tailoring_none. - Results of the existing tests func_str, func_test, ps, view and subselect_sj changed from "Illegal mix of collations" to success. In these tests ASCII-only strings were compared using collations with equal tailoring on ASCII, e.g. latin1_swedish_ci with latin2_general_ci (func_str, ps), koi8r_general_ci with latin1_swedish_ci (func_test), latin1_general_ci with latin1_swedish_ci on letters and digits (view), cp932_japanese_ci with latin1_swedish_ci (subselect_sj). Such comparisons are now allowed. Cases with collations that are still incompatible on ASCII, e.g. latin7_general_ci, were added to the tests and still fail. - Adding a number of MTR tests in plugin/func_test/mysql-test/func_test/. They display a tailoring by collation name and repertoire as returned by: cs->coll->tailoring(cs, some_repertoire) A dynamically linked plugin function collation_tailoring() was added for the purpose of these tests. Tailorings do not depend on PAD/NOPAD and on the UCA version, so the function shows "[nopad]" for NOPAD collations, and "[version X.Y.Z]" for UCA collations on the ASCII repertoire. The version is formatted by a new class UCAVersion, which uses a new method CharBuffer::append_uint8(). - Adding the test plugin/func_test/mysql-test/func_test/ ctype_caseup_ci_verify.test. It checks the order of ASCII characters for all collations declared "caseup CI" on the ASCII or IDENT range. - Adding the test plugin/func_test/mysql-test/func_test/ collation_tailoring_args.test. It checks the arguments of the plugin function collation_tailoring(): the repertoire passed by name or by number, and the NULL results for unknown names. - Adding the test plugin/func_test/mysql-test/func_test/ctype_ldml.test. It displays the tailorings of collations loaded at runtime from Index.xml: simple 8bit collations (flags detected from the weights) and Unicode collations (LDML rules which do or do not reorder ASCII). - Adding a number of MTR tests mysql-test/main/ctype_xxx_tailoring.test They display two-dimensional charts showing which collations are compatible on which repertoires. Tests for DATE_FORMAT() with an explicit locale were added to ctype_big5_tailoring.test, ctype_cp932_tailoring.test and ctype_latin5_tailoring.test, which also has the chart for latin5. - Adding a number of MTR tests in the form of the originally reported script for various collations: SELECT Insert_priv FROM mysql.user WHERE Insert_priv='...'; The repeated blocks of queries are shared through mysql-test/include/ ctype_tailoring_0{1,2,3}_mysql_user_Insert_priv*.inc. - Adding tests to ctype_cp932.test checking that ExtractValue() and UpdateXML() still raise "Illegal mix of collations". |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41540 filesort assert crash on a huge sort key sort key could go over 4GB making sort length to wrap around. let's limit sort key length by 2GB - still ridiculously unrealistically large but far enough from 4GB so that any additions in the code below would still be safe. The error is somewhat confusing, but, again, only someone who pushes the margins to exploit memory bugs will ever see it. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Otto Kekäläinen
otto@kekalainen.net |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Promote getting GitHub stars in client prompt Ask users to give MariaDB a star by having an extra line in the MariaDB client prompt: Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is X Server version: Y Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Help others discover MariaDB. Star it on GitHub: https://github.com/MariaDB/server Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> Test file updated with: nano --noconvert --nonewlines mysql-test/main/mysql-interactive.result This change is similar to 346c7afe9b7071ce9c47892a83d69944b608b3da applied on 'main', but without the SERVER_MATURITY_LEVEL check in order to have this text visible in stable releases and actually help get the GitHub star count up, and without the server log entry. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39183 CONNECT JDBC: JVM is loaded from bin\client on Windows GetJVM() only looked for %JAVA_HOME%\bin\client\jvm.dll, and for the RuntimeLib registry value of the "Java Runtime Environment" key. Since JDK 9 there is no "client" JVM, jvm.dll is in bin\server, and the JDK installers write JavaHome, but not RuntimeLib, under JavaSoft\JDK. Look for jvm.dll the way Java installations are laid out. Prefer bin\server over bin\client, in the JDK and in its jre subdirectory. Without connect_jvm_path (which stays exclusive, as before), try JAVA_HOME first, then the JavaSoft\{JDK|JRE} registry keys: CurrentVersion, then the RuntimeLib or JavaHome value of that version. Keep the first meaningful LoadLibrary or registry error (e.g. a 32/64 bit mismatch) rather than the "not found" of the last probe, reject over-long paths instead of truncating them, accept REG_EXPAND_SZ values, ignore an empty JAVA_HOME, and remove a stray printf to stdout. Tested manually with Oracle JDK 23: with JAVA_HOME unset (registry), with JAVA_HOME set, and with an invalid JAVA_HOME falling back to the registry. Assisted-By: Claude:claude-5.5-sonnet |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41553 galera_restart_on_unknown_option test regression after MDEV-40023 The problem with the test was that it restarted node 2 by writing "try:--galera-unknown-option" into the expect file. However, MariaDB mtr has never supported a "try" command. Instead mtr skipped any expect file line not starting with "restart", and node 2 was not started with the unknown option (which was the idea in the test logic). The test only slept and then restarted node 2 normally. MDEV-40023 made mtr report unknown expect file commands as errors, so the test now fails in 13.1 and later. The fix is to start node 2 with the unknown option directly with --exec. In cluster, the unknown option is found only after the state transfer has started. The server should interrupt the state transfer and abort. Galera version 26.4.25 or later is needed for graceful shutdown, and this is now required by the test. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-32401 expression cache lead to crash if table of wrong type created and the cache switched off 1) take into account TMP_TABLE_ALL_COLUMNS when we are modifying agg_item->result_field 2) remove unused now "bool materialized_subquery;" Co-Authored-By: Claude Opus 5.5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
GoldenEmperor1177
xurbanconcept1@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-31225 Crash converting IN with a nested ROW into IN subquery What was wrong: With in_predicate_conversion_threshold set low enough, an IN predicate with a nested ROW, e.g. ROW(a,(a,a)) IN ((1,(1,1)),(2,(2,NULL))) crashed the server. Before an IN list is converted into an IN subquery over a table value constructor, cmp_row_types() checks every column with subquery_type_allows_materialization(). With a nested ROW the column is a ROW itself, and for a ROW that function must not be called. How it is fixed: A table value constructor cannot have a ROW as a column, so such a predicate cannot be converted anyway. cmp_row_types() now reports the columns as not comparable when either of them is a ROW, and the IN predicate is evaluated without the conversion. How it is tested: Added a test to main.opt_tvc with IN and NOT IN over nested ROWs. It crashes the server without the fix. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41540 filesort assert crash on a huge sort key sort key could go over 4GB making sort length to wrap around. let's limit sort key length by 2GB - still ridiculously unrealistically large but far enough from 4GB so that any additions in the code below would still be safe. The error is somewhat confusing, but, again, only someone who pushes the margins to exploit memory bugs will ever see it. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alessandro Vetere
iminelink@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38056 Assertion 'bpage->state() >= buf_page_t::UNFIXED' in buf_page_get_zip() Three callers reconstruct old versions of a clustered index record, derive secondary index entries from them, and dereference the BLOB pointers of externally stored columns: row_vers_impl_x_locked_low() for the implicit lock check, row_undo_mod_sec_is_unsafe() for rollback, and row_check_index() for CHECK TABLE ... EXTENDED. purge_sys.view is what keeps those pages allocated, but purge_sys_t::view_guard froze it only for the duration of trx_undo_prev_version_build(), and the dereference happens after. On ROW_FORMAT=COMPRESSED this trips the assertion above; in a release build the freed page is read anyway, which is silent corruption. All three now hold purge_sys.latch across the dereference, and only where the version has externally stored columns, since row_build() dereferences nothing otherwise. In row_check_index() the freeze spans the whole comparison, which fetches one field at a time and may evaluate a virtual column expression; for the latest version it is also confined to a delete-marked record, the only kind that need not own what it points at. Freezing that late means the view may have advanced since the walk decided a version was reachable, so each caller re-establishes that under the freeze; trx_undo_prev_version_build() states the condition and why testing the oldest writer applied suffices. The first two callers can treat that test as an invariant and end the walk where it fails. row_check_index() cannot, because it decides reachability from the lagging purge_sys.end_view on purpose, and that lag is how it finds orphan secondary index records: where the test fails it stops and reports nothing rather than treating the record as an orphan, and no report is lost for good, end_view eventually reaching the same point. Its two purge_sys.is_purgeable() tests now read the frozen view wherever a fetch follows, which makes them atomic with the fetch they guard. trx_undo_prev_version_build(): remove the gate that was meant to stop CHECK TABLE ... EXTENDED from fetching BLOBs it may no longer own, and view_guard::is_extended() with it, which no guard mode could satisfy. That decision belongs to the caller, the only one that knows whether it will dereference anything. row_log_table_get_pk(): document why the online ALTER path may dereference without a freeze. Debug-only keywords. purge_hold_cleanup parks a purge batch between its last purged record and purge_sys_t::batch_cleanup(), the window in which a reader that goes by purge_sys.end_view can still reach history the batch has removed; a batch opens and closes it without ever returning to the test. purge_no_blob_freeze sends a version that does have externally stored columns down the path one without any takes, which is what all four call sites did before this change, and reproduces the assertion above. row_vers_impl_x_locked_purgeable, row_undo_mod_sec_is_unsafe_purgeable and row_check_index_purgeable force the re-validation to fail, reaching exits that purge_sys.view advancing mid-walk otherwise produces. The first reports no implicit lock for a row that a live transaction still holds, so a test may only check that nothing breaks; the other two make the server more cautious. Tests. old_blob and old_blob_updel cover the implicit lock check, old_blob_rollback the rollback, old_blob_check CHECK TABLE ... EXTENDED. Each parks a walk at a dereference, makes the BLOB freeable, and asserts that the counter of purged update records, which is what would free it, stays at zero while parked and advances once the walk is over. old_blob_updel covers an undo log record that stores only the 20-byte reference and needs purge_hold_cleanup to reach it; old_blob_rollback parks at a reference that the version merely inherited. All four fail with the original assertion under debug_dbug=+d,purge_no_blob_freeze, and skip above a 16k page size, which ROW_FORMAT=COMPRESSED requires. old_blob_purgeable drives the three re-validation exits, needs no synchronisation, and runs at every page size. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||