Console View
|
Categories: connectors experimental galera main |
|
| connectors | experimental | galera | main | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41012 Galera appliers hang with foreign key of types UUID, INET4, INET6 Bumped application protocol version to level 5 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Don't use an MVI for an depth-2 OVERLAPS array with no keys An element that is itself an array is flattened, the same way MVI_ENCODE flattens the document. JSON_OVERLAPS does not flatten: it only matches such an element against a document element that is an array too, compared whole (json_compare_arrays_in_order()). The flattening here is still safe as it will produce only false positives that will be eliminated by a recheck. The only exception is when the nested array yields no key at all i.e. [], [[]], [[],[]], [[[]]], etc. Such an element may match a document element that has no key of ours either, so nothing we could search for would find that row. Give up in this case, as for a failed encoding. select json_overlaps('[[]]', '[[], "aaa"]'); is true, but the document has no key in the index and the scan for the "aaa" key does not return it. Give up on the access in that case, as we already do for an element that cannot be encoded. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Make an over-long key a prefix key instead of no key A key image that does not fit in a fulltext token was rejected, which costs the index for that value entirely: no key in the document, and an OVERLAPS that mentions the value gives up on the index altogether. Cut the image down to MVI_KEY_IMAGE_MAX_LEN instead. Two values that agree on that many bytes then share a key, which costs false positives and nothing else, since the predicate is rechecked on every row the index produces. This is what the non-binary path has always done -- strnxfrm() is asked for exactly that many bytes of weights and cannot return more -- so it makes the binary path, which is the one a JSON column takes, behave the same. No wildcard is needed in the fulltext query for this. Both sides of the index cut at the same point, so the key a search builds for a long value is the same string as the token the document has for it, and an exact term match finds it. A trailing '*' would only widen the term to keys that are longer than the one searched for, and after the cut there are none. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Check the keys against the engine's fulltext token sizes Do this at both DDL and optimizer Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Kristian Nielsen
knielsen@knielsen-hq.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40745: Validate field parsing of optional_metadata Protect accesses into the m_optional_metadata buffer to not access outside of the buffer in case of corrupt/malicious event data. Protect against buffer overflow of the m_column_name array in case of excessive column names in the event data. Signed-off-by: Kristian Nielsen <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge branch '13.0' into 13.1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Petrunia
sergey@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| More comments, code readability. No functional changes. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
KhaledR57
khaled57.dev@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-37167 Nested BEGINs (4600+) cause a segmentation fault Each nested BEGIN adds one sp_pcontext. Two walks over the finished tree recursed once per nesting level and overran the thread stack. ~sp_pcontext() freed its children recursively. sp_head now threads a single linked list through the contexts and frees them iteratively, so teardown depth is constant and a context no longer frees its children. retrieve_field_definitions() descends the children to build the run-time frame. It emits them in run-time offset order, so it stays recursive, but it now checks the stack and returns an error instead of crashing. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alessandro Vetere
iminelink@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-32286 Reuse remembered clustered leaves in secondary-index scans Row_sel_get_clust_rec_for_mysql::operator() descends the clustered B-tree from the root for every row whose clustered-index record a secondary-index scan must read, although consecutive rows land on the same clustered leaf page wherever the secondary order tracks the clustered one. A non-covering scan needs one for every row, a locking read needs one whatever the secondary index holds, because an exclusive select lock type makes ha_innobase::build_template() build its template against the clustered index, and a covering scan needs one for every row of a secondary leaf whose PAGE_MAX_TRX_ID its read view cannot see. ANALYZE FORMAT=JSON charges each descent its full height, and those descents are nearly the whole cost: the secondary index is charged its own descent and one page for each further leaf, and nothing per row, because the position that its cursor holds between two rows is restored optimistically, which latches the leaf again without counting an access. So pages_accessed is the row count times the height of the clustered index, plus a handful: 1000 rows over a 2-level clustered index cost 2006 and 750 rows over a 3-level one cost 2291, where a full table scan of the same data costs 23 and 110. Let a handle remember the clustered leaves that the lookups of one statement reached, and let the next lookup try them before it descends again. The reasoning behind each value and each rejection is in the comments beside it. row0mysql.h defines clust_leaf_hint_slot, which names one leaf: its page number, copies of its first and last user record truncated to the key fields, which bound the key range that the leaf held when it was remembered, the rec_get_offsets() of both, and the dict_index_t::n_core_fields that the copies were made under. A slot of a leaf that had no right sibling names no last key, because every key above the last record of the rightmost leaf still belongs to it. CLUST_LEAF_HINT_SLOTS (4) slots hang off the new row_prebuilt_t::clust_leaf_hint, beside clust_leaf_hint_mru, the most recently used order held as slot numbers, and clust_leaf_hint_n and clust_leaf_hint_miss, the used-slot count and the miss counter. row0sel.cc holds the policy. row_sel_clust_leaf_hint_covers() compares a key against the remembered ranges, so a lookup that no slot can answer costs no buffer pool access and no pages_accessed. row_sel_clust_leaf_hint_search() probes the first slot that covers the key and moves it to the front of the order. row_sel_clust_leaf_hint_remember() records the leaf that a descent landed on, and refreshes the slot of a leaf that is remembered already rather than spend a second one on the same page. Two descents fill no slot: the first CLUST_LEAF_HINT_MIN_LOOKUPS (4) lookups of a statement, and a leaf that is the root. row_sel_clust_leaf_hint_armed() stands a scan down once the slots stop paying for themselves: a miss adds CLUST_LEAF_HINT_MISS_WEIGHT (2) to the miss counter and a hit takes one away, so a scan gives the slots up where it answers too little of its lookups to pay for them, CLUST_LEAF_HINT_MAX_MISSES (8) misses with no hit between them still reach the threshold, and one lookup in CLUST_LEAF_HINT_RETRY (1024) starts the count again, so a scan whose order becomes correlated only later recovers. Both halves of the cost stop there, the test of the slots and the copies that refresh them. Row_sel_get_clust_rec_for_mysql::operator() calls all of this in place of its btr_pcur_open_with_no_init(), and only where the adaptive hash index is disabled, whose guess solves the same problem better: it lands on the record with no page-local search and no page access to charge. That index is off by default, so the hints are active in a default configuration. btr0cur.h and btr0cur.cc add btr_cur_t::try_leaf_hint(), a PAGE_CUR_LE, BTR_SEARCH_LEAF search on one named leaf. It acquires the page with buf_page_try_get(): a hint is never derived from a latched parent page, so by the time it is tried it can precede the caller's already-latched secondary-index leaf in the latching order, where a blocking wait can deadlock. It then rejects the page unless the checks that it makes on the latched frame put the match on it. Those checks are the sole authority on the result, so a stale range costs a wasted probe or a needless descent, never a wrong result, and the ranges need no invalidation protocol. ha_innodb.cc: ha_innobase::reset() zeroes the used-slot count and the miss counter per statement, matching autoinc_last_value. row0mysql.cc: row_prebuilt_free() frees the key buffers that the slots own. innodb.clust_leaf_hint measures pages_accessed over key orders that differ in how closely the secondary order tracks the clustered one, and eight further tables check query results over the record formats and key shapes that a clustered-index lookup has to read, down to the metadata pseudo-record of instant ALTER TABLE, to leaves that split and merge while a locking read walks them, and to a record that a remembered leaf supplies for a scan that must then rebuild an older version of it. Two of the tables scan a covering index, which reads a clustered record under an exclusive select lock type and under a PAGE_MAX_TRX_ID that the read view cannot see. clust_leaf_hint_off_debug runs the same body with the hints turned off, through a debug switch that returns before a lookup tests or refreshes the slots, so a diff of the two .result files is what the hints save: 2006 to 1031 (2-level clustered index), 2291 to 1011 (3-level), 4006 to 2015 (two interleaved key ranges), 12016 to 9078 (locality in the second half alone) and 20020 to 10045 for a covering scan that FOR UPDATE makes non-covering, where the same scan without FOR UPDATE costs 20 in both files. Two orders with too little locality to pay for the slots give them up early and end within a hundred accesses of the unhinted count: 20020 to 19966 (decorrelated) and 20020 to 19999 (shuffled). innodb.clust_leaf_hint_instant_alter covers the one rejection that no count reaches, of a slot whose keys were copied under another dict_index_t::n_core_fields than the index reports. dict_index_t::clear_instant_alter() is the only writer of that value that a shared metadata lock allows, and it needs the clustered index to lose the last user record of its root page, while no leaf that is the root fills a slot, so the tree has to shrink between the two, which purge does there. The reader therefore reads uncommitted rows at READ UNCOMMITTED and waits in a stored function while a rollback and purge take them away, and one row that arrives above the position it stopped at is the lookup that tests the slots. The rejection leaves nothing that a query can read, so that branch writes the two counts to the error log under a debug switch and the case reads them back with search_pattern_in_file.inc. They are printed and not named in the pattern, so that a run which reaches the branch with other counts, or in the direction where the clear lowers them, is a difference to look at and not a pass. main.rowid_filter_innodb: 90 to 88, and its ahi combination unchanged. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| fixup! 69dad73b01a4706718bf7e078f3c861a341110fe | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| InnoDB: extract pscan_chunk_clamp_t as a separate struct from row_prebuilt_t | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
PQ: report per worker what the scan cost it, not just how much it read ANALYZE FORMAT=JSON says how many rows each worker read, which answers how the chunks divided the work but not why a worker finished when it did. Two figures the engine already keeps answer that, once they are kept per worker rather than summed: how long each worker was busy inside the engine, and how much of that was waiting for pages to come off disk. "r_rows_per_worker": [181967, 93080, 159138, 165815], "r_engine_time_per_worker_ms": [69.577, 74.895, 72.507, 69.902], "r_pages_read_time_per_worker_ms": [0.276, 0.229, 0.285, 0.283], "r_chunks": 18, "r_chunks_resplit": 1, The pair is what makes a straggler readable. Rows without time is a worker that was handed more to do; time without rows is one that was blocked. Above, four workers within seven per cent of each other on time despite a two-to-one spread in rows says the spread is the pull queue's doing, not the disk's. The read times sum to the pages_read_time_ms already reported in r_engine_stats, so the array is a decomposition of that total rather than a second measurement of it. Reading is the wait worth attributing this way: a parallel scan takes no row locks, because the gate declines the scan when the read is a locking one, so there is no lock wait to divide. The figures are undercounts wherever the pre-fetcher did its job -- a page somebody else fetched costs its reader nothing here -- so a low read time is not proof that nobody waited. Two things had to be fixed before the numbers existed rather than merely being hidden. The page read time was being collected per worker all along and thrown away at the merge: each worker resets its handler stats when it opens its tables and snapshots them into tab_hstats before closing them, and quiesce_workers() added those into the manager's handler without keeping the split. It now keeps it. Engine time was always zero for a worker, for two reasons. It is folded into the handler stats from the handler's Exec_time_tracker by close_thread_table(), which a worker's tables never go through, so snapshot_table_stats() now does that fold itself. Under that, the worker's handlers had no tracker at all to fold: set_time_tracker() is called by the optimizer on the plan's tables, and a worker's copies are opened after the plan exists, so nothing was timing them. Each worker now owns one Exec_time_tracker per table, installed as the table is opened. Both arrays are omitted when the engine recorded no time, so a scan that never touched the disk does not carry an array of zeroes; this is why the read-time array appears in some plans and not others. Timings and a per-worker split are both non-deterministic, so analyze-format.inc masks them, and the three .result files that show a parallel scan are re-recorded. This commit was prepared with Claude Code, which found that the read time was already being collected and discarded, and traced the always-zero engine time to the missing tracker on the worker's handlers. Added, r_peak_to_average_ratio for worker engine time. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Khaled Riyad
khaled57.dev@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40551 Copy/Paste friendly output format for MariaDB Command Line Client Copy/paste friendly output was only reachable by starting the client with --silent --skip-column-names, which cannot be done from a running interactive session. Add \S, a statement terminator which prints the result of one statement in the tab separated format without column names. com_silent() sets output_plain, opt_silent and column_names around com_go(), then restores them, the same way com_ego() handles vertical. output_plain selects print_tab_data() ahead of the vertical and table branches, so \S gives the same output whether the session was started plainly or with --table, --vertical or --silent. --html and --xml still win, matching \G. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| fixup! 69dad73b01a4706718bf7e078f3c861a341110fe | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Make an over-long key a prefix key instead of no key A key image that does not fit in a fulltext token was rejected, which costs the index for that value entirely: no key in the document, and an OVERLAPS that mentions the value gives up on the index altogether. Cut the image down to MVI_KEY_IMAGE_MAX_LEN instead. Two values that agree on that many bytes then share a key, which costs false positives and nothing else, since the predicate is rechecked on every row the index produces. This is what the non-binary path has always done -- strnxfrm() is asked for exactly that many bytes of weights and cannot return more -- so it makes the binary path, which is the one a JSON column takes, behave the same. No wildcard is needed in the fulltext query for this. Both sides of the index cut at the same point, so the key a search builds for a long value is the same string as the token the document has for it, and an exact term match finds it. A trailing '*' would only widen the term to keys that are longer than the one searched for, and after the cut there are none. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Make an over-long key a prefix key instead of no key A key image that does not fit in a fulltext token was rejected, which costs the index for that value entirely: no key in the document, and an OVERLAPS that mentions the value gives up on the index altogether. Cut the image down to MVI_KEY_IMAGE_MAX_LEN instead. Two values that agree on that many bytes then share a key, which costs false positives and nothing else, since the predicate is rechecked on every row the index produces. This is what the non-binary path has always done -- strnxfrm() is asked for exactly that many bytes of weights and cannot return more -- so it makes the binary path, which is the one a JSON column takes, behave the same. No wildcard is needed in the fulltext query for this. Both sides of the index cut at the same point, so the key a search builds for a long value is the same string as the token the document has for it, and an exact term match finds it. A trailing '*' would only widen the term to keys that are longer than the one searched for, and after the cut there are none. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Factor out the MVI array walk Mvi_array_iterator::start() and next() return the next thing the walk found -- a key, an element with no key, a nested array opened or closed, or one of the ways the walk ends -- and the caller loops over them with its own control flow: MVI_ENCODE goes back to its gotos, collect_mvi_keys() to plain returns, and the state it kept in a visitor is local variables again. The buffer to encode into is a constructor argument, so MVI_ENCODE still gets its keys written straight into the document it is building. No functional changes. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Petrunia
sergey@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
In Parallel_coordinator, remove partition_id() and m_partition_id. MariaDB's innodb doesn't have those. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41012 Galera appliers hang with foreign key of types UUID, INET4, INET6 Added second test for testing key collisions from transactions modifying separate rows |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
squash! b387a4a6f9f9b3194a29c1a80c39c983d5dc4fd5 handlerton::backup_file: Check if a file should be included in the backup. Implemented for ENGINE=Aria in maria_backup_file(). aria_backup_start(): Copy the Aria log files (FIXME: currently, single-threaded) backup::copy_or_stream(): Copy or stream a file. backup_context: Process-wide BACKUP SERVER context. Handles the directory traversal and copying of files for built-in storage engines that do not implement this backup interface. backup_target_phase. Wrap backup_context. backup_target_phase::step(), backup_context::step(): Process a file from a directory scan, or by invoking handlerton::backup_step(). |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Factor out the MVI array walk Mvi_array_iterator::start() and next() return the next thing the walk found -- a key, an element with no key, a nested array opened or closed, or one of the ways the walk ends -- and the caller loops over them with its own control flow: MVI_ENCODE goes back to its gotos, collect_mvi_keys() to plain returns, and the state it kept in a visitor is local variables again. The buffer to encode into is a constructor argument, so MVI_ENCODE still gets its keys written straight into the document it is building. No functional changes. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Petrunia
sergey@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Remove Parallel_coordinator::Exec_ctx::m_id, it is not used anywhere. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| new columnstore (df4f261f1f622adb2c266527e3fd3c8c4ddaa636 25.10.7) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Don't use an MVI for an depth-2 OVERLAPS array with no keys An element that is itself an array is flattened, the same way MVI_ENCODE flattens the document. JSON_OVERLAPS does not flatten: it only matches such an element against a document element that is an array too, compared whole (json_compare_arrays_in_order()). The flattening here is still safe as it will produce only false positives that will be eliminated by a recheck. The only exception is when the nested array yields no key at all i.e. [], [[]], [[],[]], [[[]]], etc. Such an element may match a document element that has no key of ours either, so nothing we could search for would find that row. Give up in this case, as for a failed encoding. select json_overlaps('[[]]', '[[], "aaa"]'); is true, but the document has no key in the index and the scan for the "aaa" key does not return it. Give up on the access in that case, as we already do for an element that cannot be encoded. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Dave Gosselin
dave.gosselin@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-36166: Accept bracketed points inside MULTIPOINT ST_GEOMFROMTEXT('MULTIPOINT((0 0),(1 1))') returned NULL while ST_GEOMFROMTEXT('MULTIPOINT(0 0,1 1)') returned the geometry. The bracketed spelling is the one the OGC WKT grammar defines. In 06-103r4 section 7.2.2 a <multipoint text> is a list of <point text>, and a <point text> has its own parentheses, the same way a <multilinestring text> is a list of <linestring text>. The bare spelling matches no production in that grammar, so the text MariaDB rejected was the conformant one. The first point now determines which of the two bracketing forms the remaining list elements will use. A mixed list such as MULTIPOINT((0 0),1 1) is an error. The bare form stays accepted because existing data and applications use it. Geometry::create_from_wkt is the single entry into the WKT reader, so ST_MPOINTFROMTEXT and a MULTIPOINT nested in a GEOMETRYCOLLECTION are covered by the same change. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Dave Gosselin
dave.gosselin@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-36166: support for notation with brackets inside MULTIPOINT ST_GEOMFROMTEXT('MULTIPOINT((0 0),(1 1))') returned NULL while ST_GEOMFROMTEXT('MULTIPOINT(0 0,1 1)') returned the geometry. The bracketed spelling is the one the OGC WKT grammar defines. In 06-103r4 section 7.2.2 a <multipoint text> is a list of <point text>, and a <point text> has its own parentheses, the same way a <multilinestring text> is a list of <linestring text>. The bare spelling matches no production in that grammar, so the text MariaDB rejected was the conformant one. The first point now fixes which of the two bracketing forms the whole list uses, requiring subsequent points to use the same bracketing. A mixed list such as MULTIPOINT((0 0),1 1) is an error. The bare form stays accepted because existing data and applications use it. Geometry::create_from_wkt is the single entry into the WKT reader, so ST_MPOINTFROMTEXT and a MULTIPOINT nested in a GEOMETRYCOLLECTION are covered by the same change. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Merge same-mvi accesses and choose the best mvi access for a table based on costs Deduplicate while we are at it. Previously we simply chose the last access. Now we merge conjunctives and keep all mvi accesses on the same table, and choose the best one at costing. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
ParadoxV5
paradox.ver5@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40996 Support `--sync_with_master 0, $variable` in mysqltest `--sync_with_master` uses `get_string()`, which has `$variable` support, but it only uses the read buffer, which is written with the unexpanded string and not the variable value. Reviewed-by: KhaledR57 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Petrunia
sergey@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| More comments, code readability. No functional changes. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Make an over-long key a prefix key instead of no key A key image that does not fit in a fulltext token was rejected, which costs the index for that value entirely: no key in the document, and an OVERLAPS that mentions the value gives up on the index altogether. Cut the image down to MVI_KEY_IMAGE_MAX_LEN instead. Two values that agree on that many bytes then share a key, which costs false positives and nothing else, since the predicate is rechecked on every row the index produces. This is what the non-binary path has always done -- strnxfrm() is asked for exactly that many bytes of weights and cannot return more -- so it makes the binary path, which is the one a JSON column takes, behave the same. No wildcard is needed in the fulltext query for this. Both sides of the index cut at the same point, so the key a search builds for a long value is the same string as the token the document has for it, and an exact term match finds it. A trailing '*' would only widen the term to keys that are longer than the one searched for, and after the cut there are none. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Factor out the MVI array walk Mvi_array_iterator::start() and next() return the next thing the walk found -- a key, an element with no key, a nested array opened or closed, or one of the ways the walk ends -- and the caller loops over them with its own control flow: MVI_ENCODE goes back to its gotos, collect_mvi_keys() to plain returns, and the state it kept in a visitor is local variables again. The buffer to encode into is a constructor argument, so MVI_ENCODE still gets its keys written straight into the document it is building. No functional changes. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41012 Galera appliers hang with foreign key of types UUID, INET4, INET6 wsrep_store_key_val_for_row() built the certification key of a row by collating the column value whenever the field reports MYSQL_TYPE_STRING or MYSQL_TYPE_VAR_STRING, taking the collation from Field::charset(). The data types implemented on Field_fbt - UUID, INET6 and INET4 report MYSQL_TYPE_STRING, and their charset() is my_charset_numeric, which is latin1. Their values are however plain binary and accordingly get_innobase_type_from_mysql_type() maps them to DATA_FIXBINARY. Their keys were therefore run through latin1_swedish_ci, which folds them. That corrupts the key in two ways: 1. A key mismatch for one and the same row. The reference key that wsrep_rec_get_foreign_key() appends for the parent of a child INSERT is built from the InnoDB record and is not collated, so it no longer matched the primary key carried by the parent row's own writeset. Certification saw no dependency between a child INSERT and a concurrent parent UPDATE, and two appliers could apply them in parallel causing a hang or crash. 2. A key collision between distinct rows. The folding is many to one, so different values collapse onto one key, Certification compares keys byte for byte, so unrelated rows were treated as the same row. Concurrent transactions on them certified as a conflict and one was aborted with ER_LOCK_DEADLOCK. Fix is for wsrep_store_key_val_for_row() to skip the collation for fields that InnoDB stores as binary, using the same condition as get_innobase_type_from_mysql_type(). This is a no-op for the types that worked before. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Factor out the MVI array walk Mvi_array_iterator::start() and next() return the next thing the walk found -- a key, an element with no key, a nested array opened or closed, or one of the ways the walk ends -- and the caller loops over them with its own control flow: MVI_ENCODE goes back to its gotos, collect_mvi_keys() to plain returns, and the state it kept in a visitor is local variables again. The buffer to encode into is a constructor argument, so MVI_ENCODE still gets its keys written straight into the document it is building. No functional changes. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Resolve two TODOs 1. Merge conjunctives on the same index. Deduplicate while we are at it 2. Update costing Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40906: rpl.rpl_gtid_thread_id assert_grep.inc failed rpl.rpl_gtid_thread_id could fail sporadically due to a non-deterministic slave state during an assert. The test asserted that a certain number of transaction's exist in the slave's binary log file; however, there was no sync between the master and slave after the last transaction executed on the master. This means the slave's binary log could be checked before the transaction ever was sent to/committed on the slave. The fix is to simply sync the master and slave before checking the slave's binary log. Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 Don't use an MVI for an depth-2 OVERLAPS array with no keys An element that is itself an array is flattened, the same way MVI_ENCODE flattens the document. JSON_OVERLAPS does not flatten: it only matches such an element against a document element that is an array too, compared whole (json_compare_arrays_in_order()). The flattening here is still safe as it will produce only false positives that will be eliminated by a recheck. The only exception is when the nested array yields no key at all i.e. [], [[]], [[],[]], [[[]]], etc. Such an element may match a document element that has no key of ours either, so nothing we could search for would find that row. Give up in this case, as for a failed encoding. select json_overlaps('[[]]', '[[], "aaa"]'); is true, but the document has no key in the index and the scan for the "aaa" key does not return it. Give up on the access in that case, as we already do for an element that cannot be encoded. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Dave Gosselin
dave.gosselin@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-36166: Accept bracketed points inside MULTIPOINT ST_GEOMFROMTEXT('MULTIPOINT((0 0),(1 1))') returned NULL while ST_GEOMFROMTEXT('MULTIPOINT(0 0,1 1)') returned the geometry. The bracketed spelling is the one the OGC WKT grammar defines. In 06-103r4 section 7.2.2 a <multipoint text> is a list of <point text>, and a <point text> has its own parentheses, the same way a <multilinestring text> is a list of <linestring text>. The bare spelling matches no production in that grammar, so the text MariaDB rejected was the conformant one. The first point now determines which of the two bracketing forms the remaining list elements will use. A mixed list such as MULTIPOINT((0 0),1 1) is an error. The bare form stays accepted because existing data and applications use it. Geometry::create_from_wkt is the single entry into the WKT reader, so ST_MPOINTFROMTEXT and a MULTIPOINT nested in a GEOMETRYCOLLECTION are covered by the same change. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| fixup! a77b37c49cf47284580e507303d538a5daa2ea79 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||