Console View
|
Categories: connectors experimental galera main |
|
| connectors | experimental | galera | main | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40768: Slave cannot apply a fragmented row event written with log_bin_compress=ON MDEV-32570 added fragmentation of large row events. When binlog compression is enabled (log_bin_compress=ON), the compression is bypassed, and these binary log events remain as regular row events, when they should be compressed row events. This is because the compression happens during the Log_event::write() function, whereas row data fragmentation happens before Log_event::write() is called. The Rows_log_event super-class stores the row data buffers to be written to disk. The regular Rows_log_event sub-classes's implementations of Log_event::write() write this data as-is. The compressed sub-classes's implementation of ::write() over-write these buffers with the compressed rows data before writing to disk. To fragment a large row event, the server fragments the Rows_log_event's row data buffers, to be written by multiple Partial_rows_log_event's, and each Partial_rows_log_event::write() call writes its portion of the row data buffer to disk. However, this happens before compression ever has a chance to take place, and thereby, an event that should be compressed, never is. MDEV-39762 added event structure validation for compressed events, and discovered that these events carry the compressed event type, but are not actually compressed. Event validation thereby fails. This patch adds a workaround to override the event type of the fragmented row events to be regular row events, to be consistent with the actual on-disk content. The underlying problem still needs to be addressed though, and is tracked by MDEV-40851. Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Arcadiy Ivanov
arcadiy@ivanov.biz |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40781 Duplicate row despite DISTINCT when tmp table converts `create_internal_tmp_table_from_heap()` writes the pending `record[0]`, the row whose write filled the in-memory table, into the new table. Since **MDEV-40376** (`636f154bb49`) that write happens *before* `ha_end_bulk_insert()` rather than after it. `ha_maria::start_bulk_insert()` disables **all** indexes of an internal temporary table that is about to receive at least `MARIA_MIN_ROWS_TO_DISABLE_INDEXES` (100) rows: ```c if (file->open_flags & HA_OPEN_INTERNAL_TABLE) { /* Internal table; If we get a duplicate something is very wrong */ file->update|= HA_STATE_CHANGED; index_disabled= share->base.keys > 0; maria_clear_all_keys_active(file->s->state.key_map); } ``` `maria_write()` then skips `_ma_check_unique()` entirely, so the unique constraint that implements `DISTINCT` for a key too wide to be an index is not enforced. The rows copied out of the in-memory table are already distinct and need no checking against each other, but the pending row is exactly the row whose duplicate status is unknown, and it was written inside that window. A `SELECT DISTINCT` over wide columns could therefore return a duplicate row. Note that the justification given in `636f154bb49` is not the mechanism at work here. It refers to the bulk insert key *tree*, a different branch of `ha_maria::start_bulk_insert()`; setting `bulk_insert_buffer_size=0` does not avoid the problem. The fix splits the copy in two: 1. `Tmp_table_row_copier` gains a second virtual, `write_pending_row()`, defaulting to a no-op. 2. `copy_rows()` now only copies the rows the in-memory table holds. 3. `create_internal_tmp_table_from_heap()` calls `ha_end_bulk_insert()` and then `write_pending_row()`, so the pending row is written with the indexes of the new table back in place and a duplicate of an already copied row is detected. `Window_rowid_remapper` keeps writing its pending row within `copy_rows()` and inherits the no-op default. Its new position is only known once the rows before it have been written, and nothing is lost by writing it with the indexes still disabled: it replaces a row that is already in the table rather than adding one, and an update of a window function value cannot collide with another row, as a deduplicating key is not built on the columns it changes. The new test covers `SELECT DISTINCT`, `SELECT DISTINCT ... ORDER BY`, `GROUP BY`, `UNION` and `INSERT ... SELECT DISTINCT`, and asserts that the conversion actually happened so that a future sizing change cannot silently void the coverage. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge branch '11.8' into 12.3 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40776 Atomic CREATE OR REPLACE silently breaks the foreign key Give an error if one tries to drop a table referenced by foreign keys if foreign_key_checks=1 Other things: - Changed the error message for ER_TRUNCATE_ILLEGAL_FK to say "Cannot drop or truncate a table ..." |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Petrunia
sergey@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Cleanup: move parallel cost functions from sql_parallel_workers.cc Now they are in sql_parallel_execution.cc, but probably should go to something like sql_parallel_optimization.cc |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39226: Push whole multi-table update/delete down into engines Give storage engines a way to take over an entire multi-table UPDATE/DELETE, the way they can already take over a SELECT. Without it the join, the row matching and every modification run in the SQL layer even when an engine could do the whole statement itself in one step; a single-table UPDATE/DELETE already avoids this via direct_update_rows()/direct_delete_rows(), but a multi-table statement has no primary handler object to drive that path. This adds a generic, engine-agnostic pushdown interface: the SQL layer offers the statement to the engine, and if the engine accepts it, it performs the whole thing and reports only the row counts. - Split select_handler into a pushdown_handler base with select_handler (result set) and a new multi_upddel_handler (runs a whole UPDATE/DELETE, reports row counts, reported as PUSHED UPDATE/PUSHED DELETE); add handlerton::create_multi_upddel, looked up in Sql_cmd_dml::execute_inner(). - multi_update/multi_delete gain direct_update_delete_done(), which records the engine's counts so send_eof() binlogs and replies without the SQL-layer loop; it forces statement-format binlogging so the change still replicates under binlog_format=ROW, and errors out instead of silently dropping counts for an unsupported result object. - FederatedX implements the interface as the reference engine used to test correctness: it prints the statement back and runs it remotely, passes the engine's error code/SQLSTATE through, reads the matched count from the remote info string, executes IGNORE locally, and only pushes down when all tables share one remote server (same as SELECT/derived/unit pushdown). Test: federated.federatedx_pushdown_upd_del. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Removed some not needed checks and add a DBUG_ASSERT() for not covered code - In ha_partition.cc:check_parallel_search(), remove check if item_field->field is null. This is not needed as the function is run after fix_field() which guarnatees that the field is always set. - Added DBUG_ASSERT(new_field) to Item_field::fix_fields() to check if a select-list item, found by name or alias when resolving ORDER BY/GROUP BY/HAVING, can have field == 0. This error path is not covered by any mtr test. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Fixed internal temporary buffer sizes to use tmp_memory_table_size tmp_memory_table_size is limiting the size of internal temporary memory tables. max_heap_table_size is there to limiting the size of explictely created memory tables. max_heap_table_size can be much larger than tmp_memory_table_size as the memory used by temporary tables is in the control of the user. This commit changes the usage of max_heap_table_size for internal buffers to min(max_heap_table_size, tmp_memory_table_size), like we do for internal temporary tables. This changes the in memory buffer allocations for: - GROUP_CONCAT() - Calculating the cost for scanning memory tables (the original code was wrong here as it used the wrong size for memory tables). - ANALYZE TABLE buffer sizes for calculating distinct column values Other things: - Add THD::ram_limitation() to provide consistent memory limitations in all code that used variables.tmp_memory_table_size as buffers. If tmp_memory_table_size == 0, then 8192 is used. This replaces Item_sum::ram_limitation which used 1024 as min buffer, which is way to little for any practical case. - Added security guard in heap_prepare_hp_create_info to ensure that max_table_size is calculated same way as in MariaDB server. - Fixed initial memory allocations for Item_func_group::concat which allocated 'max allowed memory' at start. Now it allocates only 1/16 of that memory at start. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-22992 Refactor VIO into layered transports and filters Replace the function-pointer VIO implementation with an abstract C++ interface while retaining the procedural C entry points. Implement socket and named-pipe transports and composable filters for client read-ahead, Windows thread-pool prefetch, and TLS. OpenSSL uses a custom BIO, while wolfSSL uses callbacks that perform I/O through the VIO below the TLS filter. This keeps waits and timeouts in the transport layer. Keep sockets nonblocking and implement timed I/O with transport waits. Named pipes use overlapped I/O for timeout-aware waits and report blocking waits through the same scheduler callbacks as sockets. Semi-sync temporarily changes the real VIO read timeout instead of copying VIO state. Hide transport and TLS implementation state behind accessors. Expose the TLS handle opaquely and update callers that previously accessed VIO fields directly. Compile the VIO implementations as C++ and retain PSI memory accounting for VIO allocations. Adapt Windows thread-pool pre-read to a Prefetched_vio filter inserted above the transport so both plain and TLS connections consume prefetched bytes through the same layered VIO path. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40768: Slave cannot apply a fragmented row event written with log_bin_compress=ON MDEV-32570 added fragmentation of large row events. When binlog compression is enabled (log_bin_compress=ON), the compression is bypassed, and these binary log events remain as regular row events, when they should be compressed row events. This is because the compression happens during the Log_event::write() function, whereas row data fragmentation happens before Log_event::write() is called. The Rows_log_event super-class stores the row data buffers to be written to disk. The regular Rows_log_event sub-classes's implementations of Log_event::write() write this data as-is. The compressed sub-classes's implementation of ::write() over-write these buffers with the compressed rows data before writing to disk. To fragment a large row event, the server fragments the Rows_log_event's row data buffers, to be written by multiple Partial_rows_log_event's, and each Partial_rows_log_event::write() call writes its portion of the row data buffer to disk. However, this happens before compression ever has a chance to take place, and thereby, an event that should be compressed, never is. MDEV-39762 added event structure validation for compressed events, and discovered that these events carry the compressed event type, but are not actually compressed. Event validation thereby fails. This patch adds a workaround to override the event type of the fragmented row events to be regular row events, to be consistent with the actual on-disk content. The underlying problem still needs to be addressed though, and is tracked by MDEV-40851. Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko MƤkelƤ
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
squash! 43946bab13fe78cdc3f55f0b2f2af887dfceddb3 log_t::backup_start(): If we were running with innodb_log_archive=ON, ensure that the latest file is a valid recovery starting point. That is, wait for the latest log checkpoint to be within the file. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge branch 'br-11.8-merge' into bb-12.3-release-bad | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Added proper cleanup of main.cte_update_delete.test | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40805 Do not call lock_rec_convert_impl_to_expl if a table S-lock is held lock_clust_rec_read_check_and_lock() skipped the implicit-to-explicit conversion only under a table LOCK_X. When a table LOCK_S is held the conversion is equally pointless: no other transaction can hold an implicit X-lock on the record, because modifying a row requires a table LOCK_IX and LOCK_IX is incompatible with our LOCK_S. lock_table_has() matches stronger modes, so testing LOCK_S subsumes the old LOCK_X test. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38801 Item_sum & Item_cache implement deep_copy() {Item_cache,Item_cache_row,Item_sum}::deep_copy() currently call shallow_copy_with_checks(). This causes issues when a proper independent copy is required, e.g. in add_key_part called on a key with a value containing an item inherited from Item_cache. We implement a proper deep copy that shares no nodes with the source, and add a check to ensure this. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40012 Parallel Query: execute the join in the worker threads Each parallel worker now runs the join and the WHERE over its own chunk of the driving table and ships the base-table columns of every row that qualified. The manager copies those columns back into the fields they came from in its own table instances, so that after a row is drained its records hold what a serial scan would have left there, and then evaluates the select list against them and sends the row. This replaces the model where the workers shipped raw source records and the manager ran the join. The transport carries columns rather than projected values because anything the manager does with a row beyond sending it reads a record, and not all of those reads go through Items that could be re-pointed: create_tmp_table() builds Copy_field pairs holding raw Field pointers into the base tables. make_join_readinfo()'s gate, can_run_query_in_workers(), chooses the worker-side path for an inner select-project[-join] over a parallel-scannable driving table: no tmp table, no LIMIT/SQL_CALC_FOUND_ROWS/procedure/aggregate, no outer join, no semijoin strategy the worker's plain nested loop cannot honour, and every non-driving table reached by eq_ref, ref or full scan. Every expression it hands a worker has to deep-copy into something that shares no node with the original and reads no table outside the join, or the copy would carry the manager's own items onto a worker's tables. do_select() then runs run_worker_side_join(); anything ineligible runs serially. Each worker opens a private copy of every non-const table, deep-clones and field-rebinds the conditions and the shipped column list, rebuilds each ref, scans its chunk, runs its own join and ships the columns it read for each full match. It runs under the session's query id, so an item holding a value for the statement recomputes it rather than handing out the one it was cloned with. sql_parallel_execution.cc holds all of the above; sql_parallel_workers.cc keeps the worker threads, the batch channel and their lifetime. Tests: parallel_query_worker_side, parallel_query_join and parallel_query_clone compare a parallel result set against a serial one; parallel_query and parallel_query_oom moved to a plain SELECT and were re-recorded. .claude/commits/MDEV-40012.md records what came from the 13.0 branch verbatim, what was adapted to this tree, and what is new here. This commit was prepared with Claude Code: it ported the work from the 13.0 branch onto this tree's refactored structures and renamed scan API, found and fixed the two holes this tree's deeper Item copying opens -- an expression reaching a table the worker does not have, and a statement-lifetime item cached against query id 0 -- split the file, and ran the suites. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40486 [fixup] Clamp max_length at MAX_FIELD_VARCHARLENGTH in Item_func_vec_fromtext::fix_length_and_dec And move the length check in Item_func_vec_fromtext::val_str to later This allows create table t1 (v vector(64) not null); insert into t1 select vec_fromtext(concat('[',group_concat(1),']')) from seq_1_to_64; which was banned in the previous fix bb0ac437015dec04fbee226745a8eb2bb4825917, though this also introduces the inconsistency(?) where create table t1 as select vec_fromtext(concat('[',group_concat(1),']')) from seq_1_to_64; still fails ER_TRUNCATED_WRONG_VALUE |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39492 PQ workers must read the manager's transaction snapshot A parallel worker runs in its own THD, so it runs in its own transaction and opened its own read view at its first read. Nothing tied that view to the one the manager holds: the workers and the manager could each read a different version of the table, and the chunk boundaries the engine partitioned under the manager's view did not even belong to the snapshot the workers scanned. Both directions were visible. Under REPEATABLE READ a worker returned rows another session had committed after the manager's snapshot was taken, and rows the manager's own transaction had written but not committed went missing, because to the worker's view the manager was just another active transaction. InnoDB implements it by copying the source transaction's read view. ReadView::clone() installs the source's ReadViewBase state and opens the copy. It inherits the source's m_creator_trx_id rather than keeping the worker's own, so rows written by the sharing transaction itself stay visible through the copy -- that is what changes_visible() uses the creator id for -- and it inherits the source's isolation level, so a READ UNCOMMITTED source is still read without consulting a view at all. The source view is read under its m_mutex, the same protection the purge coordinator takes in ReadView::append_to(), and from_thd's handlerton data under its LOCK_thd_data. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-22992 Refactor VIO into layered transports and filters Replace the function-pointer VIO implementation with an abstract C++ interface while retaining the procedural C entry points. Implement socket and named-pipe transports and composable filters for client read-ahead, Windows thread-pool prefetch, and TLS. OpenSSL uses a custom BIO, while wolfSSL uses callbacks that perform I/O through the VIO below the TLS filter. This keeps waits and timeouts in the transport layer. Keep sockets nonblocking and implement timed I/O with transport waits. Named pipes use overlapped I/O for timeout-aware waits and report blocking waits through the same scheduler callbacks as sockets. Semi-sync temporarily changes the real VIO read timeout instead of copying VIO state. Hide transport and TLS implementation state behind accessors. Expose the TLS handle opaquely and update callers that previously accessed VIO fields directly. Compile the VIO implementations as C++ and retain PSI memory accounting for VIO allocations. Adapt Windows thread-pool pre-read to a Prefetched_vio filter inserted above the transport so both plain and TLS connections consume prefetched bytes through the same layered VIO path. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge branch '12.3' into br-12.3-merge | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-22992 Refactor VIO into layered transports and filters Replace the function-pointer VIO implementation with an abstract C++ interface while retaining the procedural C entry points. Implement socket and named-pipe transports and composable filters for client read-ahead, Windows thread-pool prefetch, and TLS. OpenSSL uses a custom BIO, while wolfSSL uses callbacks that perform I/O through the VIO below the TLS filter. This keeps waits and timeouts in the transport layer. Keep sockets nonblocking, as they have already been for a long time, and remove the switch to block/nonblock machinery. Socket vio is now nonblocking already after construction and this never changes. Hide transport and TLS implementation state behind accessors. Expose the TLS handle opaquely and update callers that previously accessed VIO fields directly. Compile the VIO implementations as C++ and retain PSI memory accounting for VIO allocations. WolfSSL/OpenSSL headers are no longer included into VIO public headers. The files that actually use SSL, SSL_CTX, X509 get the headers via ssl_compat.h Adapt Windows thread-pool AIO pre-read to a Prefetched_vio filter, also for TLS (previously TLS was readiness-only, inefficient use of AIO) |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40768: Slave cannot apply a fragmented row event written with log_bin_compress=ON MDEV-32570 added fragmentation of large row events. When binlog compression is enabled (log_bin_compress=ON), the compression is bypassed, and these binary log events remain as regular row events, when they should be compressed row events. This is because the compression happens during the Log_event::write() function, whereas row data fragmentation happens before Log_event::write() is called. The Rows_log_event super-class stores the row data buffers to be written to disk. The regular Rows_log_event sub-classes's implementations of Log_event::write() write this data as-is. The compressed sub-classes's implementation of ::write() over-write these buffers with the compressed rows data before writing to disk. To fragment a large row event, the server fragments the Rows_log_event's row data buffers, to be written by multiple Partial_rows_log_event's, and each Partial_rows_log_event::write() call writes its portion of the row data buffer to disk. However, this happens before compression ever has a chance to take place, and thereby, an event that should be compressed, never is. MDEV-39762 added event structure validation for compressed events, and discovered that these events carry the compressed event type, but are not actually compressed. Event validation thereby fails. This patch adds a workaround to override the event type of the fragmented row events to be regular row events, to be consistent with the actual on-disk content. The underlying problem still needs to be addressed though, and is tracked by MDEV-40851. Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-40635 fixup | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39492 PQ framework for manager and worker threads Introduces parallel_worker_threads variable to control the number of worker threads created by a parallel execution query. 2 new files, sql_parallel_workers.h sql_parallel_workers.cc which contain structures for the creation, management and deletion of parallel worker threads (pwt_ in the name). Main management class created in the stack in JOIN::exec, implemented for the top level select. Threads are registed in server_threads, so are visible in information_schema.processlist and the show processlist command. We check that a kill query on a parallel worker is passed onto it's manager and the query is properly aborted, and that a kill connection is handled properly in parallel_worker.test. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
ParadoxV5
paradox.ver5@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40636 merge fix reƤpply missed merge fixes Co-authored-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40768: Slave cannot apply a fragmented row event written with log_bin_compress=ON MDEV-32570 added fragmentation of large row events. When binlog compression is enabled (log_bin_compress=ON), the compression is bypassed, and these binary log events remain as regular row events, when they should be compressed row events. This is because the compression happens during the Log_event::write() function, whereas row data fragmentation happens before Log_event::write() is called. The Rows_log_event super-class stores the row data buffers to be written to disk. The regular Rows_log_event sub-classes's implementations of Log_event::write() write this data as-is. The compressed sub-classes's implementation of ::write() over-write these buffers with the compressed rows data before writing to disk. To fragment a large row event, the server fragments the Rows_log_event's row data buffers, to be written by multiple Partial_rows_log_event's, and each Partial_rows_log_event::write() call writes its portion of the row data buffer to disk. However, this happens before compression ever has a chance to take place, and thereby, an event that should be compressed, never is. MDEV-39762 added event structure validation for compressed events, and discovered that these events carry the compressed event type, but are not actually compressed. Event validation thereby fails. This patch adds a workaround to override the event type of the fragmented row events to be regular row events, to be consistent with the actual on-disk content. The underlying problem still needs to be addressed though, and is tracked by MDEV-40851. Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-40751 Make sure that VEC_FROMTEXT results in a length of multiple of 4 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40805 Do not call lock_rec_convert_impl_to_expl if a table S-lock is held lock_clust_rec_read_check_and_lock() skipped the implicit-to-explicit conversion only under a table LOCK_X. When a table LOCK_S is held the conversion is equally pointless: no other transaction can hold an implicit X-lock on the record, because modifying a row requires a table LOCK_IX and LOCK_IX is incompatible with our LOCK_S. lock_table_has() matches stronger modes, so testing LOCK_S subsumes the old LOCK_X test. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko MƤkelƤ
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| fixup! a2020ca6c21691a3727fa2e6a5b8524242e93836 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Limit the memory used by GROUP_CONCAT() with ORDER BY GROUP_CONCAT() with ORDER BY collects all rows of the group in a TREE and only cuts it down in repack_tree(). The repack was triggered by (tree_len >> GCONCAT_REPACK_FACTOR) > thd->gconcat_max_len() with GCONCAT_REPACK_FACTOR 10, that is when the rows in the tree had produced 1024 * group_concat_max_len bytes, or 1G with the default settings. On top of that tree_len only counted the length of the strings, while the tree costs sizeof(TREE_ELEMENT) + reclength per row. For GROUP_CONCAT(int_col ORDER BY int_col) that is about 40 bytes per row against 6 bytes of result, so the tree had grown to several GB before the first repack. In practice the server ran out of memory first and the repack code was close to never used. The tree is now limited by the memory it has really allocated, tree->allocated, instead of by the length of the strings it holds. The limit is MY_MAX(thd->ram_limitation(), thd->gconcat_max_len()) and is never set so low that the tree can not hold a few rows. repack_tree() builds a new tree while the old one is still in memory, so the peak usage is the size we start the repack at plus the size we copy to. To keep the sum within the limit it is split into GCONCAT_TREE_PARTS parts; the repack starts when GCONCAT_TREE_REPACK_PARTS of them are used and copies to the remaining part. The part we do not copy to is also the room the tree has to grow before the next repack, which keeps the repacks amortized. Other changes: - tree_len is removed. It was only read by the old trigger. - repack_tree() decided that it had run out of memory by testing st.len <= st.maxlen after the walk. That test was only valid because the old trigger guaranteed that a complete copy had to overshoot st.maxlen. A repack triggered by memory can complete the walk with st.len far below st.maxlen, which would have failed the query with a wrong out of memory error. There is now an explicit flag for it. - The length that decides which rows to keep now also counts the separator that is put between two rows, so that it matches what val_str() will produce. - When the memory limit stops the copy, the result becomes shorter than group_concat_max_len. dump_leaf_key() can not detect this, as the result never reaches the maximum length. This is now remembered in result_cut and reported to the user. - All cut value reporting is moved to val_str(); dump_leaf_key() only marks that the result was cut. This removes the need to clear the truncated flag of table->blob_storage to avoid a duplicated warning, and gives one warning per group also when val_str() is called more than once for the same group, which repeated the warning before. - Added a function comment for repack_tree() that describes where the rows are cut away and why building a copy frees memory. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Arcadiy Ivanov
arcadiy@ivanov.biz |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40802 COUNT(DISTINCT <blob>) fails when its tmp table converts `COUNT(DISTINCT)` collects the distinct values in a temporary table with a unique constraint over the aggregate's arguments, and treats a duplicate key error from the write as "value already seen": ```c if (!table->file->is_fatal_error(error, HA_CHECK_DUP)) return FALSE; // duplicate, not an error ``` For a blob argument the record holds only a pointer to the value, so `Aggregator_distinct::setup()` cannot use the `Unique` tree, which compares raw record bytes, and every value goes through that write instead. When such a write overflows the in-memory table, `create_internal_tmp_table_from_heap()` copies the stored rows to an on-disk table and then writes the row that overflowed, which until then was held in `record[0]` alone. Whether a duplicate key error on that last write is fatal is decided by the caller's `ignore_last_dupp_key_error` argument, and `Aggregator_distinct::add()` passed **0** three lines below the code that ignores the very same condition. The statement failed with ERROR 1169 (23000): Can't write, because of unique constraint, to table '(temporary)' Pass **1** instead, so that a duplicate arriving through the conversion is discarded exactly like one arriving through the ordinary write. The result is `table->file->stats.records` of that table, so not storing the duplicate is what makes the count right. The argument is the same upstream, where it is unreachable: a temporary table with a blob column was created on the on-disk engine to begin with, so the conversion was never entered for the only tables whose pending row can be a duplicate. Supporting blob columns in the in-memory engine made the table start in memory and convert. New tests `heap.count_distinct_blob_convert` and `heap.count_distinct_blob_convert_debug`. A write rejected as a duplicate returns its record to the free list and never reaches the allocation of the blob value, so only the first copy of a value makes the in-memory table grow, and the write that finds it full is the second copy of the value stored last. That holds only while a record slot is what the table runs out of first. Blob values come out of the same space, and only a write that is not a duplicate ever allocates one, so when a blob allocation is the one that hits the limit, the pending row is not a duplicate at all. Which of the two runs out first follows from how records and blob values pack together, not from any threshold on the value width. Of 24 measured combinations of width and `max_heap_table_size`, 20 convert but only 8 reach a duplicate pending row, so asserting that the table was converted does not establish that the ignored duplicate was reached. The first test uses widths measured to overflow on a record slot. The second removes the dependency on that measurement, injecting the duplicate through a new debug point in `Tmp_table_default_copier::copy_rows()`, beside the one the row copy loop already carries. Every value is present twice, so whichever copy the injected duplicate discards, the other one is still written and the count does not depend on which write overflowed. The status counter is read with the in-memory limit restored. The status table is materialized into a temporary table of its own, and its VARIABLE_VALUE column is wide enough to be stored as a blob, so under the shrunken limit that table can overflow and be converted as well, and would then report its own conversion. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40768: Slave cannot apply a fragmented row event written with log_bin_compress=ON MDEV-32570 added fragmentation of large row events. When binlog compression is enabled (log_bin_compress=ON), the compression is bypassed, and these binary log events remain as regular row events, when they should be compressed row events. This is because the compression happens during the Log_event::write() function, whereas row data fragmentation happens before Log_event::write() is called. The Rows_log_event super-class stores the row data buffers to be written to disk. The regular Rows_log_event sub-classes's implementations of Log_event::write() write this data as-is. The compressed sub-classes's implementation of ::write() over-write these buffers with the compressed rows data before writing to disk. To fragment a large row event, the server fragments the Rows_log_event's row data buffers, to be written by multiple Partial_rows_log_event's, and each Partial_rows_log_event::write() call writes its portion of the row data buffer to disk. However, this happens before compression ever has a chance to take place, and thereby, an event that should be compressed, never is. MDEV-39762 added event structure validation for compressed events, and discovered that these events carry the compressed event type, but are not actually compressed. Event validation thereby fails. This patch adds a workaround to override the event type of the fragmented row events to be regular row events, to be consistent with the actual on-disk content. The underlying problem still needs to be addressed though, and is tracked by MDEV-40851. Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-22992 Refactor VIO into layered transports and filters Replace the function-pointer VIO implementation with an abstract C++ interface while retaining the procedural C entry points. Implement socket and named-pipe transports and composable filters for client read-ahead, Windows thread-pool prefetch, and TLS. OpenSSL uses a custom BIO, while wolfSSL uses callbacks that perform I/O through the VIO below the TLS filter. This keeps waits and timeouts in the transport layer. Keep sockets nonblocking and implement timed I/O with transport waits. Named pipes use overlapped I/O for timeout-aware waits and report blocking waits through the same scheduler callbacks as sockets. Semi-sync temporarily changes the real VIO read timeout instead of copying VIO state. Hide transport and TLS implementation state behind accessors. Expose the TLS handle opaquely and update callers that previously accessed VIO fields directly. Compile the VIO implementations as C++ and retain PSI memory accounting for VIO allocations. Adapt Windows thread-pool pre-read to a Prefetched_vio filter inserted above the transport so both plain and TLS connections consume prefetched bytes through the same layered VIO path. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Limit the memory used by GROUP_CONCAT() with ORDER BY GROUP_CONCAT() with ORDER BY collects all rows of the group in a TREE and only cuts it down in repack_tree(). The repack was triggered by (tree_len >> GCONCAT_REPACK_FACTOR) > thd->gconcat_max_len() with GCONCAT_REPACK_FACTOR 10, that is when the rows in the tree had produced 1024 * group_concat_max_len bytes, or 1G with the default settings. On top of that tree_len only counted the length of the strings, while the tree costs sizeof(TREE_ELEMENT) + reclength per row. For GROUP_CONCAT(int_col ORDER BY int_col) that is about 40 bytes per row against 6 bytes of result, so the tree had grown to several GB before the first repack. In practice the server ran out of memory first and the repack code was close to never used. The tree is now limited by the memory it has really allocated, tree->allocated, instead of by the length of the strings it holds. The limit is MY_MAX(thd->ram_limitation(), thd->gconcat_max_len()) and is never set so low that the tree can not hold a few rows. repack_tree() builds a new tree while the old one is still in memory, so the peak usage is the size we start the repack at plus the size we copy to. To keep the sum within the limit it is split into GCONCAT_TREE_PARTS parts; the repack starts when GCONCAT_TREE_REPACK_PARTS of them are used and copies to the remaining part. The part we do not copy to is also the room the tree has to grow before the next repack, which keeps the repacks amortized. Other changes: - tree_len is removed. It was only read by the old trigger. - repack_tree() decided that it had run out of memory by testing st.len <= st.maxlen after the walk. That test was only valid because the old trigger guaranteed that a complete copy had to overshoot st.maxlen. A repack triggered by memory can complete the walk with st.len far below st.maxlen, which would have failed the query with a wrong out of memory error. There is now an explicit flag for it. - The length that decides which rows to keep now also counts the separator that is put between two rows, so that it matches what val_str() will produce. - When the memory limit stops the copy, the result becomes shorter than group_concat_max_len. dump_leaf_key() can not detect this, as the result never reaches the maximum length. This is now remembered in result_cut and reported to the user. - All cut value reporting is moved to val_str(); dump_leaf_key() only marks that the result was cut. This removes the need to clear the truncated flag of table->blob_storage to avoid a duplicated warning, and gives one warning per group also when val_str() is called more than once for the same group, which repeated the warning before. - Added a function comment for repack_tree() that describes where the rows are cut away and why building a copy frees memory. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40776 Atomic CREATE OR REPLACE silently breaks the foreign key Give an error if one tries to drop a table referenced by a foreign keys This is needed as innodb will keep the reference to the origina table even when it is renamed to a temporary name as part of create or replace. Other things: - Changed the error message for ER_TRUNCATE_ILLEGAL_FK to say "Cannot drop or truncate a table ..." |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
ParadoxV5
paradox.ver5@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40366 merge fix reƤpply missed merge fixes Co-authored-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Brandon Nesterenko
brandon.nesterenko@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40768: Slave cannot apply a fragmented row event written with log_bin_compress=ON MDEV-32570 added fragmentation of large row events. When binlog compression is enabled (log_bin_compress=ON), the compression is bypassed, and these binary log events remain as regular row events, when they should be compressed row events. This is because the compression happens during the Log_event::write() function, whereas row data fragmentation happens before Log_event::write() is called. The Rows_log_event super-class stores the row data buffers to be written to disk. The regular Rows_log_event sub-classes's implementations of Log_event::write() write this data as-is. The compressed sub-classes's implementation of ::write() over-write these buffers with the compressed rows data before writing to disk. To fragment a large row event, the server fragments the Rows_log_event's row data buffers, to be written by multiple Partial_rows_log_event's, and each Partial_rows_log_event::write() call writes its portion of the row data buffer to disk. However, this happens before compression ever has a chance to take place, and thereby, an event that should be compressed, never is. MDEV-39762 added event structure validation for compressed events, and discovered that these events carry the compressed event type, but are not actually compressed. Event validation thereby fails. This patch adds a workaround to override the event type of the fragmented row events to be regular row events, to be consistent with the actual on-disk content. The underlying problem still needs to be addressed though, and is tracked by MDEV-40851. Signed-off-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40805 Do not call lock_rec_convert_impl_to_expl if a table S-lock is held lock_clust_rec_read_check_and_lock() skipped the implicit-to-explicit conversion only under a table LOCK_X. When a table LOCK_S is held the conversion is equally pointless: no other transaction can hold an implicit X-lock on the record, because modifying a row requires a table LOCK_IX and LOCK_IX is incompatible with our LOCK_S. lock_table_has() matches stronger modes, so testing LOCK_S subsumes the old LOCK_X test. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
ParadoxV5
paradox.ver5@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40366 merge fix reƤpply missed merge fixes Co-authored-by: Brandon Nesterenko <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||