Console View
|
Categories: connectors experimental galera main |
|
| connectors | experimental | galera | main | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| workaround for https://bugzilla.redhat.com/show_bug.cgi?id=2390105 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39845 Introduce parallel scan API Add a handler-level interface that engines can implement to support parallel table scans, with a serial-scan fallback when unsupported.. - handler::parallel_scan_support() indicating supported types - Coordinator-side methods (parallel_init_coordinator / parallel_end_coordinator, parallel_get_worker_context) driven by the master thread - Worker-side methods (parallel_init_worker / parallel_get_next_row / parallel_end_worker) driven by child threads, with the ha_parallel_get_next_row() wrapper doing the usual bookkeeping - Parallel_worker_ctx, an opaque per-worker context subclass |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexander Barkov
bar@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39518 Allow prepared statements in stored functions in assignment right hand Allowing prepared statements in stored functions when a stored function is used in an assignment right hand. Both DEFAULT clause of a variable initialization and the right side of the SET statement are supported: CREATE PROCEDURE p1() BEGIN -- case 1: DEFAULT clause DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK -- case 2: SP variable assignment statement DECLARE spvar2 INT; SET spvar2= f1_with_ps(); -- OK END; - Only assignments to SP variables works for now: * SET spvar= func_with_ps(); -- OK * SET @uvar= func_with_ps(); -- Error - Only bare function calls are supported for now. Using a function in an expression does not make it PS-safe yet: SET v= f1()+0; - The parser now does not reject PS statements in stored functions. PS applicability in stored functions is now detected at run time. Note, PS statements in triggers are still prohibited by the parser. - Functions with PS do not acquire MDL locks on tables, and no MDL is taken on the routines themselves either. They work like procedures in terms of table opening and routine locking: a concurrent DROP FUNCTION can complete while such a function is executing. - Functions with PS are not replicated as a single `SELECT f1()` call. They are replicated per-statement, like procedures. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
squash! 1fd7ee821be6dccc21b5869aa382a2bba99850dd Implement multi-threaded non-InnoDB backup. struct Aria_backup: Context for multi-threaded backup. aria_backup_start(): Prepare the context for aria_backup_step(). aria_backup_step(): Copy one file. aria_backup_data(): Copy one data file. We assume that the current working directory is the datadir, which holds for MariaDB Server but not the Embedded Server library. aria_backup_log(): Copy one log file. aria_backup_end(): Finish a copying phase and clean up the context. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40003 Parallel Query: allow parallel scans on ranges in InnoDB clustered index This commit implements partitioning of the InnoDB clustered index for range scans, i.e., when the scan is performed on one or more intervals of the primary key values. It complements the previously introduced functionality of partitioning of the whole clustered InnoDB index. Each interval is handed to the parallel scan coordinator as a separate scan, so each range is chunked the same way as the full index. That means holding an S-latch on the whole index tree and a recursive descent from the root node for each range, so it makes sense to parallelize only when the number of ranges is relatively small, and the ranges are large enough. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38869 sequence conflicts with streaming replication Sequence access conflicts with streaming replication could cause the server to hang, as shown in MDEV-38869. A sequence table is written from SEQUENCE::next_value() while SEQUENCE::mutex is held. For a streaming transaction the row write in handler::ha_write_row() would then replicate a fragment and block waiting for certification and commit order, while an applier may be waiting for the same mutex in SEQUENCE::set_value(). Neither side can proceed, the node deadlocks and the BF abort of the local transaction can never be delivered. This commit avoids the deadlock by skipping the streaming replication step for sequence table rows. The row is already in the write set and is replicated with the following fragment, or at commit. Only that one step is skipped. The skip is passed down as a parameter to wsrep_after_row() and wsrep_after_row_internal() rather than by not calling them at all, so the row is still counted against wsrep_max_ws_rows and wsrep_check_pk() still runs. A transaction using sequences heavily therefore cannot silently exceed the configured write set row limit. The commit has also a new mtr test for three sequence/SR conflict scenarios: galera.galera_sequences_bf_kill_sr - a streaming transaction and an applier competing for SEQUENCE::mutex, where both are expected to commit - the same, but with the applier also BF aborting the local transaction over a gap lock. A streaming transaction cannot be replayed, so it is rolled back and the client gets ER_LOCK_DEADLOCK. The applier is held at the abort_trx_end sync point until the abort has been issued, so that the local transaction cannot finish its fragment first - twelve row inserts on both nodes with wsrep_trx_fragment_unit=rows, so that each node reserves several sequence cache ranges and the sequence table writes land inside fragments carrying several rows. The values the two nodes hand out must not overlap |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Kristian Nielsen
knielsen@knielsen-hq.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
DRAFT: MDEV-36025: Server start fails on mariabackup with prepared transactions mariabackup --no-lock can leave InnoDB prepared transactions in the restored server; this can also happen even without --no-lock when the server is running parallel replication. But mariabackup does not back up the TC_LOG state (typically binlog, or TC_LOG_MMAP if binlog is disabled). This caused the server to complain during startup that the TC_LOG state has been lost, and requires to temporarily configure the server with --tc-heuristic-recover=rollback to roll back the offending transactions. This patch implements a new value --tc-heuristic-recover=auto which makes the server automatically handle rolling back prepared transactions as needed when the TC_LOG state is missing, without the user to change any configurations. This draft patch makes the new value AUTO the default. It is to be determined whether to do this in existing GA releases (eg. 10.11), or whether to make AUTO the default only in some new 13.x release and have the user explicitly configure --tc-heuristic-recover=auto in existing GA releases if they need it. The old behaviour of --tc-heuristic-recover does not seem very useful. If the TC_LOG state is intentionally removed by the user, the correct behaviour seems obviously to be to roll back any prepared transactions. The purpose of having those internal prepared transactions in the first place is to ensure consistency between the engine and the binlog, this has no meaning if the binlog is gone. Or to ensure consistency between engines in multi-engine transactions, and rolling back will ensure this consistency (while commit may not). Note that the prepare step for these transactions is completely internal to the server, and there is no visible semantic different for the user between the backup snapshot occurring just before or just after a transaction ends up as "prepared". Signed-off-by: Kristian Nielsen <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
drrtuy
drrtuy@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| feat: MDEV-40672 implement basic support for the pluggable aggregate functions | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Georgi (Joro) Kodinov
joro@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| days work: incorporating services | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 wip works: SET SESSION debug = '+d,test_invisible_index,test_completely_invisible'; create table t1 (c int, j json, key idx ((CAST(j->'$.tags' AS CHAR(6) ARRAY))))engine=innodb; SHOW CREATE TABLE t1; Table Create Table t1 CREATE TABLE `t1` ( `c` int(11) DEFAULT NULL, `j` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL CHECK (json_valid(`j`)) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci show index from t1; Table Non_unique Key_name Seq_in_index Column_name Collation Cardinality Sub_part Packed Null Index_type Comment Index_comment Ignored t1 1 invisible1 1 invisible1 A 0 NULL NULL YES BTREE NO t1 1 idx 1 DB_MVI_1 NULL NULL NULL NULL YES FULLTEXT NO set @old_innodb_ft_aux_table=@@global.innodb_ft_aux_table; set global innodb_ft_aux_table='test/t1'; insert into t1 values (1, '{"tags": ["1", "abcde", "34567"]}'); SELECT * FROM INFORMATION_SCHEMA.INNODB_FT_INDEX_CACHE; WORD FIRST_DOC_ID LAST_DOC_ID DOC_COUNT DOC_ID POSITION 34567 1 1 1 1 2 abcde 1 1 1 1 1 DROP TABLE t1; set global innodb_ft_aux_table=@old_innodb_ft_aux_table; select mvi_encode('[1, 42, 3]', int); mvi_encode('[1, 42, 3]', int) 8000000000000001 800000000000002a 8000000000000003 select mvi_encode('[1, 42, 3]', unsigned); mvi_encode('[1, 42, 3]', unsigned) 0000000000000001 000000000000002a 0000000000000003 select mvi_encode('[1, 42, "3 "]', char(6)); mvi_encode('[1, 42, "3 "]', char(6)) 31 3432 33 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39491 Add test harness This commit implements pseudo-parallel execution of SELECTs which allows to test the correctness of parallel algorithms. Eligible InnoDB tables that were planned to be either full-scanned or scanned on a set of ranges of the clustered index, are switched to the pseudo-parallel execution. That means the primary index is split into chunks, and those chunks are processed one after another by a single thread. This thread mimics the parallel execution by calling the parallel handler API and acting as both the coordinator and the worker. This mode is activated automatically, there is no need to set any variables before that. If an InnoDB table is set to be either full-scanned or scanned on a set of ranges of the primary index, the pseudo-parallel mode is employed. This harness allows to run MTR tests to catch possible bugs in the parallel logic implementation. NOTE: `innodb_ext_key` test failure is expected: the difference is only in the handler statistics, and it is caused by slightly different algorithms in the serial and parallel paths |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38243 Write binlog row events for changes done by cascading FK operations This commit implements a feature which changes the handling of cascading foreign key operations to write the changes of cascading operations into binlog. The applying of such transaction, in the slave node, will apply just the binlog events, and does not execute the actual foreign key cascade operation. This will simplify the slave side replication applying and make it more predictable in terms of potential interference with other parallel applying happning in the node. This feature can be turned ON/OFF by new variable: rpl_use_binlog_events_for_fk_cascade, with default value OFF The actual implementation is largely by windsurf. The commit has also mtr tests for testing rpl_use_binlog_events_for_fk_cascade feature: rpl.rpl_fk_cascade_binlog_row, rpl.rpl_fk_set_null_binlog_row and rpl.fk_cascade_binlog_row_rollback |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Petrunia
sergey@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| For &mgr->COND_data_avail, use mysql_cond_signal, not mysql_cond_broadcast. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| fix maturity | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39491 Parallel Query: InnoDB clustered-index partitioning for parallel scan Implement the InnoDB side of the handler parallel-scan API: partition the clustered index into disjoint key-range chunks and serve them to the SQL layer as pull-based scan jobs. Each chunk is read through the normal row_search_mvcc() path, so MVCC visibility, AHI and the prefetch cache keep working unchanged. InnoDB spawns no threads of its own. This commit introduces only the whole index partitioning for a subsequent full table scan in parallel. Ranges of the index are not supported and will be implemented later. - Parallel_coordinator (row0pcoord.{h,cc}): adapted from MySQL's Parallel_reader, reduced to partitioning and job distribution. Walks the index top-down into Exec_ctx chunks bounded by clustered-key tuples. - ha_innobase: implements parallel_{init,end}_coordinator, parallel_get_worker_context, parallel_{init,end}_worker and parallel_get_next_row. - Chunk-boundary clamp: row_prebuilt_t::m_pscan_end_tuple (NULL = unbounded) makes row_search_mvcc() stop before prefetching past the chunk's exclusive upper bound, treated as end-of-range so the next chunk is pulled. - btr_pcur_open_on_user_rec(page_cur_t) overload to anchor chunk boundaries. - Build: row0pcoord.cc added to CMakeLists.txt and auto_event_names[]. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40005 Parallel Query: allow parallel scans on secondary indexes in InnoDB This commit extends parallel scanning to InnoDB secondary indexes, both whole-index scans and scans of ranges of the key. It complements the previously introduced partitioning of the clustered index. Both unique and non-unique secondary indexes are supported, as well as covering and non-covering scans. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40003 Parallel Query: allow parallel scans on ranges in InnoDB clustered index This commit implements partitioning of the InnoDB clustered index for range scans, i.e., when the scan is performed on one or more intervals of the primary key values. It complements the previously introduced functionality of partitioning of the whole clustered InnoDB index. Each interval is handed to the parallel scan coordinator as a separate scan, so each range is chunked the same way as the full index. That means holding an S-latch on the whole index tree and a recursive descent from the root node for each range, so it makes sense to parallelize only when the number of ranges is relatively small, and the ranges are large enough. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40005 Parallel Query: allow parallel scans on secondary indexes in InnoDB This commit extends parallel scanning to InnoDB secondary indexes, both whole-index scans and scans of ranges of the key. It complements the previously introduced partitioning of the clustered index. Both unique and non-unique secondary indexes are supported, as well as covering and non-covering scans. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| fixed maturity | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40003 Parallel Query: allow parallel scans on ranges in InnoDB clustered index This commit implements partitioning of the InnoDB clustered index for range scans, i.e., when the scan is performed on one or more intervals of the primary key values. It complements the previously introduced functionality of partitioning of the whole clustered InnoDB index. Each interval is handed to the parallel scan coordinator as a separate scan, so each range is chunked the same way as the full index. That means holding an S-latch on the whole index tree and a recursive descent from the root node for each range, so it makes sense to parallelize only when the number of ranges is relatively small, and the ranges are large enough. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38918 Make large pages an explicit per-caller opt-in my_large_malloc() attempted large pages whenever --large-pages was enabled, silently rounding the size up and reporting it back via an in/out parameter. ut_malloc_dontdump() never passed that adjusted size on to its own callers (the InnoDB redo log buffer and recv_sys_t::tmp_buf), so freeing later used the original, smaller size, causing the reported "faux memory leak". Only the buffer pool and the MyISAM/Aria key caches are documented to benefit from large pages. Everything else that ended up calling my_large_malloc() only wanted its "do not dump to core" property and picked up large pages as an undocumented side effect; those buffers are also small and sequentially accessed, so they would have gained little from large pages anyway. Add MY_TRY_LARGE_PAGES: my_large_malloc() and my_large_virtual_alloc() now only attempt large pages when a caller passes this flag, instead of always trying whenever the global option is set. Only the buffer pool and the key caches pass it. The redo log buffer, tmp_buf, and row0log.cc's crypt buffers no longer request large pages at all, which removes the size-rounding bug for them without touching that code. Also fix a broken mtr suppression regex in main.large_pages that would fail the test on Windows. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38869 sequence conflicts with streaming replication Sequence access conflicts with streaming replication could cause the server to hang, as shown in MDEV-38869. A sequence table is written from SEQUENCE::next_value() while SEQUENCE::mutex is held. For a streaming transaction the row write in handler::ha_write_row() would then replicate a fragment and block waiting for certification and commit order, while an applier may be waiting for the same mutex in SEQUENCE::set_value(). Neither side can proceed, the node deadlocks and the BF abort of the local transaction can never be delivered. This commit avoids the deadlock by skipping the streaming replication step for sequence table rows. The row is already in the write set and is replicated with the following fragment, or at commit. Only that one step is skipped. The skip is passed down as a parameter to wsrep_after_row() and wsrep_after_row_internal() rather than by not calling them at all, so the row is still counted against wsrep_max_ws_rows and wsrep_check_pk() still runs. A transaction using sequences heavily therefore cannot silently exceed the configured write set row limit. The commit has also a new mtr test for three sequence/SR conflict scenarios: galera.galera_sequences_bf_kill_sr - a streaming transaction and an applier competing for SEQUENCE::mutex, where both are expected to commit - the same, but with the applier also BF aborting the local transaction over a gap lock. A streaming transaction cannot be replayed, so it is rolled back and the client gets ER_LOCK_DEADLOCK. The applier is held at the abort_trx_end sync point until the abort has been issued, so that the local transaction cannot finish its fragment first - twelve row inserts on both nodes with wsrep_trx_fragment_unit=rows, so that each node reserves several sequence cache ranges and the sequence table writes land inside fragments carrying several rows. The values the two nodes hand out must not overlap |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
PranavKTiwari
pranav.tiwari@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38633: Row events in statement based binlog: optimization possible? A failing multi-table UPDATE or DELETE marked every target temporary table as not up to date in the binary log, even when nothing had been changed. Any later statement reading such a table was then forced to use row logging. Only mark the tables when something was actually changed—that is, when rows were updated/deleted or a non-transactional table was modified. If nothing changed, set THD::tmp_table_binlog_handled so that mark_tmp_table_as_free_for_reuse() does not mark them either. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39491 Parallel Query: InnoDB clustered-index partitioning for parallel scan Implement the InnoDB side of the handler parallel-scan API: partition the clustered index into disjoint key-range chunks and serve them to the SQL layer as pull-based scan jobs. Each chunk is read through the normal row_search_mvcc() path, so MVCC visibility, AHI and the prefetch cache keep working unchanged. InnoDB spawns no threads of its own. This commit introduces only the whole index partitioning for a subsequent full table scan in parallel. Ranges of the index are not supported and will be implemented later. - Parallel_coordinator (row0pcoord.{h,cc}): adapted from MySQL's Parallel_reader, reduced to partitioning and job distribution. Walks the index top-down into Exec_ctx chunks bounded by clustered-key tuples. - ha_innobase: implements parallel_{init,end}_coordinator, parallel_get_worker_context, parallel_{init,end}_worker and parallel_get_next_row. - Chunk-boundary clamp: row_prebuilt_t::m_pscan_end_tuple (NULL = unbounded) makes row_search_mvcc() stop before prefetching past the chunk's exclusive upper bound, treated as end-of-range so the next chunk is pulled. - btr_pcur_open_on_user_rec(page_cur_t) overload to anchor chunk boundaries. - Build: row0pcoord.cc added to CMakeLists.txt and auto_event_names[]. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Petrunia
sergey@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Small cleanups: set thd->userstat_running in pwt_worker_base, comments. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| fixup! e84155a7958e9bbdf1ffde788523d41a99a17d84 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38243 Write binlog row events for changes done by cascading FK operations Refactoring according to Serg's review. In this version, SE/server API now narrows the SE role to just report the changes done by foreign key cascading, and server side does most of the work after that. Added a design document MDEV-38243-design.md |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39226: Push whole multi-table update/delete down into engines Give storage engines a way to take over an entire multi-table UPDATE/DELETE, the way they can already take over a SELECT. Without it the join, the row matching and every modification run in the SQL layer even when an engine could do the whole statement itself in one step; a single-table UPDATE/DELETE already avoids this via direct_update_rows()/direct_delete_rows(), but a multi-table statement has no primary handler object to drive that path. This adds a generic, engine-agnostic pushdown interface: the SQL layer offers the statement to the engine, and if the engine accepts it, it performs the whole thing and reports only the row counts. - Split select_handler into a pushdown_handler base with select_handler (result set) and a new multi_upddel_handler (runs a whole UPDATE/DELETE, reports row counts, reported as PUSHED UPDATE/PUSHED DELETE); add handlerton::create_multi_upddel, looked up in Sql_cmd_dml::execute_inner(). - multi_update/multi_delete gain direct_update_delete_done(), which records the engine's counts so send_eof() binlogs and replies without the SQL-layer loop; it forces statement-format binlogging so the change still replicates under binlog_format=ROW, and errors out instead of silently dropping counts for an unsupported result object. - FederatedX implements the interface as the reference engine used to test correctness: it prints the statement back and runs it remotely, passes the engine's error code/SQLSTATE through, reads the matched count from the remote info string, executes IGNORE locally, and only pushes down when all tables share one remote server (same as SELECT/derived/unit pushdown). Test: federated.federatedx_pushdown_upd_del. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40656 Bypass REVOKE DENY ... FROM PUBLIC privilege check. DENY ... TO PUBLIC denies everyone, including whoever tries to revoke it, via the "deny wins" merge at every scope (global, db, table, column, routine). Allow REVOKE DENY ... FROM PUBLIC when the revoker can UPDATE mysql.global_priv (same as hand-editing). Assisted-by: Claude:claude-5-sonnet |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39491 Parallel Query: InnoDB clustered-index partitioning for parallel scan Implement the InnoDB side of the handler parallel-scan API: partition the clustered index into disjoint key-range chunks and serve them to the SQL layer as pull-based scan jobs. Each chunk is read through the normal row_search_mvcc() path, so MVCC visibility, AHI and the prefetch cache keep working unchanged. InnoDB spawns no threads of its own. This commit introduces only the whole index partitioning for a subsequent full table scan in parallel. Ranges of the index are not supported and will be implemented later. - Parallel_coordinator (row0pcoord.{h,cc}): adapted from MySQL's Parallel_reader, reduced to partitioning and job distribution. Walks the index top-down into Exec_ctx chunks bounded by clustered-key tuples. - ha_innobase: implements parallel_{init,end}_coordinator, parallel_get_worker_context, parallel_{init,end}_worker and parallel_get_next_row. - Chunk-boundary clamp: row_prebuilt_t::m_pscan_end_tuple (NULL = unbounded) makes row_search_mvcc() stop before prefetching past the chunk's exclusive upper bound, treated as end-of-range so the next chunk is pulled. - btr_pcur_open_on_user_rec(page_cur_t) overload to anchor chunk boundaries. - Build: row0pcoord.cc added to CMakeLists.txt and auto_event_names[]. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexander Barkov
bar@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39518 Allow prepared statements in stored functions in assignment right hand Allowing prepared statements in stored functions when a stored function is used in an assignment right hand. Both DEFAULT clause of a variable initialization and the right side of the SET statement are supported: CREATE PROCEDURE p1() BEGIN -- case 1: DEFAULT clause DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK -- case 2: SP variable assignment statement DECLARE spvar2 INT; SET spvar2= f1_with_ps(); -- OK END; - Only assignments to SP variables works for now: * SET spvar= func_with_ps(); -- OK * SET @uvar= func_with_ps(); -- Error - Only bare function calls are supported for now. Using a function in an expression does not make it PS-safe yet: SET v= f1()+0; - The parser now does not reject PS statements in stored functions. PS applicability in stored functions is now detected at run time. Note, PS statements in triggers are still prohibited by the parser. - Functions with PS do not acquire MDL locks on tables, and no MDL is taken on the routines themselves either. They work like procedures in terms of table opening and routine locking: a concurrent DROP FUNCTION can complete while such a function is executing. - Functions with PS are not replicated as a single `SELECT f1()` call. They are replicated per-statement, like procedures. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38243 Write binlog row events for changes done by cascading FK operations Fixes according to Kristian Nielsen's review: * Removed obsolete checks for slave thread * Supporting slave with old MariaDB version. Events logged in cascade operation are additionally flagged with the long-standing NO_FOREIGN_KEY_CHECKS_F, so a replica that does not understand FK_CASCADE_EVENTS_F still disables foreign key checks and does not re-execute the cascade Also, thee are now binlog event flags to mark both original and derived events. This will make it possible for the slave to choose whether to use the derived events in applying or to execute the cascade operation There is a new test rpl.rpl_fk_cascade_binlog_row_old_slave, for checking compatibility with replication slave of old mariadb version |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39845 Introduce parallel scan API Add a handler-level interface that engines can implement to support parallel table scans, with a serial-scan fallback when unsupported.. - HA_CAN_PARALLEL_SCAN table flag and handler::is_parallel_scan_supported() - Coordinator-side methods (parallel_init_coordinator / parallel_end_coordinator, parallel_get_worker_context) driven by the master thread - Worker-side methods (parallel_init_worker / parallel_get_next_row / parallel_end_worker) driven by child threads, with the ha_parallel_get_next_row() wrapper doing the usual bookkeeping - Parallel_worker_ctx, an opaque per-worker context subclass |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexander Barkov
bar@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39518 Allow prepared statements in stored functions in assignment right hand Allowing prepared statements in stored functions when a stored function is used in an assignment right hand. Both DEFAULT clause of a variable initialization and the right side of the SET statement are supported: CREATE PROCEDURE p1() BEGIN -- case 1: DEFAULT clause DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK -- case 2: SP variable assignment statement DECLARE spvar2 INT; SET spvar= f1_with_ps(); -- OK END; - Only assignments to SP variables works for now: * SET spvar= func_with_ps(); -- OK * SET @uvar= func_with_ps(); -- Error - Only bare function calls are supported for now. Using a function in an expression does not make it PS-safe yet yet: SET v= f1()+0; - The parser now does not reject PS statements in stored functions. PS applicability in stored functions is now detected at run time. Note, PS statements in triggers are still prohibited by the parser. - Functions with PS do not acquire MDL locks on tables. They work like procedures in terms of table opening. - Functions with PS are not replicated as a single `SELECT f1()` call. They are replicated per-statement, like procedures. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38243 Write binlog row events for changes done by cascading FK operations 3'rd batch of review changes. Now the SE has to call for cascade service only to report of row before and after image and the end of cascade operation. There is new SE service interface: include/mysql/service_thd_fk_cascade.h For server, there are 3 consumers for cascade operationss: binlogging, firing triggers, client table FK checks, of which only binlogging has implemeentation |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40168 wip works: SET SESSION debug = '+d,test_invisible_index,test_completely_invisible'; create table t1 (c int, j json, key idx ((CAST(j->'$.tags' AS CHAR(6) ARRAY))))engine=innodb; SHOW CREATE TABLE t1; Table Create Table t1 CREATE TABLE `t1` ( `c` int(11) DEFAULT NULL, `j` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL CHECK (json_valid(`j`)) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci show index from t1; Table Non_unique Key_name Seq_in_index Column_name Collation Cardinality Sub_part Packed Null Index_type Comment Index_comment Ignored t1 1 invisible1 1 invisible1 A 0 NULL NULL YES BTREE NO t1 1 idx 1 DB_MVI_1 NULL NULL NULL NULL YES FULLTEXT NO set @old_innodb_ft_aux_table=@@global.innodb_ft_aux_table; set global innodb_ft_aux_table='test/t1'; insert into t1 values (1, '{"tags": ["1", "abcde", "34567"]}'); SELECT * FROM INFORMATION_SCHEMA.INNODB_FT_INDEX_CACHE; WORD FIRST_DOC_ID LAST_DOC_ID DOC_COUNT DOC_ID POSITION 34567 1 1 1 1 2 abcde 1 1 1 1 1 DROP TABLE t1; set global innodb_ft_aux_table=@old_innodb_ft_aux_table; select mvi_encode('[1, 42, 3]', int); mvi_encode('[1, 42, 3]', int) 8000000000000001 800000000000002a 8000000000000003 select mvi_encode('[1, 42, 3]', unsigned); mvi_encode('[1, 42, 3]', unsigned) 0000000000000001 000000000000002a 0000000000000003 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40903 galera.MDEV-38260 fails with ER_BINLOG_IN_USE on RESET MASTER The test fails from 12.3 onwards on the final cleanup step: mysqltest: At line 98: query 'reset master' failed: ER_BINLOG_IN_USE (4243): Cannot execute RESET MASTER as the binlog is in use by a connected slave or other RESET MASTER or binlog reader. The binlog-in-engine work added a use-count guard around RESET MASTER in MYSQL_BIN_LOG::reset_logs(): every binlog reader, a slave dump thread included, registers itself through start_use_binlog(), and RESET MASTER now refuses with ER_BINLOG_IN_USE while the count is non-zero. Earlier versions reset the binlog even with a slave still attached, which is why the test passes on 11.8 and older. The test's cleanup stops the async slave on node_2 and then immediately runs RESET MASTER on the master node_3. The dump thread on node_3 does not disappear at STOP SLAVE; it lingers until it next notices the closed connection, so it is often still registered when RESET MASTER arrives. Fix here is to use include/reset_master.inc, which terminates dump threads and retries on ER_BINLOG_IN_USE. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40656 Bypass REVOKE DENY ... FROM PUBLIC privilege check. DENY ... TO PUBLIC denies everyone, including whoever tries to revoke it, via the "deny wins" merge at every scope (global, db, table, column, routine). Allow REVOKE DENY ... FROM PUBLIC when the revoker can UPDATE mysql.global_priv (same as hand-editing). Assisted-by: Claude:claude-5-sonnet |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleg Smirnov
olernov@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40005 Parallel Query: allow parallel scans on secondary indexes in InnoDB This commit extends parallel scanning to InnoDB secondary indexes, both whole-index scans and scans of ranges of the key. It complements the previously introduced partitioning of the clustered index. Both unique and non-unique secondary indexes are supported, as well as covering and non-covering scans. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||