Console View
|
Categories: connectors experimental galera main |
|
| connectors | experimental | galera | main | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alessandro Vetere
iminelink@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40408 btr_page_reorganize_low() uses the buffer pool just to obtain a scratch block Add buf_pool.scratch_buf, a pool of page frames that the page reorganization operations use instead of taking a block from the global buffer pool. buf_pool_t::scratch_buffer: A singly linked list of chunks of buf_tmp_buffer_t slots. A thread holds at most one slot at a time, and it holds a page latch while doing so, which is why reserve() must not wait for the buffer pool or for I/O. When every slot is in use, reserve() appends a chunk that holds twice the slots of the last one, up to a limit on the size of one chunk. Only grow() waits, on the mutex that serializes it and on the allocator. A chunk is never moved or freed before close(), so a thread can keep using the slot that it reserved while another thread appends a chunk. Debug builds start with a single slot, so that a second concurrent page reorganization exercises grow(). buf_pool_t::scratch_buffer::shrink(): Free the page frames of the slots that are not in use. The slots and the chunks are kept, because a slot costs 32 bytes while a page frame costs srv_page_size. A frame survives the first pass, because that pass only clears the used flag. The master thread calls this often while the server is idle and rarely while it is active, and buf_pool_t::garbage_collect() calls it under memory pressure, where it ignores the used flag, because releasing these frames is much cheaper than shrinking the buffer pool. buf_pool_t::io_buf_t::acquire(): Factor out the scan for an unreserved slot, which io_buf_t::reserve() ran twice and the scratch buffer reuses. btr_page_reorganize_low(), page_zip_reorganize(): Obtain the scratch page frame from buf_pool.scratch_buf. This removes the buf_pool.mutex acquisition and the free block wait that buf_block_alloc() could perform while at least a page X-latch was being held. btr_page_reorganize_low() also used to leak the block on its error paths; a single exit now releases the slot. page_copy_rec_list_end_no_locks(), lock_move_reorganize_page(): Take the source page frame instead of a source buf_block_t, because the source is no longer a buffer pool block. In page_copy_rec_list_end_no_locks() the source page is page_align(rec) at every call site, so only the record is passed. buf_tmp_buffer_t::acquire(), buf_tmp_buffer_t::release(): Use acquire and release memory ordering, so that the page frame pointer of a slot is published to the next thread that reserves the slot. acquire() also reads the flag before the exchange, so that a scan across an array of slots does not write to the slots that it finds reserved. release() also marks the page frame undefined for Valgrind and MSan, because the frame stays allocated for the next reserver and no deallocation marks it. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-22992 Refactor VIO into layered transports and filters Replace the function-pointer VIO implementation with an abstract C++ interface while retaining the procedural C entry points. Implement socket and named-pipe transports and composable filters for client read-ahead, Windows thread-pool prefetch, and TLS. OpenSSL uses a custom BIO, while wolfSSL uses callbacks that perform I/O through the VIO below the TLS filter. This keeps waits and timeouts in the transport layer. Keep sockets nonblocking and implement timed I/O with transport waits. Named pipes use overlapped I/O for timeout-aware waits and report blocking waits through the same scheduler callbacks as sockets. Semi-sync temporarily changes the real VIO read timeout instead of copying VIO state. Hide transport and TLS implementation state behind accessors. Expose the TLS handle opaquely and update callers that previously accessed VIO fields directly. Compile the VIO implementations as C++ and retain PSI memory accounting for VIO allocations. Adapt Windows thread-pool pre-read to a Prefetched_vio filter inserted above the transport so both plain and TLS connections consume prefetched bytes through the same layered VIO path. Clean header files so that vio headers no longer include OpenSSL or wolfSSL headers. Remove some legacy functionality: - vio_close() with its double-close guards appeared hard to maintain in the class hierarchy, and had been unnecessary for the last 15 years -- we consistently have used vio_shutdown() for waking up threads stuck in network IO. Associated things that are also gone: preprocessor definition SIGNAL_WITH_VIO_CLOSE (always defined), VIO_STATE_CLOSED. The VIO_CLOSED type, which was used as a sentinel, was renamed to VIO_TYPE_INVALID. - vio_io_wait() used in a single place, replaced by read with timeout. - vio_reset() to create SSL, replaced by vio_wrap. Assisted-by: Claude:claude-sonnet-5 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Abdelrahman Hedia
bodyhadia44@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-29803: Change mariadb-binlog --gtid-strict-mode default to OFF The --gtid-strict-mode option in mariadb-binlog was introduced in MDEV-4989 with a default of ON. This causes mariadb-binlog to refuse to display events when it encounters out-of-order GTIDs, which commonly happens when replaying a remote binlog into a server and then reading back the resulting local binlog files. This is overly restrictive for a diagnostic/display tool. While the server's gtid_strict_mode makes sense as a safety mechanism, applying the same strict validation by default in the client tool prevents users from even inspecting problematic binlog files. Change the default to OFF so that mariadb-binlog processes binlog files without erroring on out-of-order GTIDs by default. Users who want strict validation can still explicitly pass --gtid-strict-mode. Added regression test binlog.mdev_29803 that verifies: - Default (OFF): reading binlog files with replayed events succeeds - Explicit --gtid-strict-mode: still produces the expected error |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Andrei Elkin
andrei.elkin@pp.inet.fi |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| explore deeper, incl the 2nd problematic DROP. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39226: Push whole multi-table update/delete down into engines Give storage engines a way to take over an entire multi-table UPDATE/DELETE, the way they can already take over a SELECT. Without it the join, the row matching and every modification run in the SQL layer even when an engine could do the whole statement itself in one step; a single-table UPDATE/DELETE already avoids this via direct_update_rows()/direct_delete_rows(), but a multi-table statement has no primary handler object to drive that path. This adds a generic, engine-agnostic pushdown interface: the SQL layer offers the statement to the engine, and if the engine accepts it, it performs the whole thing and reports only the row counts. - Split select_handler into a pushdown_handler base with select_handler (result set) and a new multi_upddel_handler (runs a whole UPDATE/DELETE, reports row counts, reported as PUSHED UPDATE/PUSHED DELETE); add handlerton::create_multi_upddel, looked up in Sql_cmd_dml::execute_inner(). - multi_update/multi_delete gain direct_update_delete_done(), which records the engine's counts so send_eof() binlogs and replies without the SQL-layer loop; it forces statement-format binlogging so the change still replicates under binlog_format=ROW, and errors out instead of silently dropping counts for an unsupported result object. - FederatedX implements the interface as the reference engine used to test correctness: it prints the statement back and runs it remotely, passes the engine's error code/SQLSTATE through, reads the matched count from the remote info string, executes IGNORE locally, and only pushes down when all tables share one remote server (same as SELECT/derived/unit pushdown). Test: federated.federatedx_pushdown_upd_del. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38869 sequence conflicts with streaming replication Sequence access conflicts with streaming replication could cause the server to hang, as shown in MDEV-38869. A sequence table is written from SEQUENCE::next_value() while SEQUENCE::mutex is held. For a streaming transaction the row write in handler::ha_write_row() would then replicate a fragment and block waiting for certification and commit order, while an applier may be waiting for the same mutex in SEQUENCE::set_value(). Neither side can proceed, the node deadlocks and the BF abort of the local transaction can never be delivered. This commit avoids the deadlock by skipping the streaming replication step for sequence table rows. The row is already in the write set and is replicated with the following fragment, or at commit. Only that one step is skipped. The skip is passed down as a parameter to wsrep_after_row() and wsrep_after_row_internal() rather than by not calling them at all, so the row is still counted against wsrep_max_ws_rows and wsrep_check_pk() still runs. A transaction using sequences heavily therefore cannot silently exceed the configured write set row limit. The commit has also a new mtr test for three sequence/SR conflict scenarios: galera.galera_sequences_bf_kill_sr - a streaming transaction and an applier competing for SEQUENCE::mutex, where both are expected to commit - the same, but with the applier also BF aborting the local transaction over a gap lock. A streaming transaction cannot be replayed, so it is rolled back and the client gets ER_LOCK_DEADLOCK. The applier is held at the abort_trx_end sync point until the abort has been issued, so that the local transaction cannot finish its fragment first - twelve row inserts on both nodes with wsrep_trx_fragment_unit=rows, so that each node reserves several sequence cache ranges and the sequence table writes land inside fragments carrying several rows. The values the two nodes hand out must not overlap |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| debug instrumentation | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Kristian Nielsen
knielsen@knielsen-hq.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stop using common_header_len and post_header_len from FD event The FORMAT_DESCRIPTION_EVENT contains values common_header_len and post_header_len array for individual event types, which were intended to describe the format of following events in the binlog/relaylog. However, this flexibility has been problematic and never useful: - The values have never changed in MariaDB since before 10.0, nor in MySQL. - Having to always have the correct format description event available in the code when reading an event causes a lot of complexity (and has historically caused numerous bugs). - The potential for a corrupt or malicious format description event requires careful validation that these length values are correct, which again has historically seen bugs. Since the values never change anyway, this patch removed the code that reads them when reading the format description event, just using the hard-coded values that are the only valid numbers anyway. Remove support for ancient row events with post_header_len=6. Remove support for reading binlogs created by custom MySQL forks with different common_header_len (ie. MDEV-4645). Signed-off-by: Kristian Nielsen <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
forkfun
alice.sherepa@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40303 SIGSEGV in Item_field::type_handler() on PS re-execution Re-executing a prepared statement that reads a derived-table column holding a scalar UNION subquery crashed the server: between executions st_select_lex_unit::cleanup() frees union_result and sets 'cleaned' but leaves 'prepared' set, so on re-fix set_row() walks a stale item_list whose Item_field::field are NULL. Fix: prepare() re-prepares a cleaned unit instead of treating it as still prepared. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
squash! 208ce0ae56e1ddd990555028b82aabd96909484f InnoDB_backup::commit(): Enqueue the remaining log. InnoDB_backup::checkpoint_complete(): If backup is running and commit() has not been called, add each completed innodb_archive_log=ON file to InnoDB_backup::queue. Else, skip or delete, as appropriate. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Val Doroshchuk
valbok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rename duckdb file name to allow to use duckdb as schema If duckdb is used as schema, DuckDB requires to use it in queries explicitly since the name conflicts with the catalog. This fixes Ambiguous reference to catalog or schema "duckdb" - use a fully qualified path like '.duckdb' |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thirunarayanan Balathandayuthapani
thiru@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39091 BACKUP SERVER of ENGINE=RocksDB to the local file system Wire the RocksDB (MyRocks) engine into BACKUP SERVER TO and BACKUP SERVER WITH (streaming). RocksDB SST files are immutable, so a consistent point-in-time snapshot is obtained cheaply with a rocksdb::Checkpoint. The checkpoint is frozen at BACKUP_PHASE_NO_COMMIT because the checkpoint files are immutable, the actual copy is deferred to BACKUP_PHASE_FINISH, after the backup MDL has been released, and the server-side checkpoint is removed at end of FINISH. No user-level lock is needed: MDL_BACKUP_START already serializes concurrent BACKUP SERVER. Files are copied into the target's "#rocksdb" subdirectory. The default rocksdb_datadir ("./#rocksdb") makes restore transparent: on restore the files land at <datadir>/#rocksdb, so pointing a server at the extracted backup just works, with no copy-back and no RocksDB prepare step. storage/rocksdb/rdb_backup_server.{h,cc}: New RocksDB_backup context and the three handlerton hooks. The checkpoint file list is drained across N CONCURRENT step threads via a lock-free atomic cursor; each file is copied with copy_entire_file() (directory target) or backup_stream_*(). Since SSTs are immutable, the sendfile(2) fast path (backup_stream_append_async) is used for the stream target. rdb_create_checkpoint(): Factor the checkpoint creation rdb_remove_checkpoint(): Remove the checkpoint creation logic. rdb_get_datadir(): Get the rocksdb data directory. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 Fix tests and system scripts to foreign key requirements Storing foreign key metadata in TABLE_SHARE makes opening a foreign table preopen its referenced tables, so a referenced table must be created before the foreign one and dropped after it. Rearrange the order of CREATE/DROP commands in tests and system scripts accordingly. Affected: the help tables in mariadb_system_tables{,_fix}.sql (with system_mysql_db_fix* results), fetch_first, union, insert_notembedded, instant_alter_index_rename and opt_context_store_ddls; some also index a referenced column or add a missing referenced table, and opt_context_store_ddls re-records the now-shown MyISAM foreign key. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-34392 Check foreign key column nullability in the server layer Making a foreign key column NOT NULL must be refused when a referential action can still write NULL into it -- ON UPDATE/DELETE SET NULL, or ON UPDATE CASCADE from a NULLable parent. Until now this was enforced through a per-column nullability bitmap in FK_info (fields_nullable) that the storage engine allocated and populated, and that the server read back while checking an ALTER. Since MDEV-20865 the server keeps the foreign key definitions in TABLE_SHARE, so this round trip through the engine is redundant. Perform the check directly in mysql_prepare_alter_table() from the stored referential actions and the old/new column definitions, and drop the FK_info bitmap along with its assign_nullable()/set_nullable()/is_nullable() helpers. Incompatible changes are still rejected with ER_FK_COLUMN_NOT_NULL. Fixes foreign_null test. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 Store foreign key info in TABLE_SHARE 1. Access foreign keys via TABLE_SHARE::foreign_keys and TABLE_SHARE::referenced_keys; foreign_keys and referenced_keys are lists in TABLE_SHARE. 2. Remove handler FK interface: - get_foreign_key_list() - get_parent_foreign_key_list() - referenced_by_foreign_key() 3. Invalidate referenced shares on: - RENAME TABLE - DROP TABLE - RENAME COLUMN - ADD FOREIGN KEY When foreign table is created or altered by the above operations all referenced shares are closed. This blocks the operation while any referenced shares are used (when at least one its TABLE instance is locked). 4. Update referenced shares on: - CREATE TABLE On CREATE TABLE add items to referenced_keys of referenced shares. States of referenced shares are restored in case of errors. 5. Invalidate foreign shares on: - RENAME TABLE - RENAME COLUMN The above-mentioned blocking takes effect. 6. Check foreign/referenced shares consistency on: - CHECK TABLE 7. Temporary change until MDEV-21051: InnoDB fill foreign key info at handler open(). FOREIGN_KEY_INFO is refactored to FK_info holding Lex_cstring. On first TABLE open FK_info is loaded from storage engine into TABLE_SHARE. All referenced shares (if any exists) are closed. This leads to blocking of first time foreign table open while referenced tables are used. MDEV-21311 Converge Foreign_key and supplemental generated Key together mysql_prepare_create_table() does data validation and such utilities as automatic name generation. But it does that only for indexes and ignores Foreign_key objects. Now as Foreign_key data needs to be stored in FRM files as well this processing must be done for it like for any other Key objects. Replace Key::FOREIGN_KEY type with Key::foreign flag of type Key::MULTIPLE and Key::generated set to true. Construct one object with Key::foreign == true instead of two objects of type Key::FOREIGN_KEY and Key::MULTIPLE. MDEV-21051 datadict refactorings - Move read_extra2() to datadict.cc - Refactored extra2_fields to Extra2_info - build_frm_image() readability MDEV-21051 build_table_shadow_filename() refactoring mysql_prepare_alter_table() leaks fixes MDEV-21051 amend system tables locking restriction Table mysql.help_relation has foreign key to mysql.help_keyword. On bootstrap when help_relation is opened, it preopens help_keyword for READ and fails in lock_tables_check(). If system table is opened for write then fk references are opened for write. Related to: Bug#25422, WL#3984 Tests: main.lock MDEV-21051 Store and read foreign key info into/from FRM files 1. Introduce Foreign_key_io class which creates/parses binary stream containing foreign key structures. Referenced tables store there only hints about foreign tables (their db and name), they restore full info from the corresponding tables. Foreign_key_io is stored under new EXTRA2_FOREIGN_KEY_INFO field in extra2 section of FRM file. 2. Modify mysql_prepare_create_table() to generate names for foreign keys. Until InnoDB storage of foreign keys is removed, FK names must be unique across the database: the FK name must be based on table name. 3. Keep stored data in sync on DDL changes. Referenced tables update their foreign hints after following operations on foreign tables: - RENAME TABLE - DROP TABLE - CREATE TABLE - ADD FOREIGN KEY - DROP FOREIGN KEY Foreign tables update their foreign info after following operations on referenced tables: - RENAME TABLE - RENAME COLUMN 4. To achieve 3. there must be ability to rewrite extra2 section of FRM file without full reparse. FRM binary is built from primary structures like HA_CREATE_INFO and cannot be built from TABLE_SHARE. Use shadow write and rename like fast_alter_partition_table() does. Shadow FRM is new FRM file that replaces the old one. CREATE TABLE workflow: 1. Foreign_key is constructed in parser, placed into alter_info->key_list; 2. mysql_prepare_create_table() translates them to FK_info, assigns foreign_id if needed; 3. build_frm_image() writes two FK_info lists into FRM's extra2 section, for referenced keys it stores only table names (hints); 4. init_from_binary_frm_image() parses extra2 section and fills foreign_keys and referenced_keys of TABLE_SHARE. It restores referenced_keys by reading hint list of table names, opening corresponding shares and restoring FK_info from their foreign_keys. Hints resolution is done only when initializing non-temporary shares. Usually temporary share has different (temporary) name and it is impossible to resolve foreign keys by that name (as we identify them by both foreign and referenced table names). Another not unimportant reason is performance: this saves spare share acquisitions. ALTER TABLE workflow: 1. Foreign_key is constructed in parser, placed into alter_info->key_list; 2. mysql_prepare_alter_table() prepares action lists and share list of foreigns/references; 3. mysql_prepare_alter_table() locks list of foreigns/references by MDL_INTENTION_EXCLUSIVE, acquires shares; 4. prepare_create_table() converts key_list into FK_list, assigns foreign_id; 5. shadow FRM of altered table is created; 6. data is copied; 7. altered table is locked by MDL_EXCLUSIVE; 8. fk_handle_alter() processes action lists, creates FK backups, modifies shares, writes shadow FRMs; 9. altered table is closed; 10. shadow FRMs are installed; 11. altered table is renamed, FRM backup deleted; 12. (TBD in MDEV-21053) shadow FRMs installation log closed, backups deleted; On FK backup system: In case of failed DDL operation all shares that was modified must be restored into original state. This is done by FK_ddl_backup (CREATE, DROP), FK_rename_backup (RENAME), FK_alter_backup (ALTER). On STL usage: STL is used for utility not performance-critical algorithms, core structures hold native List. A wrapper was made to convert STL exception into bool error status or NULL value. MDEV-20865 fk_check_consistency() in CHECK TABLE Self-refs fix Test table_flags fix: "debug" deviation is now gone. FIXMEs: +16 -1 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Andrei Elkin
andrei.elkin@pp.inet.fi |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| explore deeper, incl the 2nd problematic DROP. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40765 Assertion `"unexpected references" == 0' failed upon failing CREATE OR REPLACE There was a few reasons for this failure: - There was no timeout for the mdl_lock in the ddl log in execute_rename_table. This is needed to protect against a concurrent purge in InnoDB. If the purge would be running, the ddl would call rename_table without a ddl protection the assert could happen. - InnoDB locked the original table name in purge, not the temporary name used to cache the table in case of rollback. The effect is that if a purge happens during rename the assert could happen. - The table was a partitioned and the detection of partitioned tables did not take into account temporary #sql-create- prefixed tables. Other things: - The code in execute_rename_table() assumes that InnoDB never holds a MDL lock for more more than a few milliseconds. I am using a timeout for 10 seconds, just in case. In practice this timeout should be very short. - Added a 10 second timeout for the MDL lock in execute_drop_table() - Removed mysql_mutex_unlock(&LOCK_gdl) / mysql_mutex_lock(&LOCK_gdl) around calls to binlog as these are unsafe. The binlog code uses global variables that needs protection from other caller. - Fixed a few compiler warnings related to tmp_file_prefix. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| PQ introduce transport API, batch transport, tmp table transport | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| explore bb-13.2-MDEV-22992:rpl.rpl_xa | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thirunarayanan B
thiru@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge branch 'MDEV-14992' into MDEV-39091 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexander Barkov
bar@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39518 Allow prepared statements in stored functions in assignment right hand TODO: ps_in_func-stack3-pfp.test crashes with InnoDB!!! Allowing prepared statements in stored functions when a stored function is used in an assignment right hand. Both DEFAULT clause of a variable initialization and the right side of the SET statement are supported: CREATE PROCEDURE p1() BEGIN -- case 1: DEFAULT clause DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK -- case 2: SP variable assignment statement DECLARE spvar2 INT; SET spvar= f1_with_ps(); -- OK END; - The parser now does not reject PS statements in stored functions. PS applicability in stored functions is now detected at run time. Note, PS statements in triggers are still prohibited by the parser. - Functions with PS do not acquire MDL locks on tables. They work like procedures in terms of table opening. - Functions with PS are not replicated as a single `SELECT f1()` call. They are replicated per-statement, like procedures. - Only bare function calls are supported for now. Using a function in an expression does not make it PS-safe yet yet: SET v= f1()+0; |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40799 Runtime plugin/UDF load errors lost under --silent-startup Regression from MDEV-32745 (7828fb475b0), which guarded the plugin-load my_error() calls with opt_silent_startup. That option is a lifetime global, set once at startup and never reset, so the guard suppressed the SQL error for the whole server lifetime, not just during startup. Runtime operations (INSTALL PLUGIN, CREATE FUNCTION ... SONAME) then skipped my_error(), never set the diagnostics area and wrongly succeeded - e.g. main.ps's "call proc_1()" no longer failed with ER_CANT_OPEN_LIBRARY. Startup callers pass MYF(ME_ERROR_LOG); runtime callers pass MYF(0). Gate the silencing on that flag via silent_plugin_startup() so it applies only to the startup error-log path, and runtime errors always reach the client. No new test case: the runtime failure path is already covered by existing tests (e.g. main.ps's ER_CANT_OPEN_LIBRARY check). The regression stayed invisible only because stock MTR does not start servers with --silent-startup. A dedicated test would have to restart the server with --silent-startup solely to assert that a startup-only option does not affect runtime, which adds little over the restored invariant. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-22991 Support SSL over named pipes Update libmariadb, to remove named pipe SSL check. Update client.c to remove that check as well. Add MTR test for named pipe + SSL: - for Connector/C client, new test named_pipe_ssl - for in-server clent, run mariabackup with user created as "IDENTIFIED WITH named_pipe REQUIRE SSL" |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Daniel Bartholomew
db@dbart.us |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge branch 'bb-10.11-bumpversion' of github.com:MariaDB/server into bb-10.11-bumpversion | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Monty
monty@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40776 Atomic CREATE OR REPLACE silently breaks the foreign key Give an error if one tries to drop a table referenced by a foreign keys This is needed as innodb will keep the reference to the origina table even when it is renamed to a temporary name as part of create or replace. Other things: - Changed the error message for ER_TRUNCATE_ILLEGAL_FK to say "Cannot drop or truncate a table ..." |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Kristian Nielsen
knielsen@knielsen-hq.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stop using common_header_len and post_header_len from FD event Follow-up patch, remove a lot of Format_description_log_event * function arguments that are no longer needed. Signed-off-by: Kristian Nielsen <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Daniel Bartholomew
db@dbart.us |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| bump the VERSION | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 extra2_fields refactored to Extra2_info class read_extra2() is now Extra2_info::read() Additional assertions for checking size consistency. Extra2_info::write() is used by further MDEV-20865 development. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Andrei Elkin
andrei.elkin@pp.inet.fi |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| explore deeper, incl the 2nd problematic DROP. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Andrei Elkin
andrei.elkin@pp.inet.fi |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| explore bb-13.2-MDEV-22992:rpl.rpl_xa | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thirunarayanan Balathandayuthapani
thiru@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39091 BACKUP SERVER of ENGINE=RocksDB to the local file system Wire the RocksDB (MyRocks) engine into BACKUP SERVER TO and BACKUP SERVER WITH (streaming). RocksDB SST files are immutable, so a consistent point-in-time snapshot is obtained cheaply with a rocksdb::Checkpoint. The checkpoint is frozen at BACKUP_PHASE_NO_COMMIT because the checkpoint files are immutable, the actual copy is deferred to BACKUP_PHASE_FINISH, after the backup MDL has been released, and the server-side checkpoint is removed at end of FINISH. No user-level lock is needed: MDL_BACKUP_START already serializes concurrent BACKUP SERVER. Files are copied into the target's "#rocksdb" subdirectory. The default rocksdb_datadir ("./#rocksdb") makes restore transparent: on restore the files land at <datadir>/#rocksdb, so pointing a server at the extracted backup just works, with no copy-back and no RocksDB prepare step. storage/rocksdb/rdb_backup_server.{h,cc}: New RocksDB_backup context and the three handlerton hooks. The checkpoint file list is drained across N CONCURRENT step threads via a lock-free atomic cursor; each file is copied with copy_entire_file() (directory target) or backup_stream_*(). Since SSTs are immutable, the sendfile(2) fast path (backup_stream_append_async) is used for the stream target. rdb_create_checkpoint(): Factor the checkpoint creation rdb_remove_checkpoint(): Remove the checkpoint creation logic. rdb_get_datadir(): Get the rocksdb data directory. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Georgi (Joro) Kodinov
joro@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40909: main.user_limits is unstable The cleanup of the old connection goes on in the background. It can take longer on a busy server and this trigger the active sessions warning in DROP user. Stabilized the test by disabling warnings around DROP USER. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
squash! b625be27ca0a0b24730dd19d37d8bbf85917156c InnoDB_backup::context: Remove the pointer indirection and do not allow two overlapping backup operations. InnoDB_backup::init(): Wait for a possible previous BACKUP SERVER operation to reach the very end of InnoDB_backup::context::cleanup() so that the context can be safely reused. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 Refactor Share_acquire::fk_error() into acquire() Share_acquire::fk_error() inspected thd->is_error() after the acquisition to decide whether a missing referenced table is tolerable. That breaks when an outer Internal_error_handler consumes the error first, e.g. the Postponed_error_handler installed by mysql_rename_tables() (MDEV-27027). Move the decision into Share_acquire::acquire(): push a No_such_table_error_handler for the acquisition when foreign key checks are not enforced, so ER_NO_SUCH_TABLE is trapped above any outer handler rather than deferred or leaked into SHOW WARNINGS. The non-tolerated error outcome is recorded in Share_acquire::error; the four consumers check it instead of calling fk_error(). FIXME: Squash into the main "MDEV-20865 Store foreign key info in TABLE_SHARE" commit. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40311 Fix mysqldump-nl leaving slave connection behind The trailing CHANGE MASTER left master.info in the datadir, so a later test that restarts the server auto-started the slave and broke its check-testcase. Use RESET SLAVE ALL to drop the connection. MTR's internal check of the test case 'sys_vars.default_master_connection_basic' failed. This means that the test case does not preserve the state that existed before the test case was executed. Most likely the test case did not do a proper clean-up. It could also be caused by the previous test run by this thread, if the server wasn't restarted. This is the diff of the states of the servers before and after the test case was executed: -Slave_IO_Running No -Slave_SQL_Running No +Slave_IO_Running Connecting +Slave_SQL_Running Yes ... -Last_IO_Errno 0 -Last_IO_Error +Last_IO_Errno 1045 +Last_IO_Error error connecting to master '[email protected]:3306' - retry-time: 60 maximum-retries: 100000 message: Access denied for user 'root'@'localhost' (using password: NO) |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexander Barkov
bar@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39518 Allow prepared statements in stored functions in assignment right hand TODO: ps_in_func-stack3-pfp.test crashes with InnoDB!!! Allowing prepared statements in stored functions when a stored function is used in an assignment right hand. Both DEFAULT clause of a variable initialization and the right side of the SET statement are supported: CREATE PROCEDURE p1() BEGIN -- case 1: DEFAULT clause DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK -- case 2: SP variable assignment statement DECLARE spvar2 INT; SET spvar= f1_with_ps(); -- OK END; - The parser now does not reject PS statements in stored functions. PS applicability in stored functions is now detected at run time. Note, PS statements in triggers are still prohibited by the parser. - Functions with PS do not acquire MDL locks on tables. They work like procedures in terms of table opening. - Functions with PS are not replicated as a single `SELECT f1()` call. They are replicated per-statement, like procedures. - Only bare function calls are supported for now. Using a function in an expression does not make it PS-safe yet yet: SET v= f1()+0; |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 Reuse share in fk_handle_drop() mysql_rm_table_no_locks() already opens the dropped table's share, so pass it to fk_handle_drop() instead of re-acquiring it with GTS_FK_SHALLOW_HINTS. Skip fk_handle_drop() when the share is unreadable, letting the engine report the error. This restores ER_GET_ERRNO for main.partition_not_blackhole, which clear_error() used to mask by zeroing my_errno. New Share_acquire::inexistent_t lets DROP keep seeing ER_NO_SUCH_TABLE independent of foreign_key_checks (INEXISTENT_ALWAYS). Fixes main.partition_not_blackhole FIXME: Squash into the main "MDEV-20865 Store foreign key info in TABLE_SHARE" commit. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Andrei Elkin
andrei.elkin@pp.inet.fi |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| explore bb-13.2-MDEV-22992:rpl.rpl_xa | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||