Console View
|
Categories: connectors experimental galera main |
|
| connectors | experimental | galera | main | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 Fix tests and system scripts to foreign key requirements Storing foreign key metadata in TABLE_SHARE makes opening a foreign table preopen its referenced tables, so a referenced table must be created before the foreign one and dropped after it. Rearrange the order of CREATE/DROP commands in tests and system scripts accordingly. Affected: the help tables in mariadb_system_tables{,_fix}.sql (with system_mysql_db_fix* results), fetch_first, union, insert_notembedded, instant_alter_index_rename and opt_context_store_ddls; some also index a referenced column or add a missing referenced table, and opt_context_store_ddls re-records the now-shown MyISAM foreign key. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
gcov fix Fix coverage data corruption by enabling atomic profile updates When compiling with coverage instrumentation, concurrent or forked test executions can cause corruption of coverage data files (.gcda), resulting in negative hit counts and invalid coverage reports. Adding the compiler flag `-fprofile-update=atomic` ensures that updates to coverage counters are performed atomically, preventing race conditions during profile data writes. This change eliminates warnings about unexpected negative hit counts and improves the accuracy and reliability of coverage measurements in parallel or multithreaded test environments. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-14992 BACKUP SERVER The following SQL statements will be introduced: BACKUP SERVER TO '/path/to/directory' [ 1 CONCURRENT ]; BACKUP SERVER WITH [ 1 CONCURRENT ] 'command'; In place of the 1, any positive number of threads may be specified. For the first variant, '/path/to' must exist and '/path/to/directory' must be compatible with secure_file_priv and not exist; that is where the backup will be written to. For the second variant, 'command' must be the name of a script or command that will be executed in a child process. The standard input of that command will be in a format that is compatible with GNU tar --format=oldgnu (and also BSD tar variants that are also part of Microsoft Windows and Apple macOS). The command is expected to optionally compress and encrypt the stream and redirect it to a file on a local or a remote server. The BACKUP SERVER WITH will append an additional argument, a positive base-ten number in ASCII, starting with 1, to identify the current thread. In this way, each concurrent stream can write a separate file. The backup or the first stream will contain a file backup.cnf, which includes parameters needed for restoring the backup. Currently, these are innodb_log_recovery_start and innodb_log_recovery_target. If innodb_log_recovery_target>0, InnoDB will be in read-only mode, not allowing any writes to persistent files other than via the log application. To restore a streaming backup made with BACKUP SERVER WITH, an empty directory needs to be created and all streams be extracted there using the standard tar utility of the operating system, optionally after undoing any encryption or compression that had been added by the backup command. Then, the backup is prepared or MariaDB server started up on the extracted directory, similar to as if the BACKUP SERVER TO statement had been used. Note: The parameter innodb_log_recovery_start in backup.cnf is STRICTLY NECESSARY TO AVOID CORRUPTION! By default, InnoDB crash recovery starts from the latest available log checkpoint. However, for restoring a backup, recovery must start from the checkpoint that was the latest when the backup was started. Starting recovery from a possible later checkpoint will result in a corrupted database! The following will be implemented separately: MDEV-39061 mariadb-backup compatible wrapper script for BACKUP SERVER MDEV-40163 Partial backup and restore MDEV-39091 Back up ENGINE=RocksDB MDEV-39092 Less blocking backup of ENGINE=Aria The implementation introduces a basic driver Sql_cmd_backup, storage engine interfaces, and basic copying of the storage engines InnoDB, Aria, MyISAM, MERGE (MyISAM), Archive, CSV. aria_backup_end(): A crude prototype that copies non-InnoDB files. Scans and copies the data directories in a single thread, while everything is locked. This will be refactored in MDEV-39092. backup_target: A structured data type to represent a target directory. On Microsoft Windows, we must use directory paths because there is no variant of CopyFileEx() that would work on file handles. backup_sink: Wraps a per-thread output stream as well as storage engine specific context. handlerton::backup_start(), handlerton::backup_end(): Invoked at the start or end of a backup phase, in the thread that executes a BACKUP SERVER statement. handlerton::backup_step(): A backup step that can be invoked from multiple threads concurrently, between the execution of the corresponding handlerton::backup_start() and handlerton::backup_end() of the same phase. copy_entire_file(): A file copying service for POSIX systems. copy_mmap(): A zero-copy alternative to backup::copy(), to copy from a memory-mapped buffer. copy_file_range_try(): A wrapper for Linux copy_file_range(2), which may fail with EOPNOTSUPP or EXDEV and thus require a fallback to copy_mmap() or backup::copy(). backup::copy(): A partial or sparse file-copying service. On other platforms than FreeBSD or Microsoft Windows, there are shortcut alternatives to this. Note: On Linux we never invoke sendfile(2) for copying between files, because can be much slower than the alternatives. backup_stream_append_plain(): Equivalent to backup::copy(), but appending to a stream. On Linux, this uses sendfile(2), which assumes that the source data will not be changed before the data has been consumed from the pipe. backup_stream_append_async(): A variant of backup_stream_append_plain() where the source file region is guaranteed to be immutable after the call returns. We must not use zero-copy mmap(2) or Linux sendfile(2) for copying data files that may be modified in place, because it could introduce a race condition between a page write that runs concurrently with a child process that is reading the data from the pipe. backup_stream_zeropad(): Zero-pad the last tar block if needed. InnoDB_backup::context: Backup context, attached to backup_sink so that context can continue to exist between the time a BACKUP SERVER releases all locks and another BACKUP SERVER starts executing, with innodb_backup pointing to the new backup, while the old backup is still being finished. InnoDB_backup::queue: Collection of tablespace IDs and payload sizes at the start of the backup, and the log_sys.first_lsn of log files that have to be included in the backup. If any data file is created or extended while the backup is executing, we must have the corresponding write-ahead-log entries that we are copying since the latest checkpoint that was completed when the backup started. If any tablespaces are deleted during the backup, we may or may not copy them, and the application of a FILE_DELETE record will remove them. Similarly, applying FILE_RENAME or FILE_CREATE records will rename or create files during recovery as needed. log_sys.backup: Whether BACKUP SERVER is in progress. The purpose of this is to make BACKUP SERVER prevent the concurrent execution of SET GLOBAL innodb_log_archive=OFF or SET GLOBAL innodb_log_file_size when innodb_log_archive=OFF. log_sys.archived_checkpoint: Keep track of the earliest available checkpoint, corresponding to log_sys.archived_lsn. This reflects SET GLOBAL innodb_log_recovery_start (which is settable now), for incremental backup. fil_system.have_all_spaces: Whether all tablespace metadata is guaranteed to be known. To speed up startup, InnoDB does not normally open all tablespace files. fil_space_t::create_lsn: Change to Atomic_relaxed and use this to indicate tablespace creation LSN, in addition to indicate undo tablespace rebuild LSN. fil_space_t::backup_end: The first page number that is not being backed up (by default 0, to indicate that no backup is in progress). fil_space_t::BACKUP_BATCH_SIZE: The number of preceding pages that will be covered by fil_space_t::backup_end. This is the unit of "page range locking" during InnoDB backup. buf_page_t::write_fix_try(), buf_page_t::write_unfix_try(): Try to set or unset a fake "write fix" on a page, to prevent concurrent flush() during a backup batch. The atomic operations may run concurrently with set_reinit() and set_freed(). The fake "write fix" does not prevent any concurrent read or write of the page data in the buffer pool; it only blocks writes to the underlying data file. buf_page_t::flush(): Atomically test and set write fix, and skip the operation if the fake "write fix" was set. buf_page_t::set_freed(), buf_page_t::set_reinit(): Employ a compare-and-exchange loop to accommodate for the "write fix". innodb_backup_batch_wait(): Look up any pages that we are about to back up. For any dirty pages, invoke buf_page_t::write_fix_try() to try to set a fake "write fix" lock-free. If the page is currently write-fixed between buf_page_t::flush() and buf_page_t::write_complete(), acquire and release a page U-latch to wait for the conflicting write to complete. InnoDB_backup::backup_batch_start(), InnoDB_backup::backup_batch_stop(): Adjust fil_space_t::backup_end and fake "write fix" of dirty pages to protect the copying of a range of pages from the underlying file. log_t::backup_start(): If we were running with innodb_log_archive=ON, ensure that the latest file is a valid recovery starting point. That is, wait for the latest log checkpoint to be within the file. buf_flush_list_space(): Check for concurrent backup before writing each page. This is inefficient, but this function may be invoked from multiple threads concurrently, and it cannot be changed easily, especially for fil_crypt_thread(). fil_ibd_create(): Set fil_space_t::create_lsn after the file has been created. dict_load_tablespaces(): Determine the size of each file if upgrade==true. Backup depends on that. buf_dblwr_t::begin(), buf_dblwr_t::end(), buf_dblwr_t::size(): Accessors to allow BACKUP SERVER to skip the contents of the doublewrite buffer in the system tablespace. It is only useful for crash recovery in case a data page had been incompletely written by the time the server was killed. If the server is killed during a backup, the backup will be incomplete and unusable anyway. Furthermore, the page range locking makes page writes and backup mutually exclusive. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Deprecated warning fix Two compatibility warnings are meant for end users but only add noise to debug/test builds: mysqld is commonly started through a legacy "mysqld" symlink, and test .cnf files still reference long-removed options. Emit them in release builds only. - mysys/my_init.c: the deprecated-invocation-name warning (EE_NAME_DEPRECATED), shown when the program is run via a non-"mariadb" symlink to a "mariadb*" binary -- now compiled under #ifdef NDEBUG. - sql/mysqld.cc: the "'<opt>' was removed ... exists only for compatibility" warning for OPT_REMOVED_OPTION -- now under #ifdef DBUG_OFF. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 extra2_fields refactored to Extra2_info class read_extra2() is now Extra2_info::read() Additional assertions for checking size consistency. Extra2_info::write() is used by further MDEV-20865 development. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Deprecated warning fix Two compatibility warnings are meant for end users but only add noise to debug/test builds: mysqld is commonly started through a legacy "mysqld" symlink, and test .cnf files still reference long-removed options. Emit them in release builds only. - mysys/my_init.c: the deprecated-invocation-name warning (EE_NAME_DEPRECATED), shown when the program is run via a non-"mariadb" symlink to a "mariadb*" binary -- now compiled under #ifdef NDEBUG. - sql/mysqld.cc: the "'<opt>' was removed ... exists only for compatibility" warning for OPT_REMOVED_OPTION -- now under #ifdef DBUG_OFF. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 build_frm_image() readability cleanup More clear names and constants use. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 extra2_fields refactored to Extra2_info class read_extra2() is now Extra2_info::read() Additional assertions for checking size consistency. Extra2_info::write() is used by further MDEV-20865 development. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
gcov fix Fix coverage data corruption by enabling atomic profile updates When compiling with coverage instrumentation, concurrent or forked test executions can cause corruption of coverage data files (.gcda), resulting in negative hit counts and invalid coverage reports. Adding the compiler flag `-fprofile-update=atomic` ensures that updates to coverage counters are performed atomically, preventing race conditions during profile data writes. This change eliminates warnings about unexpected negative hit counts and improves the accuracy and reliability of coverage measurements in parallel or multithreaded test environments. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-14992 BACKUP SERVER The following SQL statements will be introduced: BACKUP SERVER TO '/path/to/directory' [ 1 CONCURRENT ]; BACKUP SERVER WITH [ 1 CONCURRENT ] 'command'; In place of the 1, any positive number of threads may be specified. For the first variant, '/path/to' must exist and '/path/to/directory' must be compatible with secure_file_priv and not exist; that is where the backup will be written to. For the second variant, 'command' must be the name of a script or command that will be executed in a child process. The standard input of that command will be in a format that is compatible with GNU tar --format=oldgnu (and also BSD tar variants that are also part of Microsoft Windows and Apple macOS). The command is expected to optionally compress and encrypt the stream and redirect it to a file on a local or a remote server. The BACKUP SERVER WITH will append an additional argument, a positive base-ten number in ASCII, starting with 1, to identify the current thread. In this way, each concurrent stream can write a separate file. The backup or the first stream will contain a file backup.cnf, which includes parameters needed for restoring the backup. Currently, these are innodb_log_recovery_start and innodb_log_recovery_target. If innodb_log_recovery_target>0, InnoDB will be in read-only mode, not allowing any writes to persistent files other than via the log application. To restore a streaming backup made with BACKUP SERVER WITH, an empty directory needs to be created and all streams be extracted there using the standard tar utility of the operating system, optionally after undoing any encryption or compression that had been added by the backup command. Then, the backup is prepared or MariaDB server started up on the extracted directory, similar to as if the BACKUP SERVER TO statement had been used. Note: The parameter innodb_log_recovery_start in backup.cnf is STRICTLY NECESSARY TO AVOID CORRUPTION! By default, InnoDB crash recovery starts from the latest available log checkpoint. However, for restoring a backup, recovery must start from the checkpoint that was the latest when the backup was started. Starting recovery from a possible later checkpoint will result in a corrupted database! The following will be implemented separately: MDEV-39061 mariadb-backup compatible wrapper script for BACKUP SERVER MDEV-40163 Partial backup and restore MDEV-39091 Back up ENGINE=RocksDB MDEV-39092 Less blocking backup of ENGINE=Aria The implementation introduces a basic driver Sql_cmd_backup, storage engine interfaces, and basic copying of the storage engines InnoDB, Aria, MyISAM, MERGE (MyISAM), Archive, CSV. aria_backup_end(): A crude prototype that copies non-InnoDB files. Scans and copies the data directories in a single thread, while everything is locked. This will be refactored in MDEV-39092. backup_target: A structured data type to represent a target directory. On Microsoft Windows, we must use directory paths because there is no variant of CopyFileEx() that would work on file handles. backup_sink: Wraps a per-thread output stream as well as storage engine specific context. handlerton::backup_start(), handlerton::backup_end(): Invoked at the start or end of a backup phase, in the thread that executes a BACKUP SERVER statement. handlerton::backup_step(): A backup step that can be invoked from multiple threads concurrently, between the execution of the corresponding handlerton::backup_start() and handlerton::backup_end() of the same phase. copy_entire_file(): A file copying service for POSIX systems. copy_mmap(): A zero-copy alternative to backup::copy(), to copy from a memory-mapped buffer. copy_file_range_try(): A wrapper for Linux copy_file_range(2), which may fail with EOPNOTSUPP or EXDEV and thus require a fallback to copy_mmap() or backup::copy(). backup::copy(): A partial or sparse file-copying service. On other platforms than FreeBSD or Microsoft Windows, there are shortcut alternatives to this. Note: On Linux we never invoke sendfile(2) for copying between files, because can be much slower than the alternatives. backup_stream_append_plain(): Equivalent to backup::copy(), but appending to a stream. On Linux, this uses sendfile(2), which assumes that the source data will not be changed before the data has been consumed from the pipe. backup_stream_append_async(): A variant of backup_stream_append_plain() where the source file region is guaranteed to be immutable after the call returns. We must not use zero-copy mmap(2) or Linux sendfile(2) for copying data files that may be modified in place, because it could introduce a race condition between a page write that runs concurrently with a child process that is reading the data from the pipe. backup_stream_zeropad(): Zero-pad the last tar block if needed. InnoDB_backup::context: Backup context, attached to backup_sink so that context can continue to exist between the time a BACKUP SERVER releases all locks and another BACKUP SERVER starts executing, with innodb_backup pointing to the new backup, while the old backup is still being finished. InnoDB_backup::queue: Collection of tablespace IDs and payload sizes at the start of the backup, and the log_sys.first_lsn of log files that have to be included in the backup. If any data file is created or extended while the backup is executing, we must have the corresponding write-ahead-log entries that we are copying since the latest checkpoint that was completed when the backup started. If any tablespaces are deleted during the backup, we may or may not copy them, and the application of a FILE_DELETE record will remove them. Similarly, applying FILE_RENAME or FILE_CREATE records will rename or create files during recovery as needed. log_sys.backup: Whether BACKUP SERVER is in progress. The purpose of this is to make BACKUP SERVER prevent the concurrent execution of SET GLOBAL innodb_log_archive=OFF or SET GLOBAL innodb_log_file_size when innodb_log_archive=OFF. log_sys.archived_checkpoint: Keep track of the earliest available checkpoint, corresponding to log_sys.archived_lsn. This reflects SET GLOBAL innodb_log_recovery_start (which is settable now), for incremental backup. fil_system.have_all_spaces: Whether all tablespace metadata is guaranteed to be known. To speed up startup, InnoDB does not normally open all tablespace files. fil_space_t::create_lsn: Change to Atomic_relaxed and use this to indicate tablespace creation LSN, in addition to indicate undo tablespace rebuild LSN. fil_space_t::backup_end: The first page number that is not being backed up (by default 0, to indicate that no backup is in progress). fil_space_t::BACKUP_BATCH_SIZE: The number of preceding pages that will be covered by fil_space_t::backup_end. This is the unit of "page range locking" during InnoDB backup. buf_page_t::write_fix_try(), buf_page_t::write_unfix_try(): Try to set or unset a fake "write fix" on a page, to prevent concurrent flush() during a backup batch. The atomic operations may run concurrently with set_reinit() and set_freed(). The fake "write fix" does not prevent any concurrent read or write of the page data in the buffer pool; it only blocks writes to the underlying data file. buf_page_t::flush(): Atomically test and set write fix, and skip the operation if the fake "write fix" was set. buf_page_t::set_freed(), buf_page_t::set_reinit(): Employ a compare-and-exchange loop to accommodate for the "write fix". innodb_backup_batch_wait(): Look up any pages that we are about to back up. For any dirty pages, invoke buf_page_t::write_fix_try() to try to set a fake "write fix" lock-free. If the page is currently write-fixed between buf_page_t::flush() and buf_page_t::write_complete(), acquire and release a page U-latch to wait for the conflicting write to complete. InnoDB_backup::backup_batch_start(), InnoDB_backup::backup_batch_stop(): Adjust fil_space_t::backup_end and fake "write fix" of dirty pages to protect the copying of a range of pages from the underlying file. log_t::backup_start(): If we were running with innodb_log_archive=ON, ensure that the latest file is a valid recovery starting point. That is, wait for the latest log checkpoint to be within the file. buf_flush_list_space(): Check for concurrent backup before writing each page. This is inefficient, but this function may be invoked from multiple threads concurrently, and it cannot be changed easily, especially for fil_crypt_thread(). fil_ibd_create(): Set fil_space_t::create_lsn after the file has been created. dict_load_tablespaces(): Determine the size of each file if upgrade==true. Backup depends on that. buf_dblwr_t::begin(), buf_dblwr_t::end(), buf_dblwr_t::size(): Accessors to allow BACKUP SERVER to skip the contents of the doublewrite buffer in the system tablespace. It is only useful for crash recovery in case a data page had been incompletely written by the time the server was killed. If the server is killed during a backup, the backup will be incomplete and unusable anyway. Furthermore, the page range locking makes page writes and backup mutually exclusive. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 Fix tests and system scripts to foreign key requirements Storing foreign key metadata in TABLE_SHARE makes opening a foreign table preopen its referenced tables, so a referenced table must be created before the foreign one and dropped after it. Rearrange the order of CREATE/DROP commands in tests and system scripts accordingly. Affected: the help tables in mariadb_system_tables{,_fix}.sql (with system_mysql_db_fix* results), fetch_first, union, insert_notembedded, instant_alter_index_rename and opt_context_store_ddls; some also index a referenced column or add a missing referenced table, and opt_context_store_ddls re-records the now-shown MyISAM foreign key. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40728 Recovery wrongly fails if FILE_CREATE is followed by FILE_RENAME deferred_spaces.deferred_dblwr(): Skip newly created tablespaces to avoid a bogus invocation of fil_space_free(). fil_name_process(): Simplify the logic. If no matching tablespace is found but file_name_t::create_lsn had been set in response to parsing a FILE_CREATE record, try to apply FILE_RENAME to deferred_spaces. log_parse_file(): Parse each FILE_ record only once. In multi-batch recovery, there may be redundant calls that would break the logic of fil_name_process(). fil_delete_apply(): Wrappers for fil_space_free(). When recovering a log in innodb_log_archive=ON format, we must apply FILE_DELETE records in order to avoid a future clash with FILE_CREATE or FILE_RENAME. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Deprecated warning fix Two compatibility warnings are meant for end users but only add noise to debug/test builds: mysqld is commonly started through a legacy "mysqld" symlink, and test .cnf files still reference long-removed options. Emit them in release builds only. - mysys/my_init.c: the deprecated-invocation-name warning (EE_NAME_DEPRECATED), shown when the program is run via a non-"mariadb" symlink to a "mariadb*" binary -- now compiled under #ifdef NDEBUG. - sql/mysqld.cc: the "'<opt>' was removed ... exists only for compatibility" warning for OPT_REMOVED_OPTION -- now under #ifdef DBUG_OFF. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 build_frm_image() readability cleanup More clear names and constants use. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 extra2 structures moved to datadict.h datadict.h is appropriate place for such types. These data types are used by Extra2_info, which is added to datadict.h later. unireg.cc is for older FRM routines. datadict.cc accepts new and refactored routines. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40756 Incorrect multi-batch recovery of file size file_name_t::page0_lsn: Keep track of the last applied recv_sys_t::parse_page0() so that a multi-batch recovery will not reset the file to a smaller size. Reviewed by: Thirunarayanan Balathandayuthapani (cherry picked from commit 8f00e6caca633c783140db86d3a48a96de67cf38) |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 extra2_write_len() fix Current implementation of extra2_write_len() does not guarantee of writing correct 2-byte values as it skips writing zero at the beginning. Refined DBUG_ASSERT() to more truthful limit. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 Prelocking_strategy refactoring - Moved has_prelocking_list into Prelocking_strategy (removed it from Multiupdate_prelocking_strategy). It is now initialized in Prelocking_strategy::reset(), so reset() overrides (Multiupdate_prelocking_strategy, Multidelete_prelocking_strategy) chain to the base implementation to set it. - Moved extend_table_list() into Prelocking_strategy; - Made maybe_need_prelocking() virtual method. Now maybe_need_prelocking() can be different per prelocking strategy. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 extra2_write_len() fix Current implementation of extra2_write_len() does not guarantee of writing correct 2-byte values as it skips writing zero at the beginning. Refined DBUG_ASSERT() to more truthful limit. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thirunarayanan Balathandayuthapani
thiru@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-19574: innodb_stats_method is not honored when innodb_stats_persistent=ON Problem: ======= When persistent statistics are enabled (innodb_stats_persistent=ON), the innodb_stats_method setting is not properly utilized during statistics calculation. The statistics collection functions always use a hardcoded default behavior for NULL value comparison instead of respecting the configured stats method. This affects the accuracy of n_diff_key_vals (distinct key count), particularly for indexes with nullable columns containing NULL values. Moreover, stat_n_non_null_key_vals[] was never computed for persistent statistics; it stayed at the 0 that dict_stats_empty_index() assigns. With innodb_stats_method=nulls_ignored, innodb_rec_per_key() therefore always found n_diff <= n_null and reported one record per key for every index. This impacts the query optimizer, which makes decisions based on inaccurate cardinality estimates. Solution: ======== Introduced IndexLevelStats to collect statistics at a specific B-tree level during index analysis. Introduced PageStats to collect statistics for leaf page analysis. Refactored the following functions: dict_stats_analyze_index_level() to IndexLevelStats::analyze_level() dict_stats_analyze_index_for_n_prefix() to IndexLevelStats::sample_leaf_pages() dict_stats_analyze_index_below_cur() to PageStats::scan_below() dict_stats_scan_page() to PageStats::scan() The innodb_stats_method value is read once per table in dict_stats_update_persistent() and passed down, so that all indexes of a table are analyzed with the same method. Add the stats method name to stat_description when innodb_stats_method has a non-default value. The suffix is dropped when the description is already full. Added the new stat name n_nonnull_fld01, n_nonnull_fld02, etc. with a stats description, to indicate how many non-null values exist for the nth field of the index. This value is retrieved and stored in the index statistics in dict_stats_fetch_index_stats_step(). The counts are per column, not per n-column prefix. rec_get_n_blob_pages(): Calculate the number of externally stored pages for a record, using ceiling division by the usable BLOB page payload (blob_part_size), which differs between ROW_FORMAT=COMPRESSED (zip_size minus FIL_PAGE_DATA) and the other formats (srv_page_size minus the BLOB header and the page trailer). For ROW_FORMAT=COMPRESSED the length in the field reference is the uncompressed length, so the result is an upper bound. When the leaf level is scanned in full, the number of leaf pages that were scanned is reported as n_leaf_pages for a multi level index. Before, result.n_leaf_pages was overwritten with index->stat_n_leaf_pages, which dict_stats_empty_index() had just set to 1, so every index that took the full scan path reported n_leaf_pages=1. Single page indexes report 1. This changes cardinality estimates and therefore leads to multiple changes in existing test cases. Non-null values are counted only at the leaf level, since only leaf pages hold actual records. A full scan of the leaf level counts them exactly. When the level is sampled, the per column count is derived from the sampled leaves with the same formula as n_diff: n_ordinary_leaf_pages * n_non_null_all_analyzed_pages / n_leaf_pages_to_analyze This is an estimate for NOT NULL columns as well: the sampled leaves may hold fewer or more records than the average, and a dive that stops at a boring page contributes nothing to the sum while still counting in the divisor. innodb_rec_per_key(): stat_n_non_null_key_vals[i] holds the number of records in which the i-th indexed column alone is not NULL, while what has to be excluded here is the number of records whose first i+1 columns are all not NULL, because that is the population which the n-column prefix statistic stat_n_diff_key_vals[i] has to be corrected against when innodb_stats_method=nulls_ignored: with NULLs compared as unequal, every record carrying a NULL anywhere in the prefix adds a distinct value of its own to n_diff. PageStats::scan(): n_non_null is accumulated and assigned only for leaf pages, so that a non-leaf scan cannot leave a node pointer count behind when scan_below() stops at a boring page without reaching a leaf. IndexLevelStats::reset_for_level() also clears n_diff[], and dict_stats_analyze_index() zero initializes the buffer backing it, so that a level scan which finds no records (a failed btr_pcur_open_level(), or a non-leaf page whose first record is not marked as the leftmost one on the level) leaves n_diff[] at 0 instead of stale values. IndexLevelStats::sample_leaf_pages() returns early when the group boundaries for the prefix are empty, which is the same condition. dict_stats_fetch_index_stats_step() no longer resets stat_n_non_null_key_vals[] while processing an n_diff_pfxNN row: dict_stats_empty_table() has already cleared the array before the fetch, and with n_nonnull_fldNN rows now being read too, that reset would make the result depend on the order in which the rows arrive. dict_stats_save(): now static function in dict0stats.cc that takes the innodb_stats_method value, and is removed from dict0stats.h. dict_stats_update_persistent() saves the statistics itself, so its callers no longer have to. Replaced btr_rec_get_externally_stored_len() with rec_get_n_blob_pages() in dict0stats.cc. btr_rec_get_field_ref_offs() and btr_rec_get_field_ref(), together with the BTR_BLOB_HDR_* macros, were moved from btr0cur.cc to btr0cur.h so that rec_get_n_blob_pages() can reuse them; btr_rec_get_field_ref_offs() is now a noexcept function returning size_t. Changed stat_n_diff_key_vals and stat_n_non_null_key_vals from ib_uint64_t* to uint64_t* len_is_stored(): simplified to a single comparison, which is equivalent for the unsigned lengths that it is used with. Removed the unused UNIV_STATS_DEBUG build macro (univ.i) and turned the DEBUG_PRINTF() helper in dict0stats.cc into an unconditional no-op |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 FRM image functions cleanups - build_frm_image(): clearer names and named constants - build_frm_image(): guard against overflow of the 2-byte key-info offset - engine_table_options_frm_image(): mark in/out argument directions |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
sjaakola
seppo.jaakola@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38869 sequence conflicts with streaming replication Sequence access conflicts with streaming replication could cause server hanging, as shown in MDEV-38869 This commit avoids such deadlocks, by skipping fragment replication for sequence access. The commit has also new mtr test for testing two sequence/SR conflict scenarios: galera.galera_sequences_bf_kill_sr |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 Prelocking_strategy refactoring - Moved has_prelocking_list into Prelocking_strategy (removed it from Multiupdate_prelocking_strategy). It is now initialized in Prelocking_strategy::reset(), so reset() overrides (Multiupdate_prelocking_strategy, Multidelete_prelocking_strategy) chain to the base implementation to set it. - Moved extend_table_list() into Prelocking_strategy; - Made maybe_need_prelocking() virtual method. Now maybe_need_prelocking() can be different per prelocking strategy. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 extra2 structures moved to datadict.h datadict.h is appropriate place for such types. These data types are used by Extra2_info, which is added to datadict.h later. unireg.cc is for older FRM routines. datadict.cc accepts new and refactored routines. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thirunarayanan Balathandayuthapani
thiru@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-19574: innodb_stats_method is not honored when innodb_stats_persistent=ON Problem: ======= When persistent statistics are enabled (innodb_stats_persistent=ON), the innodb_stats_method setting is not properly utilized during statistics calculation. The statistics collection functions always use a hardcoded default behavior for NULL value comparison instead of respecting the configured stats method. This affects the accuracy of n_diff_key_vals (distinct key count), particularly for indexes with nullable columns containing NULL values. Moreover, stat_n_non_null_key_vals[] was never computed for persistent statistics; it stayed at the 0 that dict_stats_empty_index() assigns. With innodb_stats_method=nulls_ignored, innodb_rec_per_key() therefore always found n_diff <= n_null and reported one record per key for every index. This impacts the query optimizer, which makes decisions based on inaccurate cardinality estimates. Solution: ======== Introduced IndexLevelStats to collect statistics at a specific B-tree level during index analysis. Introduced PageStats to collect statistics for leaf page analysis. Refactored the following functions: dict_stats_analyze_index_level() to IndexLevelStats::analyze_level() dict_stats_analyze_index_for_n_prefix() to IndexLevelStats::sample_leaf_pages() dict_stats_analyze_index_below_cur() to PageStats::scan_below() dict_stats_scan_page() to PageStats::scan() The innodb_stats_method value is read once per table in dict_stats_update_persistent() and passed down, so that all indexes of a table are analyzed with the same method. Add the stats method name to stat_description when innodb_stats_method has a non-default value. The suffix is dropped when the description is already full. Added the new stat name n_nonnull_fld01, n_nonnull_fld02, etc. with a stats description, to indicate how many non-null values exist for the nth field of the index. This value is retrieved and stored in the index statistics in dict_stats_fetch_index_stats_step(). The counts are per column, not per n-column prefix. rec_get_n_blob_pages(): Calculate the number of externally stored pages for a record, using ceiling division by the usable BLOB page payload (blob_part_size), which differs between ROW_FORMAT=COMPRESSED (zip_size minus FIL_PAGE_DATA) and the other formats (srv_page_size minus the BLOB header and the page trailer). For ROW_FORMAT=COMPRESSED the length in the field reference is the uncompressed length, so the result is an upper bound. When the leaf level is scanned in full, the number of leaf pages that were scanned is reported as n_leaf_pages for a multi level index. Before, result.n_leaf_pages was overwritten with index->stat_n_leaf_pages, which dict_stats_empty_index() had just set to 1, so every index that took the full scan path reported n_leaf_pages=1. Single page indexes report 1. This changes cardinality estimates and therefore leads to multiple changes in existing test cases. Non-null values are counted only at the leaf level, since only leaf pages hold actual records. A full scan of the leaf level counts them exactly. When the level is sampled, the per column count is derived from the sampled leaves with the same formula as n_diff: n_ordinary_leaf_pages * n_non_null_all_analyzed_pages / n_leaf_pages_to_analyze This is an estimate for NOT NULL columns as well: the sampled leaves may hold fewer or more records than the average, and a dive that stops at a boring page contributes nothing to the sum while still counting in the divisor. innodb_rec_per_key(): stat_n_non_null_key_vals[i] holds the number of records in which the i-th indexed column alone is not NULL, while what has to be excluded here is the number of records whose first i+1 columns are all not NULL, because that is the population which the n-column prefix statistic stat_n_diff_key_vals[i] has to be corrected against when innodb_stats_method=nulls_ignored: with NULLs compared as unequal, every record carrying a NULL anywhere in the prefix adds a distinct value of its own to n_diff. PageStats::scan(): n_non_null is accumulated and assigned only for leaf pages, so that a non-leaf scan cannot leave a node pointer count behind when scan_below() stops at a boring page without reaching a leaf. IndexLevelStats::reset_for_level() also clears n_diff[], and dict_stats_analyze_index() zero initializes the buffer backing it, so that a level scan which finds no records (a failed btr_pcur_open_level(), or a non-leaf page whose first record is not marked as the leftmost one on the level) leaves n_diff[] at 0 instead of stale values. IndexLevelStats::sample_leaf_pages() returns early when the group boundaries for the prefix are empty, which is the same condition. dict_stats_fetch_index_stats_step() no longer resets stat_n_non_null_key_vals[] while processing an n_diff_pfxNN row: dict_stats_empty_table() has already cleared the array before the fetch, and with n_nonnull_fldNN rows now being read too, that reset would make the result depend on the order in which the rows arrive. dict_stats_save(): now static function in dict0stats.cc that takes the innodb_stats_method value, and is removed from dict0stats.h. dict_stats_update_persistent() saves the statistics itself, so its callers no longer have to. Replaced btr_rec_get_externally_stored_len() with rec_get_n_blob_pages() in dict0stats.cc. btr_rec_get_field_ref_offs() and btr_rec_get_field_ref(), together with the BTR_BLOB_HDR_* macros, were moved from btr0cur.cc to btr0cur.h so that rec_get_n_blob_pages() can reuse them; btr_rec_get_field_ref_offs() is now a noexcept function returning size_t. Changed stat_n_diff_key_vals and stat_n_non_null_key_vals from ib_uint64_t* to uint64_t* len_is_stored(): simplified to a single comparison, which is equivalent for the unsigned lengths that it is used with. Removed the unused UNIV_STATS_DEBUG build macro (univ.i) and turned the DEBUG_PRINTF() helper in dict0stats.cc into an unconditional no-op |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 Prelocking_strategy refactoring - Moved has_prelocking_list into Prelocking_strategy (removed it from Multiupdate_prelocking_strategy). It is now initialized in Prelocking_strategy::reset(), so reset() overrides (Multiupdate_prelocking_strategy, Multidelete_prelocking_strategy) chain to the base implementation to set it. - Moved extend_table_list() into Prelocking_strategy; - Made maybe_need_prelocking() virtual method. Now maybe_need_prelocking() can be different per prelocking strategy. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 Fix tests and system scripts to foreign key requirements Storing foreign key metadata in TABLE_SHARE makes opening a foreign table preopen its referenced tables, so a referenced table must be created before the foreign one and dropped after it. Rearrange the order of CREATE/DROP commands in tests and system scripts accordingly. Affected: the help tables in mariadb_system_tables{,_fix}.sql (with system_mysql_db_fix* results), fetch_first, union, insert_notembedded, instant_alter_index_rename and opt_context_store_ddls; some also index a referenced column or add a missing referenced table, and opt_context_store_ddls re-records the now-shown MyISAM foreign key. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 extra2_fields refactored to Extra2_info class read_extra2() is now Extra2_info::read() Additional assertions for checking size consistency. Extra2_info::write() is used by further MDEV-20865 development. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
gcov fix Fix coverage data corruption by enabling atomic profile updates When compiling with coverage instrumentation, concurrent or forked test executions can cause corruption of coverage data files (.gcda), resulting in negative hit counts and invalid coverage reports. Adding the compiler flag `-fprofile-update=atomic` ensures that updates to coverage counters are performed atomically, preventing race conditions during profile data writes. This change eliminates warnings about unexpected negative hit counts and improves the accuracy and reliability of coverage measurements in parallel or multithreaded test environments. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexander Barkov
bar@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39518 Allow prepared statements in stored functions in assignment right hand Allowing prepared statements in stored functions when a stored function is used in an assignment right hand. Both DEFAULT clause of a variable initialization and the right side of the SET statement are supported: CREATE PROCEDURE p1() BEGIN -- case 1: DEFAULT clause DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK -- case 2: SP variable assignment statement DECLARE spvar2 INT; SET spvar= f1_with_ps(); -- OK END; - The parser now does not reject PS statements in stored functions. PS applicability in stored functions is now detected at run time. Note, PS statements in triggers are still prohibited by the parser. - Functions with PS do not acquire MDL locks on tables. They work like procedures in terms of table opening. - Functions with PS are not replicated as a single `SELECT f1()` call. They are replicated per-statement, like procedures. - Only bare function calls are supported for now. Using a function in an expression does not make it PS-safe yet yet: SET v= f1()+0; |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 extra2_write_len() fix Current implementation of extra2_write_len() does not guarantee of writing correct 2-byte values as it skips writing zero at the beginning. Refined DBUG_ASSERT() to more truthful limit. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40728 Recovery wrongly fails if FILE_CREATE is followed by FILE_RENAME deferred_spaces.deferred_dblwr(): Skip newly created tablespaces to avoid a bogus invocation of fil_space_free(). fil_name_process(): Simplify the logic. If no matching tablespace is found but file_name_t::create_lsn had been set in response to parsing a FILE_CREATE record, try to apply FILE_RENAME to deferred_spaces. log_parse_file(): Parse each FILE_ record only once. In multi-batch recovery, there may be redundant calls that would break the logic of fil_name_process(). |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-22991 Support SSL over named pipes Update libmariadb, to remove named pipe SSL check. Update client.c to remove that check as well. Add MTR test for named pipe + SSL: - for Connector/C client, new test named_pipe_ssl - for in-server clent, run mariabackup with user created as "IDENTIFIED WITH named_pipe REQUIRE SSL" |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-22992 Refactor VIO into layered transports and filters Replace the function-pointer VIO implementation with an abstract C++ interface while retaining the procedural C entry points. Implement socket and named-pipe transports and composable filters for client read-ahead, Windows thread-pool prefetch, and TLS. OpenSSL uses a custom BIO, while wolfSSL uses callbacks that perform I/O through the VIO below the TLS filter. This keeps waits and timeouts in the transport layer. Keep sockets nonblocking and implement timed I/O with transport waits. Named pipes use overlapped I/O for timeout-aware waits and report blocking waits through the same scheduler callbacks as sockets. Semi-sync temporarily changes the real VIO read timeout instead of copying VIO state. Hide transport and TLS implementation state behind accessors. Expose the TLS handle opaquely and update callers that previously accessed VIO fields directly. Compile the VIO implementations as C++ and retain PSI memory accounting for VIO allocations. Adapt Windows thread-pool pre-read to a Prefetched_vio filter inserted above the transport so both plain and TLS connections consume prefetched bytes through the same layered VIO path. Cleaned header files so that vio header no longer include OpenSSL or WolfSSL headers. Removed some legacy functionality - vio_close() with its double-close guards appeared hard to maintain in class hierarchy, and had been unnecessary for the last 15 years. Associated things that are also gone : preprocessor definition SIGNAL_WITH_VIO_CLOSE (always defined), VIO_STATE_CLOSED. VIO_CLOSED type, which was used as sentinel, renamed to VIO_TYPE_INVALID. - vio_io_wait() used in a single place, replaces by read with timeout. - vio_reset() to create SSL, replaced by vio_wrap |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-20865 extra2 structures moved to datadict.h datadict.h is appropriate place for such types. These data types are used by Extra2_info, which is added to datadict.h later. unireg.cc is for older FRM routines. datadict.cc accepts new and refactored routines. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||