Console View

Categories: connectors experimental galera main

Legend:

Passed

Failed

Warnings

Failed Again

Running

Exception

Offline

No data

connectors

experimental

galera

main

df06d8c63462...

df06d8c634625555f6c31bd4f2028dbf8e60d76c

Yuchen Pei

ycp@mariadb.com

MDEV-38179 [wip] Construct an OK packet string at server start for session variable tracking

d5115614f5d2...

d5115614f5d251c89bf3973a559d537aaa621628

Michael Widenius

monty@mariadb.org

MDEV-32745 Add a simple MySQL to MariaDB upgrade helper

The tool is named mariadb-migrate-config-file.
The main purpose of the tool is to change MySQL option
files to work both for MySQL and MariaDB.
There are options to do the changes in the options file inline,
or at-end-of-file. One can also remove or comment unknown options.

The list of supported options is generated compile time from
mariadbd --help. All server options, including compiled plugins, are
supported.

The bulk of the code comes from Väinö.
Monty has updated it with a lot of extra options.
Wlad helped with cmake integration

Other things:
- Fixed a memory leak in sql_plugin.cc
- plugin-load will now in case of errors try to load all given plugins
before aborted
- If silent-startup is used, plugin-load will not give errors for
plugins it cannot load or warnings about plugin marturity level.
- my_rm_tree() will now delete symlinks, not the actual file, if
MY_NOSYMLINK flag is used.
- my_stat() will now give data for symlink if MY_NOSYMLINKS is used.
- Added 'number of lines' option to mysqltest --cat_file

@Authors: Väinö Mäkelä <[email protected]>,[email protected]

c6ec3ddfc5a6...

c6ec3ddfc5a6b8794286ddf2a69820fe5585f704

Oleg Smirnov

olernov@gmail.com

MDEV-38045 Fix type conversion errors

c3f4efca3be4...

c3f4efca3be464d20af13cc3192b83feb9c34071

Sergei Golubchik

serg@mariadb.org

more fixups

c39c9bf6c4f1...

c39c9bf6c4f1ba9a256174dfbdfad3140bce865b

Georgi (Joro) Kodinov

joro@mariadb.com

MDEV-39123: Replace sprintf with snprintf in the InnoDB binlog code

sprintf is deprecated (and triggers a deprecation warning) on some platforms,
saying it needs to be replaced with snprintf.
This is a good idea, since sprintf doesn't explicitly check the size of the buffer
it is printing to.
Fixed by adding an extra size argument to the helper function and making sure
this extra argument is passed down from the caller to snprintf.

bd98b9c211ff...

bd98b9c211ff8bcc3f99e42fc83da95955f0f6ba

Lawrin Novitsky

lawrin.novitsky@mariadb.com

Trick to fix tests on macos-arm

Looks like tests and DM use different alignment and because of that 10th
paramemter of SQLSpesialColumns of type short gets read incorrectly by
the DM

b69cb60a83d1...

b69cb60a83d1da736f8594ecc11583430d3aac34

Christian Hesse

mail@eworm.de

systemd: add missing quotes in examples

The line is commented, but let's have valid quoting anyway...

b64c778b0171...

b64c778b01710aece224a6f579a6cedb9a58b44d

Aleksey Midenkov

midenok@gmail.com

MDEV-25365 Cleanups

ae006216f57e...

ae006216f57ee1ec983ea49ec9c9b91ae1c269d8

Monty

monty@mariadb.org

MDEV-25292 Atomic CREATE OR REPLACE TABLE

Atomic CREATE OR REPLACE allows to keep an old table intact if the
command fails or during the crash. That is done by renaming the
original table to temporary name, as a backup and restoring it if the
CREATE fails. When the command is complete and logged the backup
table is deleted.

Atomic replace algorithm

Two DDL chains are used for CREATE OR REPLACE:
ddl_log_state_create (C) and ddl_log_state_rm (D).

1. (C) Log rename of ORIG to TMP table (Rename TMP to original).
2. Rename orignal to TMP.
3. (C) Log CREATE_TABLE_ACTION of ORIG (drops ORIG);
4. Do everything with ORIG (like insert data)
5. (D) Log drop of TMP
6. Write query to binlog (this marks (C) to be closed in
case of failure)
7. Execute drop of TMP through (D)
8. Close (C) and (D)

If there is a failure before 6) we revert the changes in (C)
Chain (D) is only executed if 6) succeded (C is closed on
crash recovery).

Foreign key errors will be found at the 1) stage.

Additional notes

- CREATE TABLE without REPLACE and temporary tables is not affected
by this commit.
set @@drop_before_create_or_replace=1 can be used to
get old behaviour where existing tables are dropped
in CREATE OR REPLACE.

- CREATE TABLE is reverted if binlogging the query fails.

- Engines having HTON_EXPENSIVE_RENAME flag set are not affected by
this commit. Conflicting tables marked with this flag will be
deleted with CREATE OR REPLACE.

- Replication execution is not affected by this commit.
- Replication will first drop the conflicting table and then
creating the new one.

- CREATE TABLE .. SELECT XID usage is fixed and now there is no need
to log DROP TABLE via DDL_CREATE_TABLE_PHASE_LOG (see comments in
do_postlock()). XID is now correctly updated so it disables
DDL_LOG_DROP_TABLE_ACTION. Note that binary log is flushed at the
final stage when the table is ready. So if we have XID in the
binary log we don't need to drop the table.

- Three variations of CREATE OR REPLACE handled:

1. CREATE OR REPLACE TABLE t1 (..);
2. CREATE OR REPLACE TABLE t1 LIKE t2;
3. CREATE OR REPLACE TABLE t1 SELECT ..;

- Test case uses 6 combinations for engines (aria, aria_notrans,
myisam, ib, lock_tables, expensive_rename) and 2 combinations for
binlog types (row, stmt). Combinations help to check differences
between the results. Error failures are tested for the above three
variations.

- expensive_rename tests CREATE OR REPLACE without atomic
replace. The effect should be the same as with the old behaviour
before this commit.

- Triggers mechanism is unaffected by this change. This is tested in
create_replace.test.

- LOCK TABLES is affected. Lock restoration must be done after new
table is created or TMP is renamed back to ORIG

- Moved ddl_log_complete() from send_eof() to finalize_ddl(). This
checkpoint was not executed before for normal CREATE TABLE but is
executed now.

- CREATE TABLE will now rollback also if writing to the binary
logging failed. See rpl_gtid_strict.test

backup ddl log changes

- In case of a successfull CREATE OR REPLACE we only log
the CREATE event, not the DROP TABLE event of the old table.

ddl_log.cc changes

ddl_log_execute_action() now properly return error conditions.
ddl_log_disable_entry() added to allow one to disable one entry.
The entry on disk is still reserved until ddl_log_complete() is
executed.

On XID usage

Like with all other atomic DDL operations XID is used to avoid
inconsistency between master and slave in the case of a crash after
binary log is written and before ddl_log_state_create is closed. On
recovery XIDs are taken from binary log and corresponding DDL log
events get disabled. That is done by
ddl_log_close_binlogged_events().

On linking two chains together

Chains are executed in the ascending order of entry_pos of execute
entries. But entry_pos assignment order is undefined: it may assign
bigger number for the first chain and then smaller number for the
second chain. So the execution order in that case will be reverse:
second chain will be executed first.

To avoid that we link one chain to another. While the base chain
(ddl_log_state_create) is active the secondary chain
(ddl_log_state_rm) is not executed. That is: only one chain can be
executed in two linked chains.

The interface ddl_log_link_chains() was defined in "MDEV-22166
ddl_log_write_execute_entry() extension".

Atomic info parameters in HA_CREATE_INFO

Many functions in CREATE TABLE pass the same parameters. These
parameters are part of table creation info and should be in
HA_CREATE_INFO (or whatever). Passing parameters via single
structure is much easier for adding new data and
refactoring.

InnoDB changes
Added ha_innobase::can_be_renamed_to_backup() to check if
a table with foreign keys can be renamed.

Aria changes:
- Fixed issue in Aria engine with CREATE + locked tables
that data was not properly commited in some cases in
case of crashes.

Other changes:
- Removed some auto variables in log.cc for better code readability.
- Fixed old bug that CREATE ... SELECT would not be able to auto repair
a table that is part of the SELECT.
- Marked MyISAM that it does not support ROLLBACK (not required but
done for better consistency with other engines).

Known issues:
- InnoDB tables with foreign key definitions are not fully supported
with atomic create and replace:
- ha_innobase::can_be_renamed_to_backup() can detect some cases
where InnoDB does not support renaming table with foreign key
constraints. In this case MariaDB will drop the old table before
creating the new one.
The detected cases are:
- The new and old table is using the same foreign key constraint
name.
- The old table has self referencing constraints.
- If the old and new table uses the same name for a constraint the
create of the new table will fail. The orignal table will be
restored in this case.
- The above issues will be fixed in a future commit.
- CREATE OR REPLACE TEMPORARY table is not full atomic. Any conflicting
table will always be dropped before creating a new one. (Old behaviour).

Bug fixes related to this MDEV:

MDEV-36435 Assertion failure in finalize_locked_tables()
MDEV-36439 Assertion `thd_arg->lex->sql_command != SQLCOM_CREATE_SEQUENCE...
MDEV-36498 Failed CoR in non-atomic mode no longer generates DROP in RBR...
MDEV-36508 Temporary files #sql-create-....frm occasionally stay after
crash recovery
MDEV-38479 Crash in CREATE OR REPLACE SEQUENCE when new sequence cannot
be created
MDEV-36497 Assertion failure after atomic CoR with Aria under lock in
transactional context

InnoDB related changes:
- ha_innodb::rename_table() does not handle foreign key constraint
when renaming an normal table to internal tempory tables. This
causes problems for CREATE OR REPLACE as the old constraints causes
failure when creating a new table with the same constraints.
This is fixed inside InnoDB by not threating tempfiles (#sql-create-..),
created as part of CREATE OR REPLACE, as temporary files.
- In ha_innobase::delete_table(), ignore checking of constraints when
dropping a #sql-create temporary table.
- In tablename_to_filename() and filename_to_tablename(), don't do
filename conversion for internal temporary tables (#sql-...)

Other things:
- maria_create_trn_for_mysql() does not register a new transaction
handler for commits. This was needed to ensure create or replace
will not end with an active transaction.
- We do not get anymore warnings about "Engine not supporting atomic
create" when doing a legal CREATE OR REPLACE on a table with
foreign key constraints.
- Updated VIDEX engine flags to disable CREATE SEQUENCE.

Reverted commits:
MDEV-36685 "CREATE-SELECT may lose in binlog side-effects of
stored-routine" as it did not take into account that it safe to clear
binlogs if the created table is non transactional and there are no
other non transactional tables used.
- This was done because it caused extra logging when it is not needed
(not using any non transactional tables) and it also did not solve
side effects when using statement based loggging.

a9998ddda861...

a9998ddda8611d6454804df04341e2408e3eedfc

Michael Widenius

monty@mariadb.org

9f8f6d24dbf2...

9f8f6d24dbf23c7293a1fd437f1605eeeb0d42df

Denis Protivensky

denis.protivensky@mariadb.com

MDEV-39011: Fix THD leak in async slave error path with wsrep_restart_slave

- fix the leak by properly releasing resources before repeat
- rework the error path handling to wait for Wsrep readiness state
instead of repeating endlessly
- improve Wsrep readiness state by also singlaing node's shutdown

9984d6885f6e...

9984d6885f6e8a17bca55e74001773d1f6fec51e

Alexander Barkov

bar@mariadb.com

MDEV-39115 `FUNCTION does not exist` error on package body assoc array variable

Problem:
When parsing `a.b()`, LEX::make_item_func_or_method_call() could not find
a package body variable 'a'.

Fix:
When searching for a variable, search not only in the current parse context,
but also in the parent package body context (if exists).

96561be1cdee...

96561be1cdeec673e703e75339979867529e8a11

Aleksey Midenkov

midenok@gmail.com

MDEV-32317 Trace for ref_ptrs array (ref_array keyword)

Usage:

mtr --mysqld=--debug=d,ref_array,query:i:o,/tmp/mdl.log

93b1cd839f2e...

93b1cd839f2e39340ba69f82be7e7706e65fcada

forkfun

alice.sherepa@gmail.com

MDEV-30295 mysqldump produces syntactically incorrect statement

Remove version-specific executable comments, that were used for backward compatibility
with old MySQL versions (3.2, 4.0, 4.1, 5.0).

8f9338cb37b5...

8f9338cb37b5ae717ccd65314de2ab3519081e7a

Marko Mäkelä

marko.makela@mariadb.com

MDEV-39139 InnoDB fails to start up with small RLIMIT_AS

innodb_init_params(): When no innodb_buffer_pool_size_max is
is specified, cap the default to a quarter of the address space limit.

807bfd05d8e4...

807bfd05d8e499aab1bed04ecbc8ccc3df542e23

Marko Mäkelä

marko.makela@mariadb.com

MDEV-39040 log_sys.latch performance lost to PERFORMANCE_SCHEMA

log_sys.latch: Remove the PERFORMANCE_SCHEMA instrumentation.
We already know that this is a very busy latch. All code paths
where a shared log_sys.latch is being held should already be
highly optimized. The few paths where an exclusive log_sys.latch
is being held are known to be potentially problematic, but rare.

Removing the PERFORMANCE_SCHEMA instrumentation for this latch
is expected to improve performance. On a quick Sysbench run with

performance_schema_instrument=wait/synch/rwlock/%=on

I observed an 2.9% improvement on throughput.

srw_lock_debug_simple: A non-instrumented version of srw_lock_debug,
to allow PERFORMANCE_SCHEMA to work WITH_INNODB_EXTRA_DEBUG=ON.

Reviewed by: Alessandro Vetere

7f544e9a9732...

7f544e9a9732a6722e1e056755658f6ddd3241a2

Aleksey Midenkov

midenok@gmail.com

MDEV-25365 Server hangs or crashes in Query_cache::move_by_type upon FLUSH QUERY CACHE

Caused by Bug#988 in 702b5ad.

How it works

QUERY blocks has all its tables in the array coming after
Query_cache_block header. Each such table element
(Query_cache_block_table) is also an element of doubly-linked circular
list associated with TABLE block used to find all queries by a
table. TABLE block data() is actually Query_cache_table: identificator
for table elements by real name. Every element in the circular list
has `parent` set to such Query_cache_table, except the list root. The
list root is the only Query_cache_block_table element owned by a TABLE
block, all other elements are owned by QUERY blocks. Thus TABLE block
n_tables is always 1.

FREE blocks are transformed into gap pointed by `border`. All the
QUERY and TABLE blocks are moved in place of the gap providing free
space defragmentation. Moving is done by creating new block at the
location of the gap and then doing memmove() from an old block to new
block. This may involve blocks overlap when the distance between the
blocks is less than the block size.

What was broken

When the old and the new QUERY blocks overlap, pointer can be in range
of both of them and checking the range does not guarantee affiliation
of the pointer with the old block. Such intra-block checks added by
Bug#988 can wrongly work on an already updated pointer.

In the test case small gap allows both tables 16 and 17 to be in the
overlapping region of old and new blocks. Iteration 16 updated
`prev->next` on the normal manner and the `prev` was table 17.
Iteration 17 sees this `next` as a pointer coming from an old block
though really it is already for a new block (as small gap shift allows
table 16 to be in both blocks), and it shifts it to the wrong
location. That have broken TABLE block circular chain.

How it was fixed

The fix modifies intra-block checks to work only in the forward
direction. F.ex., current table jx references intra-block by `next`
another table jy in later iteration (jx < jy) and we shift this `next`
pointer. Then by normal code (pre-Bug#988) we update `next->prev`. At
iteration jy we skip intra-block check for `prev` as it is in the past
iteration and was updated by iteration jx. The same happens for the
opposite direction, there is no semantic difference between `next` and
`prev` pointers.

69ad31dd7b7b...

69ad31dd7b7bcc1ae39fce86309b23d435613899

Aleksey Midenkov

midenok@gmail.com

MDEV-28619 with_flags cleanup

The best practice is to init as much info as possible in class
constructor. with_flags access may be needed before fix_fields() or
fix_fields() may be not called at all like it takes place in
MDEV-28619. The fix for MDEV-28619 requires WINDOW_FUNC check on
unfixed item.

68e891356a55...

68e891356a55978994b2a961600f68d81903092b

Alexey Botchkov

holyfoot@askmonty.org

progress
MDEV-37262 XMLTYPE:
20.03

6622d36ebed3...

6622d36ebed334ce3be49739770ff955c10b6e9a

Marko Mäkelä

marko.makela@mariadb.com

MDEV-39139 InnoDB fails to start up with small RLIMIT_AS

innodb_init_params(): When RLIMIT_AS is limited,
limit innodb_buffer_pool_size_max to half of it.

660a90e7fe33...

660a90e7fe33aff58327d52e5bcb019864f6f965

Marko Mäkelä

marko.makela@mariadb.com

MDEV-39139 InnoDB fails to start up with small RLIMIT_AS

innodb_init_params(): When no innodb_buffer_pool_size_max has been
specified, cap the default to a quarter of the address space limit.

64771e05e4fc...

64771e05e4fcb7c166a65eea97b79268a4f997ac

Aleksey Midenkov

midenok@gmail.com

MDEV-25365 More query cache debug trace

Usage:

mtr --mysqld=--debug=d,qcache,query:i:O,/tmp/qcache_full.log
sed -nre '/(move_by_type|dispatch_command|pack_cache)/ p' \
< /tmp/qcache_full.log \
> /tmp/qcache.log

641ccdc648ec...

641ccdc648ec2c7387ddf740cf630289fc80a94b

Alexander Barkov

bar@mariadb.com

MDEV-39022 Add `LOCAL spvar` syntax for prepared statements and SYS_REFCURSORs

This patch adds the following syntax:

OPEN c0 FOR LOCAL spvar_with_ps_name;
PREPARE LOCAL spvar_with_ps_name FROM 'dynamic sql';
EXECUTE LOCAL spvar_with_ps_name;
DEALLOCATE PREPARE LOCAL spvar_with_ps_name;

OPEN c0 FOR PREPARE stmt;

5ff5281e5afb...

5ff5281e5afb8188e22c858c8c10a07016ce099c

Aleksey Midenkov

midenok@gmail.com

MDEV-28619 Server crash and UBSAN null-pointer-use in Window_funcs_sort::setup

Optimization in st_select_lex_unit::prepare() removes ORDER BY for
certain subqueries. That excludes ORDER BY items from being fixed, but
sl->window_funcs still contains window function items and those
related to optimized out ORDER BY are unfixed. The error about missing
window spec is thrown when the item is fixed. Hence we get redundant
processing of window function items without checking window spec
existence.

The fix removes the related window function items when ORDER BY is
optimized out. ORDER accumulates window_funcs at parser stage which
are then removed from SELECT_LEX::window_funcs. The fix also updates
similar optimization in mysql_make_view().

5e63cb279c41...

5e63cb279c41c166ee0d0dd058eebb47947a1bcc

Marko Mäkelä

marko.makela@mariadb.com

backup_innodb stub

5ae44f2a914e...

5ae44f2a914e5df8aad84dece46faa2c024c0f73

Alexey Botchkov

holyfoot@askmonty.org

progress

5853d939054e...

5853d939054eeb4211fd17a7168dfe071c3462b8

Aleksey Midenkov

midenok@gmail.com

MDEV-36362 MariaDB crashes when parsing fuzzer generated PARTITION

Function calls in INTERVAL expression of DDL have little
sense. SETVAL() fails because sequences require opened tables and
vers_set_interval() is called at earlier stage.

The fix disables function calls in INTERVAL expressions and limits it
to signal_allowed_expr (literal, variable, simple_ident).

52d0e75ba8f7...

52d0e75ba8f77fd33e92d96de386b529a1b25839

Alexey Botchkov

holyfoot@askmonty.org

progress

4e2a46b90ea3...

4e2a46b90ea3e669158edb55bcfc602aaf2d38eb

Monty

monty@mariadb.org

3c32991a4122...

3c32991a4122a2b67fbfa20375a11f70a00cc802

Daniel Black

daniel@mariadb.org

Bump columnstore to latest stable-23.10

3ab87db19b82...

3ab87db19b823e64d991c1b1705cc58f69ffa46e

Marko Mäkelä

marko.makela@mariadb.com

MDEV-39104 Hang in the test innodb.log_file_size,debug

srv_prepare_to_delete_redo_log_file(), srv_start(): When injecting a
failure, resume the ability to write a checkpoint, so that the
shutdown will not hang. Also, adjust the test so that it tolerates
the checkpoint during shutdown.

Fixes up commit bf363a573ab8f96657545de97ead568d09a9f1a0 (MDEV-38968)

37b3b302367f...

37b3b302367f3c57ca69283f9c6093803d28ccc0

Marko Mäkelä

marko.makela@mariadb.com

MDEV-37949: Implement innodb_log_archive

The InnoDB write-ahead log file in the old innodb_log_archive=OFF
format is named ib_logfile0, pre-allocated to innodb_log_file_size and
written as a ring buffer. This is good for write performance and space
management, but unsuitable for arbitrary point-in-time recovery or for
facilitating efficient incremental backup.

innodb_log_archive=ON: A new format where InnoDB will create and
preallocate files ib_%016x.log, instead of writing a circular file
ib_logfile0. Each file will be pre-allocated to innodb_log_file_size
(between 4M and 4G; we impose a stricter upper limit of 4 GiB for
innodb_log_archive=ON). Once a log fills up, we will create and
pre-allocate another log file, to which log records will be written.
Upon the completion of the first log checkpoint in a recently created
log file, the old log file will be marked read-only, signaling that
there will be no further writes to that file, and that the file may
safely be moved to long-term storage.

The file name includes the log sequence number (LSN) at file offset
12288 (log_t::START_OFFSET). Limiting the file size to 4 GiB allows us
to identify each checkpoint by storing a 32-bit big-endian offset into
the optional FILE_MODIFY and the mandatory FILE_CHECKPOINT records,
between 12288 and the end of the file.

The innodb_encrypt_log format is identified by storing the encryption
information at the start of the log file. The first 32-bit value will
be 1, which is an invalid checkpoint offset. Each
innodb_log_archive=ON log must use the same encryption parameters.
Changing innodb_encrypt_log or related parameters is only possible by
setting innodb_log_archive=OFF and restarting the server, which will
permanently lose the history of the archived log.

The maximum number of log checkpoints that the innodb_log_archive=ON
file header can represent is limited to 12288/4=3072 when using
innodb_encrypt_log=OFF. If we run out of slots in a log file, each
subsequently completed checkpoint in that log file will overwrite the
last slot in the checkpoint header, until we switch to the next log.

innodb_log_recovery_start: The checkpoint LSN to start recovery from.
This will be useful when recovering from an archived log. This is useful
for restoring an incremental backup (applying InnoDB log files that were
copied since the previous restore).

innodb_log_recovery_target: The requested LSN to end recovery at.
When this is set, all persistent InnoDB tables will be read-only, and
no writes to the log are allowed. The intended purpose of this setting
is to prepare an incremental backup, as well as to allow data
retrieval as of a particular logical point of time.

Setting innodb_log_recovery_target>0 is much like setting
innodb_read_only=ON, with the exception that the data files may be
written to by crash recovery, and locking reads will conflict with any
incomplete transactions as necessary, and all transaction isolation
levels will work normally (not hard-wired to READ UNCOMMITTED).

srv_read_only_mode: When this is set (innodb_read_only=ON), also
recv_sys.rpo (innodb_log_recovery_target) will be set to the current LSN.
This ensures that it will suffice to check only one of these variables
when blocking writes to persistent tables.

The status variable innodb_lsn_archived will reflect the LSN
since when a complete InnoDB log archive is available. Its initial
value will be that of the new parameter innodb_log_archive_start.
If that variable is 0 (the default), the innodb_lsn_archived will
be recovered from the available log files. If innodb_log_archive=OFF,
innodb_lsn_archived will be adjusted to the latest checkpoint every
time a log checkpoint is executed. If innodb_log_archive=ON, the value
should not change.

SET GLOBAL innodb_log_archive=!@@GLOBAL.innodb_log_archive will take
effect as soon as possible, possibly after a log checkpoint has been
completed. The log file will be renamed between ib_logfile0 and
ib_%016x.log as appropriate.

When innodb_log_archive=ON, the setting SET GLOBAL innodb_log_file_size
will affect subsequently created log files when the file that is being
currently written is running out. If we are switching log files exactly
at the same time, then a somewhat misleading error message
"innodb_log_file_size change is already in progress" will be issued.

no_checkpoint_prepare.inc: A new file, to prepare for subsequent
inclusion of no_checkpoint_end.inc. We will invoke the server to
parse the log and to determine the latest checkpoint.

All --suite=encryption tests that use innodb_encrypt_log
will be skipped for innodb_log_archive=ON, because enabling
or disabling encryption on the log is not possible without
temporarily setting innodb_log_archive=OFF and restarting
the server. The idea is to add the following arguments to an
invocation of mysql-test/mtr:

--mysqld=--loose-innodb-log-archive \
--mysqld=--loose-innodb-log-recovery-start=12288 \
--mysqld=--loose-innodb-log-file-mmap=OFF \
--skip-test=mariabackup

Alternatively, specify --mysqld=--loose-innodb-log-file-mmap=ON
to cover both code paths.

The mariabackup test suite must be skipped when using the
innodb_log_archive=ON format, because mariadb-backup will only
support the old ib_logfile0 format (innodb_log_archive=OFF).

A number of tests would fail when the parameter
innodb_log_recovery_start=12288 is present, which is forcing
recovery to start from the beginning of the history
(the database creation). The affected tests have been adjusted with
explicit --innodb-log-recovery-start=0 to override that:

(0) Some injected corruption may be "healed" by replaying the log
from the beginning. Some tests expect an empty buffer pool after
a restart, with no page I/O due to crash recovery.
(1) Any test that sets innodb_read_only=ON would fail with an error
message that the setting prevents crash recovery, unless
innodb_log_recovery_start=0.
(2) Any test that changes innodb_undo_tablespaces would fail in crash
recovery, because crash recovery assumes that the undo tablespace ID
that is available from the undo* files corresponds with the start of
the log. This is an unforunate design bug which we cannot fix easily.

log_sys.first_lsn: The start of the current log file, to be consulted
in log_t::write_checkpoint() when renaming files.

log_sys.archived_lsn: New field: The value of innodb_lsn_archived.

log_sys.end_lsn: New field: The log_sys.get_lsn() when the latest
checkpoint was initiated. That is, the start LSN of a possibly empty
sequence of FILE_MODIFY records followed by FILE_CHECKPOINT.

log_sys.resize_target: The value of innodb_log_file_size that will be
used for creating the next archive log file once the current file (of
log_sys.file_size) fills up.

log_sys.archive: New field: The value of innodb_log_archive.

log_sys.next_checkpoint_no: Widen to uint16_t. There may be up to
12288/4=3072 checkpoints in the header.

log_sys.log: If innodb_log_archive=ON, this file handle will be kept
open also in the PMEM code path.

log_sys.resize_log: If innodb_log_archive=ON, we may have two log
files open both during normal operation and when parsing the log. This
will store the other handle (old or new file).

log_sys.resize_buf: In the memory-mapped code path, this will point
to the file resize_log when innodb_log_archive=ON.

recv_sys.log_archive: All innodb_log_archive=ON files that will be
considered in recovery.

recv_sys.was_archive: A flag indicating that an innodb_log_archive=ON
file is in innodb_log_archive=OFF format.

log_sys.is_pmem, log_t::is_mmap_writeable(): A new predicate.
If is_mmap_writeable(), we assert and guarantee buf_size == capacity().

log_t::archive_new_write(): Create and allocate a new log file, and
write the outstanding data to both the current and the new file, or
only to the new file, until write_checkpoint() completes the first
checkpoint in the new file.

log_t::archived_mmap_switch_prepare(): Create and memory-map a new log
file, and update file_size to resize_target. Remember the file handle
of the current log in resize_log, so that write_checkpoint() will be
able to make it read-only.

log_t::archived_mmap_switch_complete(): Switch to the buffer that was
created in archived_mmap_switch_prepare().

log_t::write_checkpoint(): Allow an old checkpoint to be completed in
the old log file even after a new one has been created. If we are
writing the first checkpoint in a new log file, we will mark the old
log file read-only. We will also update log_sys.first_lsn unless it
was already updated in ARCHIVED_MMAP code path. In that code path,
there is the special case where log_sys.resize_buf == nullptr and
log_sys.checkpoint_buf points to log_sys.resize_log (the old log file
that is about to be made read-only). In this case, log_sys.first_lsn
will already point to the start of the current log_sys.log, even
though the switch has not been fully completed yet.

log_t::header_rewrite(my_bool): Rewrite the log file header before or
after renaming the log file, and write a message about the change,
so that there will be a chance to recover in case the server is being
killed during this operation. The recovery of the last ib_%016%.log
does tolerate also the ib_logfile0 format.

log_t::set_archive(my_bool,THD): Implement SET GLOBAL innodb_log_archive.
An error will be returned if non-archived SET GLOBAL innodb_log_file_size
(log file resizing) is in progress. Wait for checkpoint if necessary.
The current log file will be renamed to either ib_logfile0 or
ib_%016x.log, as appropriate.

log_t::archive_rename(): Rename an archived log to ib_logfile0 on recovery
in case there had been a crash during set_archive().

log_t::archive_set_size(): A new function, to ensure that
log_sys.resize_target is set on startup.

log_checkpoint_low(): Do not prevent a checkpoint at the start of a file.
We want the first innodb_log_archive=ON file to start with a checkpoint.

log_t::create(lsn_t): Initialize last_checkpoint_lsn. Initialize the
log header as specified by log_sys.archive (innodb_log_archive).

log_write_buf(): Add the parameter max_length, the file wrap limit.

log_write_up_to(), mtr_t::commit_log_release<bool mmap=true>():
If we are switching log files, invoke buf_flush_ahead(lsn, true)
to ensure that a log checkpoint will be completed in the new file.

mtr_t::finish_writer(): Specialize for innodb_log_archive=ON.

mtr_t::commit_file(): Ensure that log archive rotation will complete.

log_t::append_prepare<log_t::ARCHIVED_MMAP>(): Special case.

log_t::get_path(): Get the name of the current log file.

log_t::get_circular_path(size_t): Get the path name of a circular file.
Replaces get_log_file_path().

log_t::get_archive_path(lsn_t): Return a name of an archived log file.

log_t::get_next_archive_path(): Return the name of the next archived log.

log_t::append_archive_name(): Append the archive log file name
to a path string.

mtr_t::finish_writer(): Invoke log_close() only if innodb_log_archive=OFF.
In the innodb_log_archive=ON, we only force log checkpoints after creating
a new archive file, to ensure that the first checkpoint will be written
as soon as possible.

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().
If a new archived log file has been created, wait for the
first checkpoint in that file.

srv_log_rebuild_if_needed(): Never rebuild if innodb_log_archive=ON.
The setting innodb_log_file_size will affect the creation of
subsequent log files. The parameter innodb_encrypt_log cannot be
changed while the log is in the innodb_log_archive=ON format.

log_t::attach(), log_mmap(): Add the parameter log_access,
to distinguish memory-mapped or read-only access.

log_t::attach(): When disabling innodb_log_file_mmap, read
checkpoint_buf from the last innodb_log_archive=ON file.

log_t::clear_mmap(): Clear the tail of the checkpoint buffer
if is_mmap_writeable().

log_t::set_recovered(): Invoke clear_mmap(), and restore the
log buffer to the correct position.

recv_sys_t::apply(): Let log_t::clear_mmap() enable log writes.

recv_sys_t::find_checkpoint(): Find and remember the checkpoint position
in the last file when innodb_log_recovery_start points to an older file.
When innodb_log_file_mmap=OFF, restore log_sys.checkpoint_buf from
the latest log file. If the last archive log file is actually
in innodb_log_archive=OFF format despite being named ib_%016.log,
try to recover it in that format. If the circular ib_logfile0 is missing,
determine the oldest archived log file with contiguous LSN.
If innodb_log_archive=ON, refuse to start if ib_logfile0 exists.
Open non-last archived log files in read-only mode.

recv_sys_t::find_checkpoint_archived(): Validate each checkpoint in
the current file header, and by default aim to recover from the last
valid one. Terminate the search if the last validated checkpoint
spanned two files. If innodb_log_recovery_start has been specified,
attempt to validate it even if there is no such information stored
in the checkpoint header.

log_parse_file(): Do not invoke fil_name_process() during
recv_sys_t::find_checkpoint_archived(), when we tolerate FILE_MODIFY
records while looking for a FILE_CHECKPOINT record.

recv_scan_log(): Invoke log_t::archived_switch_recovery() upon
reaching the end of the current archived log file.

log_t::archived_switch_recovery_prepare(): Make use of
recv_sys.log_archive and open all but the last file read-only.

log_t::archived_switch_recovery(): Switch files in the pread() code path.

log_t::archived_mmap_switch_recovery_complete(): Switch files in the
memory-mapped code path.

recv_warp: A pointer wrapper for memory-mapped parsing that spans two
archive log files.

recv_sys_t::parse_mmap(): Use recv_warp for innodb_log_archive=ON.

recv_sys_t::parse(): Tweak some logic for innodb_log_archive=ON.

log_t::set_recovered_checkpoint(): Set the checkpoint on recovery.
Updates also the end_lsn.

log_t::set_recovered_lsn(): Also update flush_lock and write_lock,
to ensure that log_write_up_to() will be a no-op.

log_t::persist(): Even if the flushed_to_disk_lsn does not change,
we may want to reset the write_lsn_offset.

36e12d347034...

36e12d347034a9bbfa502c5d589958196fcd6258

Alexander Barkov

bar@mariadb.com

MDEV-38933 Implement basic low level functions for DBMS_SQL

Adding SYS_REFCURSOR methods:
- id()
- attach()
- detach()

Adding non-DBMS_SQL-specific function plugins (pseudo-procedures):
- SQL.BIND_PARAM_BY_NAME(ps_name, pos, value)
- SQL.COLUMN_VALUE(sys_refcursor_id, pos, dst)

Adding DBMS_SQL-specific function plugins (pseduo-procedures)
- DBMS_SQL.BIND_VARIABLE(dbms_sql_cursor_id, pos, value)
- DBMS_SQL.COLUMN_VALUE(dbms_sql_cursor_id, pos, value)

269f7ae52ee9...

269f7ae52ee930125f468095654317d3b1616c0a

forkfun

alice.sherepa@gmail.com

MDEV-30295 mysqldump produces syntactically incorrect statement

Remove version-specific executable comments, that were used for backward compatibility
with old MySQL versions (3.2, 4.0, 4.1, 5.0).

1e715e0fe209...

1e715e0fe209bdd741aa3fcde03b88349fb2e39a

Michael Widenius

monty@mariadb.org

1cdbb7216255...

1cdbb721625537f5dbf525ef134d5f40c9bd3981

Michael Widenius

monty@mariadb.org

1707088332e1...

1707088332e1b05114a0806177233756c7fe0c22

Sergei Golubchik

serg@mariadb.org

fix tests after d755574c47f1

136a6f9ef3e2...

136a6f9ef3e20ca5929eeb450bfe8065336d1791

forkfun

alice.sherepa@gmail.com

MDEV-30295 mysqldump produces syntactically incorrect statement

Remove version-specific executable comments, that were used for backward compatibility
with old MySQL versions (3.2, 4.0, 4.1, 5.0).

07f70cc5cab2...

07f70cc5cab298f1fcc1ce9ce74a3511a544225b

Sergei Golubchik

serg@mariadb.org

fix tests after d755574c47f1