Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Kristian Nielsen
Binlog-in-engine: Disable test binlog_in_engine.rpl_gtid_index for Valgrind

The test requires large amounts of CPU and runs too slowly in Valgrind to
make sense, can even occasionally time out on loaded machines.

Signed-off-by: Kristian Nielsen <[email protected]>
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

After hash chain is unlocked and buffer pool mutex is released in
buf_page_init_for_read(), the chain list can be changed by concurent
thread, that's why we need zero out extended buffer page pointer.

Make ext_bp_size to be atomic.

Process asinc errors in os_aio() correctly.
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

Some code review fixes. Needs RQG testing as some new assertions were
added and some conditions were changed.
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

RocksDB test fix.
Brandon Nesterenko
MDEV-38117: Replication stops with ERROR when Primary Key is not defined in Multi Master

When replicating tables with different structures between a master and
slave (via binlog_row_metadata=FULL, i.e. MDEV-36290), if the table
didn't have any keys, replication could break from a
  Can't find record in '<table>', Error_code: 1032;
error. This was due to the table's internal tracking of provided values
(i.e. TABLE::has_value_set) persisting across transactions. I.e., when
applying a new row event, the has_value_set bitset would start in a
state indicative of the last event. Then, if the new row event required
first finding a row, the has_value_set could represent a larger bitset
than what was actually unpacked from the master. More specifically, if
the last row event inserted something into the table, where slave-only
columns would be given default/auto-generated values, these would
update the TABLE::has_value_set. However, when the next row event would
come in, those auto-generated values from the last transaction would
not be present, but the row-search would include them to look-up the
row, and not be able to find anything.

This is fixed by resetting the table's internal state of provided
values (TABLE::has_value_set) before unpacking the row data. Doing so
revealed a bug where unpacked fields which explicitly provided NULL
values would skip indicating that an explicit value was provided
(and thereby couldn't be found by find_row()). To fix this, the
slave-side field will always call has_explicit_value() if it is
present in the packed data.

Test case based on work from: Deepthi Sreenivas <[email protected]>

Signed-off-by: Brandon Nesterenko <[email protected]>
Reviewed-by: Monty <[email protected]>
Marko Mäkelä
Atomic doublewrite buffer creation

buf_dblwr_t::create(): Create the doublewrite buffer in a single
atomic mini-transaction. Do not write any log records for
initializing any doublewrite buffer pages, in order to avoid
recovery failure with innodb_log_archive=ON starting from the
very beginning.

Cherry-picking from 1d1699eef3dfca20a075de7a282ec92ccf03166e
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

Fix some tests. Make ext_buf_pool test more stable avoiding race
conditions for read/write counters.
Brandon Nesterenko
MDEV-38435: Update test results
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

Get rid of ext_buf_page_t::frame.
Brandon Nesterenko
MDEV-38641: Failure of Replication of System Versioning Tables

System versioned table UPDATES would fail to replicate on debug builds
with the debug assertion:

rpl_utility_server.cc:1058: bool RPL_TABLE_LIST::give_compatibility_error(rpl_group_info *, uint): Assertion `m_tabledef.master_column_name[col]' failed.

Though caught with system versioned tables, the problem is
generalizable to any transactions which have multiple Rows_log_events
that update the same table. That is, during the error reporting for
columns which were present in the Rows_log_event but not on the slave
table, there is a debug assertion that validates that the master's
table name exists.  After reporting this error, the pointer to that
master table name is nullified. This means that future Rows_log_events
would not have this table name, and the assertion would fail (release
builds would likely segfault when logging the error).

The fix for this is not to nullify the pointer after reporting the
error, so future Rows_log_events can continue using the pointer to the
master's table name.

Signed-off-by: Brandon Nesterenko <[email protected]>
Monty
MDEV-36290: Improved support of replication between tables of different structure

One can have data loss in multi-master setups when 1) both masters
update the same table, 2) ALTER TABLE is run on one master which
re-arranges the column ordering, and 3) transactions are binlogged
in ROW binlog_format.

This is because the slave assumes that all columns are in the same
order on the master and slave and all columns on the master also
exists on the slave. This happens even if binlog_row_metadata=FULL is
used.  If this is not the case, this will lead to silent data loss.

A new option for slave_type_conversions bit field,
ERROR_IF_MISSING_FIELD, has been added, along with a new error,
ER_SLAVE_INCOMPATIBLE_TABLE_DEF. This allows the user to define if
the slave should abort replication if it is missing some field that
existed on the master. The option is off by default to keep things
compatible with earlier versions.
If a field is missing on the slave and log_warnings >= 1, a warning
will be logged to the error log.

This patch fixes this, when binlog_row_metadata=FULL is used on the
master, by mapping fields with identical names on the master and slave.
If slave has fields that does not exist in the row event, these will
be set to their default value.

The main idea is that we added two conversion tables:
m_tabledef.master_to_slave_map[master_column_index] -> slave_column_index
and m_tabledef.master_to_slave_error[master_column_index] which contains
an error number if the master_column does not exist on the slave or
it is not possible to convert the master data to the slave column.
master_to_slave_error[#] contains 0 if the column exists and is compatible.

General code changes:
- Instead of looping over row fields in the order of slave table
  we are instead looping over fields in the order of the binary log.
- We are using table->write_set to know which fields should be updated
  on the slave. This is reflected in unpack_row
- We are calling TABLE::mark_columns_per_binlog_row_image() to ensure
  that rpl_write_set is properly set. This is needed if the slave also
  is doing binary logging.
- Before replication aborted if the master and slave tables were too
  different.  Now replication is only aborted if the row actually uses
  columns that does not exists on the slave (and ERROR_IF_MISSING_FIELD
  is used) or uses columns that cannot be converted.
  - Instead of giving errors in compatible_with(), used when table is
    accessed by first the row event, we are instead giving errors
    when we examine a row event and notice that it is accessing
    a not existing or not compatible field.

Other code changes:
- Removed conv_table argument from compatible_with() and store it
  directly in RPL_TABLE_LIST->m_conv_table
- table_def::compatible_with() returns now 1 on error (not 0).
- Remove m_width and skip arguments from prepare_record() as we are
  now using table->write_set() to check which elements need a default
  value.
- Moved DBUG_ENTER() to it's proper place (after variable
  declarations) in a few functions.
- Some changes in unpack_row():
  - Replaced null_mask and null_ptr with an indexed bit check for
    simplicity.
  - Removed check of rgi == null and table_found which never worked.
  - Updated comments to reflect current code.
  - Indentation changes as the code now uses 'continue' instead of
    'if-else' in the main loop.
  - The code to throw away 'extra master fields' is not needed as we
    are now looping over fields in binary log, not over fields in
    slave table.
- Simplified get_table_data(TABLE *table_arg) by returning found
  table_list.
- Errors for row events are now initialized in compatible_with(),
  checked in check_wrong_column_usage() and reported in
  give_compatibility_error().

Co-authored-by: Brandon Nesterenko <[email protected]>
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

Squash it.

Fix for the following RQG test failures:

2. Scenario: The server is under load (9 concurrent sessions).
At some point of time he crashes with
    mariadbd: 11.8-MDEV-31956-ext_buf_pool/storage/innobase/buf/buf0flu.cc:294: void buf_page_t::write_complete(buf_page_t::space_type, bool, uint32_t): Assertion `persistent == (om > 2)' failed.

4. Scenario: The server was some time under load (one connection).
Intentional SIGKILL DB server followed by restart and running certain checks.
All that did not show some error. But the shutdown hang like
Fragment of rqg.log:

    # 2026-01-16T13:15:57 [1467965] INFO: DBServer_e::MySQL::MySQLd::stopServer: server[1]: Stopping server on port 25140
    ...
    # 2026-01-16T13:28:22 [1467965] ERROR: DBServer_e::MySQL::MySQLd::stopServer: server[1]: Did not shut down properly. Terminate it
                    == RQG loses the "patience" and sends finally SIGABRT to the process of the DB server.

The server error log shows
      2026-01-16 13:15:58 0 [Note] /data/Server_bin/11.8-MDEV-31956-ext_buf_pool_debug_Og/bin/mariadbd (initiated by: root[root] @ localhost [127.0.0.1]): Normal shutdown
      ...
    2026-01-16 13:15:58 0 [Note] InnoDB: FTS optimize thread exiting.
    2026-01-16 13:16:01 0 [Note] InnoDB: Starting shutdown...
    ....
    2026-01-16 13:16:01 0 [Note] InnoDB: Buffer pool(s) dump completed at 260116 13:16:01
    2026-01-16 13:18:37 0 [Note] InnoDB: Waiting for page cleaner thread to exit
    ....
    2026-01-16 13:26:24 0 [Note] InnoDB: Waiting for page cleaner thread to exit
Michael Widenius
MDEV-19683 Add support for Oracle TO_DATE()

Syntax:
TO_DATE(string_expression [DEFAULT string_expression ON CONVERSION ERROR],
        format_string [,NLS_FORMAT_STRING])
The format_string has the same format elements as TO_CHAR(), except a
few elements that are not supported/usable for TO_DATE().
TO_DATE() returns a datetime or date value, depending on if the format
element FF is used.

Allowed separators, same as TO_CHAR():
space, tab and any of !#%'()*+,-./:;<=>

'&' can also be used if next character is not a character a-z or A-Z
"text' indicates a text string that is verbatim in the format. One cannot
use " as a separator.

Format elements supported by TO_DATE():
AD          Anno Domini ("in the year of the Lord")
AD_DOT      Anno Domini ("in the year of the Lord")
AM          Meridian indicator (Before midday)
AM_DOT      Meridian indicator (Before midday)
DAY        Name of day
DD          Day (1-31)
DDD        Day of year (1-336)
DY          Abbreviated name of day
FF[1-6]    Fractional seconds
HH          Hour (1-12)
HH12        Hour (1-12)
HH24        Hour (0-23)
MI          Minutes (0-59)
MM          Month (1-12)
MON        Abbreviated name of month
MONTH      Name of Month
PM          Meridian indicator (After midday)
PM_DOT      Meridian indicator (After midday)
RR          20th century dates in the 21st century. 2 digits
            50-99 is assumed from 2000, 0-49 is assumed from 1900.
RRRR        20th century dates in the 21st century. 4 digits
SS          Seconds
SYYYY      Signed 4 digit year; MariaDB only supports positive years
Y          1 digit year
YY          2 digits year
YYY        3 digits year
YYYY        4 digits year

Note that if there is a missing part of the date, the current date is used!
For example if 'MM-DD HH-MM-SS' then the current year will be used.
(Oracle behaviour)

Not supported options:
- BC, D, DL, DS, E, EE, FM, FX, RM, SSSSS, TS, TZD, TZH, TZR, X,SY
  BC is not supported by MariaDB datetime.
- Most of the other are exotic formats does not make sence in MariaDB as
  we return datetime or datetime with fractions, not string.
- D (day-of-week) is not supported as it is not clear exactly how it would
  map to MariaDB. This element depends on the NLS territory of the session.
- RR only works with 2 digit years (In Oracle RR can also work with 4
  digit years in some context but the rules are not clear).

Extensions / differences compared to Oracle;
- MariaDB supports FF (fractional seconds).  If FF[#] is used,
  then TO_DATE will return a datetime with # of subseconds.
  If FF is not used a datetime will be returned.
  There is warning (no error) if string contains more digts than what
  is specified with F(#]
- Names can be shortened to it's unique prefix. For example January and Ja
  works fine.
- No error if the date string is shorter format_string and the next
  not used character is not a number.. This is useful to get a date
  from a mixed set of strings in date or datetime format.
  Oracle gives an error if date string is too short.
- MariaDB supports short locales as language names
- NLS_DATE_FORMAT can use both " and ' for quoting.
- NLS_DATE_FORMAT must be a constant string.
  - This is to ensure that the server knows which locale to use
    when executing the function.

New formats handled by TO_CHAR():
FF[1-6]    Fractional seconds
DDD        Daynumber 1-366
IW          Week 1-53 according to ISO 8601
I          1 digit year according to ISO 8601
IY          2 digit year according to ISO 8601
IYY        3 digit year according to ISO 8601
IYYY        4 digit year according to ISO 8601
SYYY        4 digit year according to ISO 8601 (Oracle can do signed)

Supported NLS_FORMAT_STRING options are:
NLS_CALENDAR=GREGORIAN
NLS_DATE_LANGUAGE=language

Support languages are:
- All MariaDB short locales, like en_AU.
- The following Oracle language names:
ALBANIAN, AMERICAN, ARABIC, BASQUE, BELARUSIAN, BRAZILIAN PORTUGUESE
BULGARIAN, CANADIAN FRENCH, CATALAN, CROATIAN, CYRILLIC SERBIAN CZECH,
DANISH, DUTCH, ENGLISH, ESTONIAN, FINNISH, FRENCH, GERMAN,
GREEK, HEBREW, HINDI, HUNGARIAN, ICELANDIC, INDONESIAN ITALIAN,
JAPANESE, KANNADA, KOREAN, LATIN AMERICAN SPANISH, LATVIAN,
LITHUANIAN, MACEDONIAN, MALAY, MEXICAN SPANISH, NORWEGIAN, POLISH,
PORTUGUESE, ROMANIAN, RUSSIAN, SIMPLIFIED CHINESE, SLOVAK, SLOVENIAN,
SPANISH, SWAHILI, SWEDISH, TAMIL, THAI, TRADITIONAL CHINESE, TURKISH,
UKRAINIAN, VIETNAMESE

Development bugs fixed:
MDEV-38403 Server crashes in Item_func_to_date::fix_length_and_dec upon
          using an invalid argument
MDEV-38400 compat/oracle.func_to_date fails with PS protocol and cursor
          protocol (Fixed by Serg)
MDEV-38404 TO_DATE: MTR coverage omissions, round 1
MDEV-38509 TO_DATE: AD_DOT does not appear to be supported
MDEV-38513 TO_DATE: NULL value for format string causes assertion failure
MDEV-38521 TO_DATE: Date strings with non-ASCII symbols cause warnings
          and wrong results
MDEV-38578 TO_DATE: Possibly unexpected results upon wrong input
MDEV-38582 TO_DATE: NLS_DATE_LANGUAGE=JAPANESE does not parse values
          which work in Oracle
MDEV-38584 TO_DATE: NLS_DATE_LANGUAGE=VIETNAMESE does not parse values
          which work in Oracle
MDEV-38703 TO_DATE: Quotation for multi-word NLS_DATE_LANGUAGE leads
          to syntax error in view definition
MDEV-38675 TO_DATE: MSAN/Valgrind/UBSAN errors in
          extract_oracle_date_time

Known issues:
- Format string character matches inside quotes are done
  one-letter-to-one-letter, like in LIKE predicate. That means things
  like expansions and contractions do not work.
  For example 'ss' does not match 'ß' in collations which treat them
  as equal for the comparison operator.
  Match is done taking into account case and accent sensitivity
  of the subject argument collation, so for example this now works:
  MariaDB [test]> SELECT TO_DATE('1920á12','YYYY"a"MM') AS c;
  +---------------------+
  | c                  |
  +---------------------+
  | 1920-12-17 00:00:00 |
  +---------------------+

Co-author and reviewer: Alexander Barkov <[email protected]>
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

Fix Windows and liburing issues.
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

Evict page on write completion if it's space was removed.

Lock external buffer pool file on Linux.
Monty
MDEV-38246 aria_read index failed on encrypted database during backup

The backup of encrypted Aria tables was not supported.
Added support for this. One complication is that the page checksum is
for the not encrypted page. To be able to verify the checksum I have to
temporarly decrypt the page.
In the backup we store the encrypted pages.

Other things:
- Fixed some (not critical) memory leaks in mariabackup
KhaledR57
MDEV-37072: Implement IS JSON predicate

Add support for the SQL standard IS JSON predicate with the syntax:
expr IS [ NOT ] JSON [ { VALUE | ARRAY | OBJECT | SCALAR } ]
[ { WITH | WITHOUT } UNIQUE [ KEYS ] ]

The predicate allows checking if an expression is valid JSON
and optionally constrains the JSON type (VALUE, ARRAY, OBJECT,
SCALAR) and whether object keys are unique.

The implementation includes:
- Basic IS JSON validation
- Support for NOT operator
- Type constraints (VALUE, ARRAY, OBJECT, SCALAR)
- Unique keys constraint (WITH/WITHOUT UNIQUE KEYS)
Michael Widenius
MDEV-19683 Add support for Oracle TO_DATE()

Syntax:
TO_DATE(string_expression [DEFAULT string_expression ON CONVERSION ERROR],
        format_string [,NLS_FORMAT_STRING])
The format_string has the same format elements as TO_CHAR(), except a
few elements that are not supported/usable for TO_DATE().
TO_DATE() returns a datetime or date value, depending on if the format
element FF is used.

Allowed separators, same as TO_CHAR():
space, tab and any of !#%'()*+,-./:;<=>

'&' can also be used if next character is not a character a-z or A-Z
"text' indicates a text string that is verbatim in the format. One cannot
use " as a separator.

Format elements supported by TO_DATE():
AD          Anno Domini ("in the year of the Lord")
AD_DOT      Anno Domini ("in the year of the Lord")
AM          Meridian indicator (Before midday)
AM_DOT      Meridian indicator (Before midday)
DAY        Name of day
DD          Day (1-31)
DDD        Day of year (1-336)
DY          Abbreviated name of day
FF[1-6]    Fractional seconds
HH          Hour (1-12)
HH12        Hour (1-12)
HH24        Hour (0-23)
MI          Minutes (0-59)
MM          Month (1-12)
MON        Abbreviated name of month
MONTH      Name of Month
PM          Meridian indicator (After midday)
PM_DOT      Meridian indicator (After midday)
RR          20th century dates in the 21st century. 2 digits
            50-99 is assumed from 2000, 0-49 is assumed from 1900.
RRRR        20th century dates in the 21st century. 4 digits
SS          Seconds
SYYYY      Signed 4 digit year; MariaDB only supports positive years
Y          1 digit year
YY          2 digits year
YYY        3 digits year
YYYY        4 digits year

Note that if there is a missing part of the date, the current date is used!
For example if 'MM-DD HH-MM-SS' then the current year will be used.
(Oracle behaviour)

Not supported options:
- BC, D, DL, DS, E, EE, FM, FX, RM, SSSSS, TS, TZD, TZH, TZR, X,SY
  BC is not supported by MariaDB datetime.
- Most of the other are exotic formats does not make sence in MariaDB as
  we return datetime or datetime with fractions, not string.
- D (day-of-week) is not supported as it is not clear exactly how it would
  map to MariaDB. This element depends on the NLS territory of the session.
- RR only works with 2 digit years (In Oracle RR can also work with 4
  digit years in some context but the rules are not clear).

Extensions / differences compared to Oracle;
- MariaDB supports FF (fractional seconds).  If FF[#] is used,
  then TO_DATE will return a datetime with # of subseconds.
  If FF is not used a datetime will be returned.
  There is warning (no error) if string contains more digts than what
  is specified with F(#]
- Names can be shortened to it's unique prefix. For example January and Ja
  works fine.
- No error if the date string is shorter format_string and the next
  not used character is not a number.. This is useful to get a date
  from a mixed set of strings in date or datetime format.
  Oracle gives an error if date string is too short.
- MariaDB supports short locales as language names
- NLS_DATE_FORMAT can use both " and ' for quoting.
- NLS_DATE_FORMAT must be a constant string.
  - This is to ensure that the server knows which locale to use
    when executing the function.

New formats handled by TO_CHAR():
FF[1-6]    Fractional seconds
DDD        Daynumber 1-366
IW          Week 1-53 according to ISO 8601
I          1 digit year according to ISO 8601
IY          2 digit year according to ISO 8601
IYY        3 digit year according to ISO 8601
IYYY        4 digit year according to ISO 8601
SYYY        4 digit year according to ISO 8601 (Oracle can do signed)

Supported NLS_FORMAT_STRING options are:
NLS_CALENDAR=GREGORIAN
NLS_DATE_LANGUAGE=language

Support languages are:
- All MariaDB short locales, like en_AU.
- The following Oracle language names:
ALBANIAN, AMERICAN, ARABIC, BASQUE, BELARUSIAN, BRAZILIAN PORTUGUESE
BULGARIAN, CANADIAN FRENCH, CATALAN, CROATIAN, CYRILLIC SERBIAN CZECH,
DANISH, DUTCH, ENGLISH, ESTONIAN, FINNISH, FRENCH, GERMAN,
GREEK, HEBREW, HINDI, HUNGARIAN, ICELANDIC, INDONESIAN ITALIAN,
JAPANESE, KANNADA, KOREAN, LATIN AMERICAN SPANISH, LATVIAN,
LITHUANIAN, MACEDONIAN, MALAY, MEXICAN SPANISH, NORWEGIAN, POLISH,
PORTUGUESE, ROMANIAN, RUSSIAN, SIMPLIFIED CHINESE, SLOVAK, SLOVENIAN,
SPANISH, SWAHILI, SWEDISH, TAMIL, THAI, TRADITIONAL CHINESE, TURKISH,
UKRAINIAN, VIETNAMESE

Development bugs fixed:
MDEV-38403 Server crashes in Item_func_to_date::fix_length_and_dec upon
          using an invalid argument
MDEV-38400 compat/oracle.func_to_date fails with PS protocol and cursor
          protocol (Fixed by Serg)
MDEV-38404 TO_DATE: MTR coverage omissions, round 1
MDEV-38509 TO_DATE: AD_DOT does not appear to be supported
MDEV-38513 TO_DATE: NULL value for format string causes assertion failure
MDEV-38521 TO_DATE: Date strings with non-ASCII symbols cause warnings
          and wrong results
MDEV-38578 TO_DATE: Possibly unexpected results upon wrong input
MDEV-38582 TO_DATE: NLS_DATE_LANGUAGE=JAPANESE does not parse values
          which work in Oracle
MDEV-38584 TO_DATE: NLS_DATE_LANGUAGE=VIETNAMESE does not parse values
          which work in Oracle
MDEV-38703 TO_DATE: Quotation for multi-word NLS_DATE_LANGUAGE leads
          to syntax error in view definition
MDEV-38675 TO_DATE: MSAN/Valgrind/UBSAN errors in
          extract_oracle_date_time

Known issues:
- Format string character matches inside quotes are done
  one-letter-to-one-letter, like in LIKE predicate. That means things
  like expansions and contractions do not work.
  For example 'ss' does not match 'ß' in collations which treat them
  as equal for the comparison operator.
  Match is done taking into account case and accent sensitivity
  of the subject argument collation, so for example this now works:
  MariaDB [test]> SELECT TO_DATE('1920á12','YYYY"a"MM') AS c;
  +---------------------+
  | c                  |
  +---------------------+
  | 1920-12-17 00:00:00 |
  +---------------------+

Co-author and reviewer: Alexander Barkov <[email protected]>
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

Added testing for:

1. Async read
2. Space removing during async io
3. IO error during async write

The case of io error during sync and async io should also be added.

Squash it.
KhaledR57
MDEV-37072: Implement IS JSON predicate

Add support for the SQL standard IS JSON predicate with the syntax:
expr IS [ NOT ] JSON [ { VALUE | ARRAY | OBJECT | SCALAR } ]
[ { WITH | WITHOUT } UNIQUE [ KEYS ] ]

The predicate allows checking if an expression is valid JSON
and optionally constrains the JSON type (VALUE, ARRAY, OBJECT,
SCALAR) and whether object keys are unique.

The implementation includes:
- Basic IS JSON validation
- Support for NOT operator
- Type constraints (VALUE, ARRAY, OBJECT, SCALAR)
- Unique keys constraint (WITH/WITHOUT UNIQUE KEYS)
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

Use persistent named files for external buffer pool instead of temporary
one.
Michael Widenius
MDEV-37674: Replace std::string with LEX_CSTRING in Optional_metadata_fields

- Less memory allocations (less fragmentation), less memory usage,
  faster performance, less code (both source and executed).
  - Added option used by RPL_TABLE_LIST::create_column_mapping() to
    only parse and allocate column names.
  - Avoid duplicate memory allocations of column names.
- Some protection for out-of-memory (when allocating field names).
  - More work is needed to remove all allocation problems in
    Optional_metadata_field().
- All allocated strings are ending with \0, which makes code safer.
Oleg Smirnov
MDEV-38129 Match probability
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

Some code review fixes.
Thirunarayanan Balathandayuthapani
MDEV-32067 InnoDB linear read ahead had better be logical

- Ported MDEV-32067 branch into main. Added a few interface
for multi-range read limit and InnoDB also aware of number
of pages to be read. Yet to integrate with buf_read_ahead_pages()
Sergei Petrunia
MDEV-36055 Allow left join reordering

(Based on a patch by Yuchen Pei)
When computing table dependencies in simplify_joins(), do not make
LEFT JOIN-ed table (or join nest) to be dependent on all preceding tables.

For queries in form

  t1
  LEFT JOIN t2 ON t2.col=t1.col
  LEFT JOIN t3 ON t3.col=t1.col

this allows the optimizer to construct join order of t1-t3-t2. Before
this patch, t1-t2-t3 was the only possible order.

Note that queries that use Oracle's Outer Join syntax (col1(+)=col2) are
converted into chained independent LEFT JOINs like the above.

The optimization is controlled by optimizer_switch='reorder_outer_joins=ON'

It is NOT enabled by default as it is known to expose problems with
join pruning. It is advised to set optimizer_prune_level=0 when setting
reorder_outer_joins=on.

Co-authored-by: Yuchen Pei <[email protected]>
Monty
MDEV-38435 Add Gtid_binlog_pos to SHOW MASTER STATUS

Other things:
- Extended mysqltest to write GTID's for master and slave if
  sync_slave_with_master fails.

Reviewer: Brandon Nesterenko <[email protected]>
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

Remove double-write buffer workaround.
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

In one of the practical cloud MariaDB setups, a server node accesses its
datadir over the network, but also has a fast local SSD storage for
temporary data. The content of such temporary storage is lost when the
server container is destroyed.

The commit uses this ephemeral fast local storage (SSD) as an extension of
the portion of InnoDB buffer pool (DRAM) that caches persistent data
pages. This cache is separated from the persistent storage of data files
and ib_logfile0 and ignored during backup.

The following system variables were introduced:

innodb_extended_buffer_pool_size - the size of external buffer pool
file, if it equals to 0, external buffer pool will not be used;

innodb_extended_buffer_pool_path - the path to external buffer pool
file.

If innodb_extended_buffer_pool_size is not equal to 0, external buffer
pool file will be created on startup.

Only clean pages will be flushed to external buffer pool file. There is
no need to flush dirty pages, as such pages will become clean after
flushing, and then will be evicted when they reach the tail of LRU list.

The general idea of this commit is to flush clean pages to external
buffer pool file when they are evicted.

A page can be evicted either by transaction thread or by background
thread of page cleaner. In some cases transaction thread is waiting for
page cleaner thread to finish its job. We can't do flushing in external
buffer pool file when transaction threads are waithing for eviction,
that would heart performance. That's why the only case for flushing is
when page cleaner thread evicts pages in background and there are no
waiters. For this purprose buf_pool_t::done_flush_list_waiters_count
variable was introduced, we flush evicted clean pages only if the
variable is zeroed.

Clean pages are evicted in buf_flush_LRU_list_batch() to keep some
amount of pages in buffer pool's free list. That's why we flush every
second page to external buffer pool file, otherwise there could be not
enought amount of pages in free list to let transaction threads to
allocate buffer pool pages without page cleaner waiting. This might be
not a good solution, but this is enought for prototyping.

External buffer pool page is introduced to store information in buffer
pool page hash about the certain page can be read from external buffer
pool file. The first several members of such page must be the same as the
members of internal page. External page frame must be equal to the
certain value to disthinguish external page from internal one. External
buffer pages are preallocated on startup in external pages array. We
could get rid of the frame in external page, and check if the page's
address belongs to the array to distinguish external and internal pages.

There are also external pages free and LRU lists. When some internal page
is decided to be flushed in external buffer pool file, a new external
page is allocated eighter from the head of external free list, or from
the tail of external LRU list. Both lists are protected with
buf_pool.mutex. It makes sense, because a page is removed from internal
LRU list during eviction under buf_pool.mutex.

Then internal page is locked and the allocated external page is attached
to io request for external buffer pool file, and when write request is
completed, the internal page is replaced with external one in page hash,
external page is pushed to the head of external LRU list and internal
page is unlocked. After internal page was removed from external free list,
it was not placed in external LRU, and placed there only
after write completion, so the page can't be used by the other threads
until write is completed.

Page hash chain get element function has additional template parameter,
which notifies the function if external pages must be ignored or not. We
don't ignore external pages in page hash in two cases, when some page is
initialized for read and when one is reinitialized for new page creating.

When an internal page is initialized for read and external page with the
same page id is found in page hash, the internal page is locked,
the external page in replaced with newly initialized internal page in the
page hash chain, the external page is removed from external LRU list and
attached to io request to external buffer pool file. When the io request
is completed, external page is returned to external free list,
internal page is unlocked. So during read external page absents in both
external LRU and free lists and can't be reused.

When an internal page is initialized for new page creating and external
pages with the same page id is found in page hash, we just remove external
page from the page hash chain and external LRU list and push it to the
head of external free list. So the external page can be used for future
flushing.

The pages are flushed to and read from external buffer pool file with
the same manner as they are flushed to their spaces, i.e. compressed and
encrypted pages stay compressed and encrypted in external buffer pool
file.
Brandon Nesterenko
MDEV-36290: Fix optional_metadata_len type mismatch

Parameter optional_meteradata_len to Optional_metadata_fields should
have been a size_t rather than uint.

Signed-off-by: Brandon Nesterenko <[email protected]>
Kristian Nielsen
Binlog-in-engine: Fix uninitialized function parameters

Don't pass uninitialized values in function call. MSAN complains about this
(even when the called function never accesses the uninitialized values, and
even when the function is constexpr .oO).

Signed-off-by: Kristian Nielsen <[email protected]>
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

Some code review fixes.
Thirunarayanan Balathandayuthapani
MDEV-32067 InnoDB linear read ahead had better be logical

- Ported MDEV-32067 branch into main. Added a few interface
for multi-range read limit and InnoDB also aware of number
of pages to be read. Yet to integrate with buf_read_ahead_pages()
Thirunarayanan Balathandayuthapani
MDEV-32067 InnoDB linear read ahead had better be logical

- Ported MDEV-32067 branch into main. Added a few interface
for multi-range read limit and InnoDB also aware of number
of pages to be read. Yet to integrate with buf_read_ahead_pages()
Marko Mäkelä
First steps towards multi-file parsing

log/log0recv.cc:3578: ut_ad(l + log_sys.is_encrypted() * 8 + 5 == el)
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

Some code review fixes. Needs RQG testing as some new assertions were
added and some conditions were changed.

The previous commit passed RQG testing.
Kristian Nielsen
Binlog-in-engine: Fix sporadic test failure of binlog_in_engine.purge_locking

The test was for some reason incorrectly doing SHOW BINLOG EVENTS when the
binlogging of the prior event is deliberately non-deterministic in which
binlog file it will appear in, causing test to depend on timing.

Signed-off-by: Kristian Nielsen <[email protected]>