Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Thirunarayanan Balathandayuthapani
MDEV-19194  ASAN use-after-poison in fk_prepare_copy_alter_table upon dropping FK

Problem:
========
The issue occurs when ALTER TABLE contains duplicate DROP FOREIGN KEY
operations (e.g., "DROP FOREIGN KEY f1, DROP FOREIGN KEY f1").
In fk_prepare_copy_alter_table(), Removes from the list all foreign
keys which are to be dropped. But iterator continues to accessing the
removed entry when it encounters duplicate foreign key.

Solution:
=========
mysql_prepare_alter_table(): Add duplicate detection logic to handle
cases where the same foreign key is specified multiple times in a
single ALTER TABLE statement's DROP statement.

fk_prepare_copy_alter_table(): fk_parent_key_list and fk_child_key_list
containing all existing foreign keys where the table acts as
parent or child respectively, which are populated through storage
engine calls. After this phase, the code iterates through these
collections comparing each foreign key against alter_info->drop_list
using case-insensitive string matching for constraint names and
database/table names.

- When matches are found, instead of removing them immediately,
the matching FOREIGN_KEY_INFO objects are added to the
keys_to_remove collection, which acts as a temporary holding area.
Once all matching keys are collected in keys_to_remove, use it to
remove the key entries from original lists.
Raghunandan Bhat
MDEV-36678: Various crashes upon stored procedure querying view with concat/group_concat

Problem:
  A stored procedure querying a view with `CONCAT`/`GROUP_CONCAT` could
  crash due to a NULL pointer dereference during query optimization.

  During condition pushdown, `Item_direct_ref_to_item::deep_copy`
  creates a clone where the clone's `ref` pointer incorrectly remains
  pointing to the original object's `m_item`, rather than its own.

  Because the clone is tethered to the original object, subsequent
  transformations happen on the original item instead of the clone.
  Calling `fix_fields` on malformed `Item_direct_ref_to_item` clone
  resolves to `Item_field::fix_fields` instead of `Item_ref::fix_fields`
  leaving the member `Item_ref::ref` uninitialized (NULL). When this is
  dereferenced in `Item_ref::const_item`, server crashes.

Fix:
  The `Item_direct_ref_to_item::set_item` method is made to update the
  item and reference, making the clone complete. This results in correct
  `fix_fields` resolution, making `Item_ref::ref` safe to access.
Abhishek Bansal
MDEV-38474: ASAN heap-use-after-free in st_select_lex_unit::cleanup

cleanup_stranded_units() was added at the start of
st_select_lex_unit::cleanup() by 34a8209d6657. This causes a
use-after-free when nested subqueries are merged into their parent
unit. With nested subqueries like:

  SELECT * FROM t1
  WHERE a IN (SELECT b FROM t2
              WHERE a IN (SELECT c FROM t3 WHERE FALSE HAVING c < 0));

the stranded_clean_list chains the units as: Unit1 -> Unit2 -> Unit3.
Because cleanup_stranded_units() was called first, Unit1->cleanup()
would recursively trigger Unit2->cleanup(), which in turn would
trigger Unit3->cleanup(). Unit3's cleanup frees its heap-allocated
join structures. But since Unit3 was merged into Unit2, Unit2 still
holds references to Unit3's structures (e.g., st_join_table). When
control returns to Unit2 for its own local cleanup, it accesses
already-freed memory.

Fix: move cleanup_stranded_units() to the end of cleanup(). This way,
each unit completes its own local cleanup first—clearing its
references to any child structures—before triggering cleanup of its
stranded (child) units. This enforces a parent-first cleanup order.
Marko Mäkelä
MDEV-38968 Redundant FILE_CHECKPOINT writes

Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records. Let us simplify the logic by making the dedicated
buf_flush_page_cleaner() thread responsible for checkpoints.

log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which replaces log_sys.next_checkpoint_lsn.

log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner
thread will write checkpoints, hence there is no possibility of a
race condition.

log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.

buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().

log_make_checkpoint(): Simply wrap buf_flush_sync_batch(0, true).

buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.

buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.

buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.

buf_flush_buffer_pool(): Remove.

buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.

logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.

fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.

innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().

lot_t::write_buf(): Remove a call set_check_for_checkpoint(false)
that commit 7443ad1c8a8437a761e1a2d3ba53f7c0ba0dd3bb (MDEV-32374)
had added. The flag should only be cleared when the checkpoint has
advanced far enough.

srv_undo_tablespaces_reinit(): Simplify the logic and remove the
fault injection point after_reinit_undo_abort, which would cause
a failure on Microsoft Windows. Changing innodb_undo_tablespaces is
not fully crash-safe.
Daniel Black
deb: fix no errmsg-utf8.txt
Marko Mäkelä
MDEV-38968 Redundant FILE_CHECKPOINT writes

Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records. Let us simplify the logic by making the dedicated
buf_flush_page_cleaner() thread responsible for checkpoints.

log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which replaces log_sys.next_checkpoint_lsn.

log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner
thread will write checkpoints, hence there is no possibility of a
race condition.

log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.

buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().

log_make_checkpoint(): Simply wrap buf_flush_sync_batch(0, true).

buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.

buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.

buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.

buf_flush_buffer_pool(): Remove.

buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.

logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.

fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.

innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().

lot_t::write_buf(): Remove a call set_check_for_checkpoint(false)
that commit 7443ad1c8a8437a761e1a2d3ba53f7c0ba0dd3bb (MDEV-32374)
had added. The flag should only be cleared when the checkpoint has
advanced far enough.

srv_undo_tablespaces_reinit(): Simplify the logic and remove the
fault injection point after_reinit_undo_abort, which would cause
a failure on Microsoft Windows. Changing innodb_undo_tablespaces is
not fully crash-safe.
Vladislav Vaintroub
MDEV-14443 DENY statement

Implements DENY/REVOKE DENY and associated tasks.
bsrikanth-mariadb
MDEV-38805:Store optimizer_context into a IS table

Currently, optimizer context is written as JSON sub-element in the
Optimizer Trace.

In this task, we separate it out from the optimizer trace, and instead
store it in optimizer_context Information Schema table.

The structure of the context is changed to look like below: -
----------------------------------
SET var_name1=val1;

SET var_name2=val2;
.
.
.

CREATE TABLE t1 ( ... );
-- in case it is a constant table
REPLACE INTO t1 VALUES (...);

CREATE TABLE t2 ( ... );
...

set @context='{ JSON with all the captured calls }';
set @optimizer_context='context';

... the original query;
----------------------------------

The IS can be used to read the current stored context, as well as to
dump it to a sql file which can later be replayed in a different
environment.

It is done like below: -
--------------------------------------
set optimizer_record_context=ON;
set optimizer_trace=1

-- sample query

select into outfile '/tmp/captured-context.sql' context from information_schema.OPTIMIZER_CONTEXT;
---------------------------------------

All the existing tests are modified to query OPTIMIZER_CONTEXT IS table
Aleksey Midenkov
MDEV-38987 ASAN heap-use-after-free upon using a view through a trigger in ORACLE mode

Outdated table_list->derived_result after first trigger run. It was
allocated on statment mem_root and freed when trigger procedure
finishes.

Next INSERT trigger on new view in mysql_derived_prepare() jumps to
exit on error before it had a chance to allocate new derived_result.

It jumps here:

797          if ((res= sl->handle_derived(lex, DT_PREPARE)))
798            goto exit;

and the allocation is later:

837      if ((!derived->is_with_table_recursive_reference() ||
838            !derived->derived_result) &&
839          !(derived->derived_result= new (thd->mem_root) select_unit(thd)))
840        DBUG_RETURN(TRUE); // out of memory

on exit it does destruction of already freed object from previous statement:

901    exit:
...
921      if (res)
922      {
923        if (!derived->is_with_table_recursive_reference())
924        {
...
927          delete derived->derived_result;

The most obvious fix is to cleanup derived_result when the TABLE_LIST
is reinited for new use. Also as a precaution the patch does the
cleanup after delete operator.
Vladislav Vaintroub
MDEV-14443 DENY statement

Implements DENY/REVOKE DENY and associated tasks.
Alexander Barkov
MDEV-19635 System package SYS.DBMS_SQL

In progress
Abhishek Bansal
MDEV-38033: JSON_SCHEMA_VALID returns wrong result for array of objects

Fix a bug in Json_schema_items::validate where it was missing a
json_skip_level() call. Without this skip, the validator would
incorrectly recurse into non-scalar array elements (like objects)
and try to validate their internal keys/values against the array's
item schema.
Alexander Barkov
MDEV-38933 Implement basic low level functions for DBMS_SQL

Adding SQL functions:
- DBMS_SQL_COMMAND(ps_name)
- DBMS_SQL_BIND_PARAM_BY_NAME(ps_name,pos,value)
- DBMS_SQL_COLUMN_VALUE(c,pos,dst)
Marko Mäkelä
MDEV-38968 Redundant FILE_CHECKPOINT writes

Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records. Let us simplify the logic by making the dedicated
buf_flush_page_cleaner() thread responsible for checkpoints.

log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which replaces log_sys.next_checkpoint_lsn.

log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner
thread will write checkpoints, hence there is no possibility of a
race condition.

log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.

buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().

log_make_checkpoint(): Delegate all work to buf_flush_page_cleaner().

buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.

buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.

buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.

buf_flush_buffer_pool(): Remove.

buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.

logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.

fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.

innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().
Daniel Black
MDEV-20277 remove -fno-strict-aliasing

Incorrect typing can cause inefficient code and
optimizations. The optimization of undefined behaviour
can result in server crashes. Its better to know
about our violations.

Revert "Bug#42733: Type-punning warnings when compiling MySQL -- strict aliasing violations."

This reverts commit c192be206ae36398804b473f7b0e4ee4431b71da.
Hemant Dangi
MDEV-38916: Galera test failure on galera_3nodes.MDEV-36360

Issue:
wsrep_slave_FK_checks is deprecated(MDEV-38787) and has no effect,
while applier threads are forced to run with FK checks enabled.
The MDEV-36360 test expects that no FK constraint failures appear
in the error log, which does not happen because of
wsrep_slave_FK_checks has no effect and so test fails.

Solution:
Remove the assert_grep.inc block so the test remains as a check
for table-level X-lock / deadlock behavior only.
Marko Mäkelä
MDEV-38968 Redundant FILE_CHECKPOINT writes

Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records. Let us simplify the logic by making the dedicated
buf_flush_page_cleaner() thread responsible for checkpoints.

log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which replaces log_sys.next_checkpoint_lsn.

log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner
thread will write checkpoints, hence there is no possibility of a
race condition.

log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.

buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().

log_make_checkpoint(): Simply wrap buf_flush_sync_batch(0, true).

buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.

buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.

buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.

buf_flush_buffer_pool(): Remove.

buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.

logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.

fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.

innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().

lot_t::write_buf(): Remove a call set_check_for_checkpoint(false)
that commit 7443ad1c8a8437a761e1a2d3ba53f7c0ba0dd3bb (MDEV-32374)
had added. The flag should only be cleared when the checkpoint has
advanced far enough.

srv_undo_tablespaces_reinit(): Simplify the logic and remove the
fault injection point after_reinit_undo_abort, which would cause
a failure on Microsoft Windows. Changing innodb_undo_tablespaces is
not fully crash-safe.
Vladislav Vaintroub
MDEV-14443 DENY statement

Implements DENY/REVOKE DENY and associated tasks.
Dave Gosselin
MDEV-38693: FULL JOIN Algorithm, Phase 1

This is a WIP for the FULL JOIN Algorithm and is not inteded for
delivery into the main branch.  It's foundational work for
investigating the overall FULL JOIN algorithm which is coming in
phases 2 and 3.

This is a hacky algorithm for FULL OUTER JOIN when we have two base
tables and will not scale beyond that case: basically, in do_select,
we run the join "forwards" (for the LEFT JOIN) to get LEFT JOIN
results for the FULL JOIN and to keep track of emitted rows.  Then,
without touching the JOIN_TAB layout in memory, we run the join
"backwards" (to get RIGHT JOIN results): we swap the select condition
between the two JOIN_TABs, reset the left and right tables’ to begin
reading from their beginnings, swap the read_first_record and
next_select funcs, and set a “direction” flag on the JOIN. Then,
anywhere in sub_select or evaluate_join_record where we need to move
to the next JOIN_TAB, we look at the direction and move accordingly
(“to the right” for LEFT JOIN rows, “to the left” for RIGHT JOIN
rows). This produces correct results for simple examples like
  SELECT * FROM t1 FULL JOIN t2 ON t1.a=t2.a.
However, it is quite obvious this won’t work in general.
Marko Mäkelä
MDEV-38968 Redundant FILE_CHECKPOINT writes

Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records. Let us simplify the logic by making the dedicated
buf_flush_page_cleaner() thread responsible for checkpoints.

log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which replaces log_sys.next_checkpoint_lsn.

log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner
thread will write checkpoints, hence there is no possibility of a
race condition.

log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.

buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().

log_make_checkpoint(): Simply wrap buf_flush_sync_batch(0, true).

buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.

buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.

buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.

buf_flush_buffer_pool(): Remove.

buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.

logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.

fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.

innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().

lot_t::write_buf(): Remove a call set_check_for_checkpoint(false)
that commit 7443ad1c8a8437a761e1a2d3ba53f7c0ba0dd3bb (MDEV-32374)
had added. The flag should only be cleared when the checkpoint has
advanced far enough.

srv_undo_tablespaces_reinit(): Simplify the logic.
bsrikanth-mariadb
MDEV-38805:Store optimizer_context into a IS table

Currently, optimizer context is written as JSON sub-element in the
Optimizer Trace.

In this task, we separate it out from the optimizer trace, and instead
store it in optimizer_context Information Schema table.

The structure of the context is changed to look like below: -
----------------------------------
SET var_name1=val1;

SET var_name2=val2;
.
.
.

CREATE TABLE t1 ( ... );
-- in case it is a constant table
REPLACE INTO t1 VALUES (...);

CREATE TABLE t2 ( ... );
...

set @context='{ JSON with all the captured calls }';
set @optimizer_context='context';

... the original query;
----------------------------------

The IS can be used to read the current stored context, as well as to
dump it to a sql file which can later be replayed in a different
environment.

It is done like below: -
--------------------------------------
set optimizer_record_context=ON;
set optimizer_trace=1

-- sample query

select into outfile '/tmp/captured-context.sql' context from information_schema.OPTIMIZER_CONTEXT;
---------------------------------------

All the existing tests are modified to query OPTIMIZER_CONTEXT IS table
Abhishek Bansal
MDEV-38033: JSON_SCHEMA_VALID returns wrong result for array of objects

Fix a bug in Json_schema_items::validate where it was missing a
json_skip_level() call. Without this skip, the validator would
incorrectly recurse into non-scalar array elements (like objects)
and try to validate their internal keys/values against the array's
item schema.
Alexey Botchkov
MDEV-37262 XMLTYPE: validation.
forkfun
MDEV-38967: STR_TO_DATE() cannot be used in indexed virtual columns

Update STR_TO_DATE() to accept an optional third argument for the locale.
DATE_FORMAT() and STR_TO_DATE() return NULL when the locale argument is NULL;
issue a warning and fall back to the session @@lc_time_names for invalid locale names.
STR_TO_DATE() can now be used in indexed virtual columns while having all 3 arguments,
but not allowed when 2 as it depends then on the session @@lc_time_names
Marko Mäkelä
MDEV-38968 Redundant FILE_CHECKPOINT writes

Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records. Let us simplify the logic by making the dedicated
buf_flush_page_cleaner() thread responsible for checkpoints.

log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which replaces log_sys.next_checkpoint_lsn.

log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner
thread will write checkpoints, hence there is no possibility of a
race condition.

log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.

buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().

log_make_checkpoint(): Delegate all work to buf_flush_page_cleaner().

buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.

buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.

buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.

buf_flush_buffer_pool(): Remove.

buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.

logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.

fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.

innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().

srv_undo_tablespaces_reinit(): Simplify the logic and remove the
fault injection point after_reinit_undo_abort, which would cause
a failure on Microsoft Windows. Changing innodb_undo_tablespaces is
not fully crash-safe.
bsrikanth-mariadb
MDEV-38805:Store optimizer_context into a IS table

Currently, optimizer context is written as JSON sub-element in the
Optimizer Trace.

In this task, we separate it out from the optimizer trace, and instead
store it in optimizer_context Information Schema table.

The structure of the context is changed to look like below: -
----------------------------------
SET var_name1=val1;

SET var_name2=val2;
.
.
.

CREATE TABLE t1 ( ... );
-- in case it is a constant table
REPLACE INTO t1 VALUES (...);

CREATE TABLE t2 ( ... );
...

set @context='{ JSON with all the captured calls }';
set @optimizer_context='context';

... the original query;
----------------------------------

The IS can be used to read the current stored context, as well as to
dump it to a sql file which can later be replayed in a different
environment.

It is done like below: -
--------------------------------------
set optimizer_record_context=ON;
set optimizer_trace=1

-- sample query

select into outfile '/tmp/captured-context.sql' context from information_schema.OPTIMIZER_CONTEXT;
---------------------------------------

All the existing tests are modified to query OPTIMIZER_CONTEXT IS table
Marko Mäkelä
MDEV-38968 Redundant FILE_CHECKPOINT writes

Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records. Let us simplify the logic by making the dedicated
buf_flush_page_cleaner() thread responsible for checkpoints.

log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which replaces log_sys.next_checkpoint_lsn.

log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner
thread will write checkpoints, hence there is no possibility of a
race condition.

log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.

buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().

log_make_checkpoint(): Simply wrap buf_flush_sync_batch(0, true).

buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.

buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.

buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.

buf_flush_buffer_pool(): Remove.

buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.

logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.

fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.

innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().

lot_t::write_buf(): Remove a call set_check_for_checkpoint(false)
that commit 7443ad1c8a8437a761e1a2d3ba53f7c0ba0dd3bb (MDEV-32374)
had added. The flag should only be cleared when the checkpoint has
advanced far enough.

srv_undo_tablespaces_reinit(): Simplify the logic and remove the
fault injection point after_reinit_undo_abort, which would cause
a failure on Microsoft Windows. Changing innodb_undo_tablespaces is
not fully crash-safe.
bsrikanth-mariadb
MDEV-38805:Store optimizer_context into a IS table

Currently, optimizer context is written as JSON sub-element in the
Optimizer Trace.

In this task, we separate it out from the optimizer trace, and instead
store it in optimizer_context Information Schema table.

The structure of the context is changed to look like below: -
----------------------------------
SET var_name1=val1;

SET var_name2=val2;
.
.
.

CREATE TABLE t1 ( ... );
-- in case it is a constant table
REPLACE INTO t1 VALUES (...);

CREATE TABLE t2 ( ... );
...

set @context='{ JSON with all the captured calls }';
set @optimizer_context='context';

... the original query;
----------------------------------

The IS can be used to read the current stored context, as well as to
dump it to a sql file which can later be replayed in a different
environment.

It is done like below: -
--------------------------------------
set optimizer_record_context=ON;
set optimizer_trace=1

-- sample query

select into outfile '/tmp/captured-context.sql' context from information_schema.OPTIMIZER_CONTEXT;
---------------------------------------

All the existing tests are modified to query OPTIMIZER_CONTEXT IS table
bsrikanth-mariadb
MDEV-38805:Store optimizer_context into a IS table

Currently, optimizer context is written as JSON sub-element in the
Optimizer Trace.

In this task, we separate it out from the optimizer trace, and instead
store it in optimizer_context Information Schema table.

The structure of the context is changed to look like below: -
----------------------------------
SET var_name1=val1;

SET var_name2=val2;
.
.
.

CREATE TABLE t1 ( ... );
-- in case it is a constant table
REPLACE INTO t1 VALUES (...);

CREATE TABLE t2 ( ... );
...

set @context='{ JSON with all the captured calls }';
set @optimizer_context='context';

... the original query;
----------------------------------

The IS can be used to read the current stored context, as well as to
dump it to a sql file which can later be replayed in a different
environment.

It is done like below: -
--------------------------------------
set optimizer_record_context=ON;
set optimizer_trace=1

-- sample query

select into outfile '/tmp/captured-context.sql' context from information_schema.OPTIMIZER_CONTEXT;
---------------------------------------

All the existing tests are modified to query OPTIMIZER_CONTEXT IS table
forkfun
MDEV-38967: STR_TO_DATE() cannot be used in indexed virtual columns

Update STR_TO_DATE() to accept an optional third argument for the locale.
DATE_FORMAT() and STR_TO_DATE() return NULL when the locale argument is NULL;
issue a warning and fall back to the session @@lc_time_names for invalid locale names.
STR_TO_DATE() can now be used in indexed virtual columns while having all 3 arguments,
but not allowed when 2 as it depends then on the session @@lc_time_names
Marko Mäkelä
MDEV-38968 Redundant FILE_CHECKPOINT writes

Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records. Let us simplify the logic by making the dedicated
buf_flush_page_cleaner() thread responsible for checkpoints.

log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which replaces log_sys.next_checkpoint_lsn.

log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner
thread will write checkpoints, hence there is no possibility of a
race condition.

log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.

buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().

log_make_checkpoint(): Delegate all work to buf_flush_page_cleaner().

buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.

buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.

buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.

buf_flush_buffer_pool(): Remove.

buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.

logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.

fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.

innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().

srv_undo_tablespaces_reinit(): Simplify the logic and remove the
fault injection point after_reinit_undo_abort, which would cause
a failure on Microsoft Windows. Changing innodb_undo_tablespaces is
not fully crash-safe.
Marko Mäkelä
MDEV-38968 Redundant FILE_CHECKPOINT writes

Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records. Let us simplify the logic by making the dedicated
buf_flush_page_cleaner() thread responsible for checkpoints.

log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which replaces log_sys.next_checkpoint_lsn.

log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner
thread will write checkpoints, hence there is no possibility of a
race condition.

log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.

buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().

log_make_checkpoint(): Delegate all work to buf_flush_page_cleaner().

buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.

buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.

buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.

buf_flush_buffer_pool(): Remove.

buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.

logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.

fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.

innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().

srv_undo_tablespaces_reinit(): Simplify the logic and remove the
fault injection point after_reinit_undo_abort, which would cause
a failure on Microsoft Windows. Changing innodb_undo_tablespaces is
not fully crash-safe.
Daniel Black
fix rpm
Hemant Dangi
MDEV-38916: Galera test failure on galera_3nodes.MDEV-36360

Issue:
wsrep_slave_FK_checks is deprecated(MDEV-38787) and has no effect,
while applier threads are forced to run with FK checks enabled.
The MDEV-36360 test expects that no FK constraint failures appear
in the error log, which does not happen because of
wsrep_slave_FK_checks has no effect and so test fails.

Solution:
Remove the assert_grep.inc block so the test remains as a check
for table-level X-lock / deadlock behavior only.
Marko Mäkelä
MDEV-38968 Redundant FILE_CHECKPOINT writes

Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records. Let us simplify the logic by making the dedicated
buf_flush_page_cleaner() thread responsible for checkpoints.

log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which replaces log_sys.next_checkpoint_lsn.

log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner
thread will write checkpoints, hence there is no possibility of a
race condition.

log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.

buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().

log_make_checkpoint(): Delegate all work to buf_flush_page_cleaner().

buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.

buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.

buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.

buf_flush_buffer_pool(): Remove.

buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.

logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.

fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.

innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().
Raghunandan Bhat
MDEV-38864: Use mmap for MEMORY engine allocations

MEMORY engine's data and indexes blocks are allocated using `my_malloc`.
This created internal fragmentation within the system allocator, causing
gradual memory growth and leading to OOM server crash.

This patch introduces memory mapping for MEMORY engine's allocations.
- Implements `my_vmalloc` and `my_vmrelease` functions.
- These functions wrap `my_virtual_mem_alloc` & `my_virtual_mem_release`
  to provide error handling, interface with Performance Schema(PSI) and
  global memory accounting.
Marko Mäkelä
MDEV-38968 Redundant FILE_CHECKPOINT writes

Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records. Let us simplify the logic by making the dedicated
buf_flush_page_cleaner() thread responsible for checkpoints.

log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which replaces log_sys.next_checkpoint_lsn.

log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner
thread will write checkpoints, hence there is no possibility of a
race condition.

log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.

buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().

log_make_checkpoint(): Delegate all work to buf_flush_page_cleaner().

buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.

buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.

buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.

buf_flush_buffer_pool(): Remove.

buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.

logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.

fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.

innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().
Marko Mäkelä
MDEV-38968 Redundant FILE_CHECKPOINT writes

Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records. Let us simplify the logic by making the dedicated
buf_flush_page_cleaner() thread responsible for checkpoints.

log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which replaces log_sys.next_checkpoint_lsn.

log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner
thread will write checkpoints, hence there is no possibility of a
race condition.

log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.

buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().

log_make_checkpoint(): Simply wrap buf_flush_sync_batch(0, true).

buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.

buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.

buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.

buf_flush_buffer_pool(): Remove.

buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.

logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.

fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.

innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().

lot_t::write_buf(): Remove a call set_check_for_checkpoint(false)
that commit 7443ad1c8a8437a761e1a2d3ba53f7c0ba0dd3bb (MDEV-32374)
had added. The flag should only be cleared when the checkpoint has
advanced far enough.

srv_undo_tablespaces_reinit(): Simplify the logic and remove the
fault injection point after_reinit_undo_abort, which would cause
a failure on Microsoft Windows. Changing innodb_undo_tablespaces is
not fully crash-safe.
Alexander Barkov
MDEV-38933 Implement basic low level functions for DBMS_SQL

Part1:

OPEN c0 FOR PREPARE stmt;
OPEN c0 FOR LOCAL spvar_with_ps_name;
PREPARE LOCAL spvar_with_ps_name FROM 'dynamic sql';
EXECUTE LOCAL spvar_with_ps_name;
DEALLOCATE PREPARE LOCAL spvar_with_ps_name;
forkfun
MDEV-38967: STR_TO_DATE() cannot be used in indexed virtual columns

Update STR_TO_DATE() to accept an optional third argument for the locale.
DATE_FORMAT() and STR_TO_DATE() return NULL when the locale argument is NULL;
issue a warning and fall back to the session @@lc_time_names for invalid locale names.
STR_TO_DATE() can now be used in indexed virtual columns while having all 3 arguments,
but not allowed when 2 as it depends then on the session @@lc_time_names