Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Oleksandr Byelkin
Merge branch '11.8' into 12.3
bsrikanth-mariadb
MDEV-39226: Push whole multi-table update/delete down into engines

Give storage engines a way to take over an entire multi-table
UPDATE/DELETE, the way they can already take over a SELECT. Without it the
join, the row matching and every modification run in the SQL layer even
when an engine could do the whole statement itself in one step; a
single-table UPDATE/DELETE already avoids this via
direct_update_rows()/direct_delete_rows(), but a multi-table statement has
no primary handler object to drive that path.

This adds a generic, engine-agnostic pushdown interface: the SQL layer
offers the statement to the engine, and if the engine accepts it, it
performs the whole thing and reports only the row counts.

- Split select_handler into a pushdown_handler base with select_handler
  (result set) and a new multi_upddel_handler (runs a whole UPDATE/DELETE,
  reports row counts, reported as PUSHED UPDATE/PUSHED DELETE); add
  handlerton::create_multi_upddel, looked up in Sql_cmd_dml::execute_inner().
- multi_update/multi_delete gain direct_update_delete_done(), which records
  the engine's counts so send_eof() binlogs and replies without the
  SQL-layer loop; it forces statement-format binlogging so the change still
  replicates under binlog_format=ROW, and errors out instead of silently
  dropping counts for an unsupported result object.
- FederatedX implements the interface as the reference engine used to test
  correctness: it prints the statement back and runs it remotely, passes
  the engine's error code/SQLSTATE through, reads the matched count from the
  remote info string, executes IGNORE locally, and only pushes down when all
  tables share one remote server (same as SELECT/derived/unit pushdown).

Test: federated.federatedx_pushdown_upd_del.
Marko Mäkelä
WIP MDEV-40791: Less aggressive flushing

log_t::set_archive(): When a checkpoint needs to be forced, set a
minimum target that guarantees progress, instead of flushing the
entire buffer pool.

InnoDB_backup::end(): Add temporary diagnostics

FIXME: Make log_sys.circular_recovery_from_sequence_bit hold less often.
PranavKTiwari
modified logic.
Alessandro Vetere
MDEV-40408 btr_page_reorganize_low() uses the buffer pool just to obtain a scratch block

Add buf_pool.scratch_buf, a pool of page frames that the page
reorganization operations use instead of taking a block from the global
buffer pool.

buf_pool_t::scratch_buffer: A singly linked list of chunks of
buf_tmp_buffer_t slots. A thread holds at most one slot at a time, and
it holds a page latch while doing so, which is why reserve() must not
wait for the buffer pool or for I/O. When every slot is in use,
reserve() appends a chunk that holds twice the slots of the last one, up
to a limit on the size of one chunk. Only grow() waits, on the mutex
that serializes it and on the allocator. A chunk is never moved or freed
before close(), so a thread can keep using the slot that it reserved
while another thread appends a chunk. Debug builds start with a single
slot, so that a second concurrent page reorganization exercises grow().

buf_pool_t::scratch_buffer::shrink(): Free the page frames of the slots
that are not in use. The slots and the chunks are kept, because a slot
costs 32 bytes while a page frame costs srv_page_size. A frame survives
the first pass, because that pass only clears the used flag. The master
thread calls this often while the server is idle and rarely while it is
active, and buf_pool_t::garbage_collect() calls it under memory
pressure, where it ignores the used flag, because releasing these frames
is much cheaper than shrinking the buffer pool.

buf_pool_t::io_buf_t::acquire(): Factor out the scan for an unreserved
slot, which io_buf_t::reserve() ran twice and the scratch buffer reuses.

btr_page_reorganize_low(), page_zip_reorganize(): Obtain the scratch
page frame from buf_pool.scratch_buf. This removes the buf_pool.mutex
acquisition and the free block wait that buf_block_alloc() could
perform while at least a page X-latch was being held.
btr_page_reorganize_low() also used to leak the block on its error
paths; a single exit now releases the slot.

page_copy_rec_list_end_no_locks(), lock_move_reorganize_page(): Take
the source page frame instead of a source buf_block_t, because the
source is no longer a buffer pool block. In
page_copy_rec_list_end_no_locks() the source page is page_align(rec) at
every call site, so only the record is passed.

buf_tmp_buffer_t::acquire(), buf_tmp_buffer_t::release(): Use acquire
and release memory ordering, so that the page frame pointer of a slot
is published to the next thread that reserves the slot. acquire() also
reads the flag before the exchange, so that a scan across an array of
slots does not write to the slots that it finds reserved. release()
also marks the page frame undefined for Valgrind and MSan, because the
frame stays allocated for the next reserver and no deallocation marks
it.
Marko Mäkelä
MDEV-40791 SET GLOBAL innodb_log_archive=OFF triggers full flush

log_t::set_archive(): When a checkpoint needs to be forced, set a
minimum target that guarantees progress, instead of flushing
the entire buffer pool. Remove some duplicated reads of
last_checkpoint_lsn.

When switching innodb_log_archive from OFF to ON,
accurately remember whether any log records may have been
written with get_sequence_bit() == 0, to ensure that a
checkpoint will be waited for on a subsequent switch
from ON to OFF.

log_t::set_recovered(): If innodb_log_archive=ON, only make a
future set_archive(false) trigger a checkpoint if we may have
recovered a server that was killed between set_archive(true)
and write_checkpoint().
Monty
MDEV-40776 Atomic CREATE OR REPLACE silently breaks the foreign key

Give an error if one tries to drop a table referenced by a foreign keys
This is needed as innodb will keep the reference to the origina table
even when it is renamed to a temporary name as part of create or replace.

Other things:
- Changed the error message for  ER_TRUNCATE_ILLEGAL_FK to say
  "Cannot drop or truncate a table ..."
Brandon Nesterenko
MDEV-40768: Slave cannot apply a fragmented row event written with log_bin_compress=ON

MDEV-32570 added fragmentation of large row events. When binlog
compression is enabled (log_bin_compress=ON), the compression is
bypassed, and these binary log events remain as regular row events, when
they should be compressed row events.

This is because the compression happens during the Log_event::write()
function, whereas row data fragmentation happens before
Log_event::write() is called. The Rows_log_event super-class stores the
row data buffers to be written to disk. The regular Rows_log_event
sub-classes's implementations of Log_event::write() write this data
as-is. The compressed sub-classes's implementation of ::write()
over-write these buffers with the compressed rows data before writing to
disk.

To fragment a large row event, the server fragments the Rows_log_event's
row data buffers, to be written by multiple Partial_rows_log_event's,
and each Partial_rows_log_event::write() call writes its portion of the
row data buffer to disk. However, this happens before compression ever
has a chance to take place, and thereby, an event that should be
compressed, never is.

MDEV-39762 added event structure validation for compressed events, and
discovered that these events carry the compressed event type, but are
not actually compressed. Event validation thereby fails.

This patch adds a workaround to override the event type of the
fragmented row events to be regular row events, to be consistent with
the actual on-disk content. The underlying problem still needs to be
addressed though, and is tracked by MDEV-40851.

Signed-off-by: Brandon Nesterenko <[email protected]>
Marko Mäkelä
squash! 43946bab13fe78cdc3f55f0b2f2af887dfceddb3

log_t::backup_start(): If we were running with innodb_log_archive=ON,
ensure that the latest file is a valid recovery starting point.
That is, wait for the latest log checkpoint to be within the file.
Oleksandr Byelkin
Merge branch 'br-11.8-merge' into bb-12.3-release-bad
Yuchen Pei
MDEV-40805 Do not call lock_rec_convert_impl_to_expl if a table S-lock is held

lock_clust_rec_read_check_and_lock() skipped the implicit-to-explicit
conversion only under a table LOCK_X. When a table LOCK_S is held the
conversion is equally pointless: no other transaction can hold an
implicit X-lock on the record, because modifying a row requires a
table LOCK_IX and LOCK_IX is incompatible with our LOCK_S.
lock_table_has() matches stronger modes, so testing LOCK_S subsumes
the old LOCK_X test.
Dmitry Shulga
ASAN: heap-use-after-free with concurrent create/drop system trigger

In case there are 'on shutdown' triggers it could result in abnormal
server termination if at the moment server shutdown is in progress
the server received the DROP TRIGGER statement for one of 'ON SHUTDOWN'
system triggers being already executed as part shutdown process. Another
words, there is the race condition between running triggers on shutdown
and execution of DROP TRIGGER for system triggers ON SHUTDOWN event.

To fix the issue protect running on shutdown triggers and drop/create
of system triggers under the lock to avoid race condition.
Check under the new lock for the flag that shutdown is in progress and
don't add/remove a trigger instance into/from internal array as part of
handling CREATE/DROP TRIGGER for ON SHUTDOWN event. That is, add/drop
metadata about the trigger but don't modify the internal runtime data
structures.
Sergei Golubchik
MDEV-40445 TLS 1.3 early data (wolfssl)

Compiles with WolfSSL but hits protocol errors.
Disable 0-RTT for WolfSSL for now - compile it off, simply
setting SSL_CTX_set_max_early_data(ssl, 0) is not enough.
PranavKTiwari
MDEV-38633: Row events in statement based binlog: optimization possible?
A failing multi-table UPDATE or DELETE marked every target temporary table as not up to date in the binary log, even when nothing had been changed.
Any later statement reading such a table was then forced to use row logging.
Only mark the tables when something was actually changed—that is, when rows were updated/deleted or a non-transactional table was modified.
If nothing changed, set THD::tmp_table_binlog_handled so that mark_tmp_table_as_free_for_reuse() does not mark them either.
Yuchen Pei
MDEV-40486 [fixup] Clamp max_length at MAX_FIELD_VARCHARLENGTH in Item_func_vec_fromtext::fix_length_and_dec

And move the length check in Item_func_vec_fromtext::val_str to later

This allows

create table t1 (v vector(64) not null);
insert into t1 select vec_fromtext(concat('[',group_concat(1),']')) from seq_1_to_64;

which was banned in the previous fix
bb0ac437015dec04fbee226745a8eb2bb4825917, though this also introduces
the inconsistency(?) where

create table t1 as select vec_fromtext(concat('[',group_concat(1),']')) from seq_1_to_64;

still fails ER_TRUNCATED_WRONG_VALUE
Oleksandr Byelkin
Merge branch '12.3' into br-12.3-merge
Marko Mäkelä
squash! e514bdd3f0d99f28c07404b7841b2d14b2081219

log_t::set_archive(): When switching from OFF to ON,
accurately remember whether any log records may have been
written with get_sequence_bit() == 0, to ensure that a
checkpoint will be waited for on a subsequent switch
from ON to OFF. Remove some duplicated reads of
last_checkpoint_lsn.

log_t::set_recovered(): Correct a comment.
Marko Mäkelä
fixup! 8f20451a983d6b579527ca7dda3f17747855905e

(cherry picked from commit 68683b7cfb3e978a43e17855716bad0ae32c9cfd)
Brandon Nesterenko
MDEV-40768: Slave cannot apply a fragmented row event written with log_bin_compress=ON

MDEV-32570 added fragmentation of large row events. When binlog
compression is enabled (log_bin_compress=ON), the compression is
bypassed, and these binary log events remain as regular row events, when
they should be compressed row events.

This is because the compression happens during the Log_event::write()
function, whereas row data fragmentation happens before
Log_event::write() is called. The Rows_log_event super-class stores the
row data buffers to be written to disk. The regular Rows_log_event
sub-classes's implementations of Log_event::write() write this data
as-is. The compressed sub-classes's implementation of ::write()
over-write these buffers with the compressed rows data before writing to
disk.

To fragment a large row event, the server fragments the Rows_log_event's
row data buffers, to be written by multiple Partial_rows_log_event's,
and each Partial_rows_log_event::write() call writes its portion of the
row data buffer to disk. However, this happens before compression ever
has a chance to take place, and thereby, an event that should be
compressed, never is.

MDEV-39762 added event structure validation for compressed events, and
discovered that these events carry the compressed event type, but are
not actually compressed. Event validation thereby fails.

This patch adds a workaround to override the event type of the
fragmented row events to be regular row events, to be consistent with
the actual on-disk content. The underlying problem still needs to be
addressed though, and is tracked by MDEV-40851.

Signed-off-by: Brandon Nesterenko <[email protected]>
Marko Mäkelä
Temporary diagnostics for crude profiling
Marko Mäkelä
MDEV-40791 SET GLOBAL innodb_log_archive=OFF triggers full flush

log_t::set_archive(): When a checkpoint needs to be forced, set a
minimum target that guarantees progress, instead of flushing
the entire buffer pool. Remove some duplicated reads of
last_checkpoint_lsn.

When switching innodb_log_archive from OFF to ON,
accurately remember whether any log records may have been
written with get_sequence_bit() == 0, to ensure that a
checkpoint will be waited for on a subsequent switch
from ON to OFF.

log_t::set_recovered(): If innodb_log_archive=ON, only make a
future set_archive(false) trigger a checkpoint if we may have
recovered a server that was killed between set_archive(true)
and write_checkpoint().
Yuchen Pei
MDEV-40751 Make sure that VEC_FROMTEXT results in a length of multiple of 4
Yuchen Pei
MDEV-40805 Do not call lock_rec_convert_impl_to_expl if a table S-lock is held

lock_clust_rec_read_check_and_lock() skipped the implicit-to-explicit
conversion only under a table LOCK_X. When a table LOCK_S is held the
conversion is equally pointless: no other transaction can hold an
implicit X-lock on the record, because modifying a row requires a
table LOCK_IX and LOCK_IX is incompatible with our LOCK_S.
lock_table_has() matches stronger modes, so testing LOCK_S subsumes
the old LOCK_X test.
Marko Mäkelä
fixup! a2020ca6c21691a3727fa2e6a5b8524242e93836
Sergei Golubchik
cleanup: revert now-obsolete wolfssl workaround

was added in d510f8054912 for 5.5.4, now it only causes

In file included from extra/wolfssl/wolfssl/wolfcrypt/src/aes_asm.S:45:
extra/wolfssl/user_settings.h:84:9: warning: 'WOLFSSL_X86_64_BUILD' redefined
  84 | #define WOLFSSL_X86_64_BUILD
Alexander Barkov
Cherry-pick from 12.3: MDEV-40790 SELECT INTO row_type_of.field crashes the server

The server crashed on DBUG_ASSERT on a SELECT into:
- a `ROW TYPE OF table1` field variable
- a `ROW TYPE OF cursor1` field variable

Fix:

- Adding a class my_var_sp_row_field_by_name
- Adding a method sp_rcontext::set_variable_row_field_by_name()
- Fixing the DBUG_ASSERT
Marko Mäkelä
MDEV-40791 SET GLOBAL innodb_log_archive=OFF triggers full flush

log_t::set_archive(): When a checkpoint needs to be forced, set a
minimum target that guarantees progress, instead of flushing
the entire buffer pool. Remove some duplicated reads of
last_checkpoint_lsn.

When switching innodb_log_archive from OFF to ON,
accurately remember whether any log records may have been
written with get_sequence_bit() == 0, to ensure that a
checkpoint will be waited for on a subsequent switch
from ON to OFF.

log_t::set_recovered(): If innodb_log_archive=ON, only make a
future set_archive(false) trigger a checkpoint if we may have
recovered a server that was killed between set_archive(true)
and write_checkpoint().
Alexander Barkov
MDEV-39563 Implement UPDATE ... RETURNING ... INTO

Adding support for UPDATE .. RETURNING .. INTO queries.

For example:

  UPDATE t1 SET a=10,b=20 RETURNING a,b INTO va,vb;
  UPDATE t1 SET a=10,b=20 RETURNING a,b INTO @a,@b;

Limitations:
1. These types of queries:
  - REPLACE .. RETURNING .. INTO
  - DELETE .. RETURNING .. INTO
  - INSERT .. RETURNING .. INTO
  do not work - they return an error.
  They will be implemented separately, when needed.

2. UPDATE..RETURNING..INTO with --binlog_format=statement is not allowed
  and an error is raised.

3. Using OLD_VALUE(col) inside UPDATE..RETURNING..INTO is not allowed
  and an error is raised.

Notes:

1. ANALYZE and EXPLAIN
  Both
    ANALYZE UPDATE .. RETURNING .. INTO ..
    EXPLAIN UPDATE .. RETURNING .. INTO ..
  return this error:
    'RETURNING..INTO' is not allowed in this context

2. Behavior on no data

  a. If the updated table contains no rows, no errors are raised.

  b. In case of degenerated plans (WHERE 1=0, LIMIT 0),
    no errors are raised.

  c. If there are some rows, but non of them match the WHERE condition,
    then this error is raised:
      No data - zero rows fetched, selected, or processed

  d. If some rows where found but none of them actually
    got changed by the SET, still this error is raised:
      No data - zero rows fetched, selected, or processed
    The error message might be misleading. However, if we read
    it as "zero rows [that required updates] fetched", it looks OK.
    Let's don't introduce a new error message for now.

Helper changes:

1. The grammar in analyze_stmt_command was changed to have
  LEX::analyze_stmt set to true earlier, so
  LEX::set_returning_into_result() already knows if this
  is an ANALYZE statement.

2. The Sql_cmd_update constructor / orig_multitable → m_sql_command_code
  change is needed to be able to run Sql_cmd_update constructor earlier in
  the grammar, to be able to call Sql_cmd_update::set_with_old_value_items()
  in the SET and RETURNING clauses.
Marko Mäkelä
Add read-ahead hints
ParadoxV5
MDEV-40366 merge fix

reäpply missed merge fixes

Co-authored-by: Brandon Nesterenko <[email protected]>
PranavKTiwari
modified logic.
Brandon Nesterenko
MDEV-40768: Slave cannot apply a fragmented row event written with log_bin_compress=ON

MDEV-32570 added fragmentation of large row events. When binlog
compression is enabled (log_bin_compress=ON), the compression is
bypassed, and these binary log events remain as regular row events, when
they should be compressed row events.

This is because the compression happens during the Log_event::write()
function, whereas row data fragmentation happens before
Log_event::write() is called. The Rows_log_event super-class stores the
row data buffers to be written to disk. The regular Rows_log_event
sub-classes's implementations of Log_event::write() write this data
as-is. The compressed sub-classes's implementation of ::write()
over-write these buffers with the compressed rows data before writing to
disk.

To fragment a large row event, the server fragments the Rows_log_event's
row data buffers, to be written by multiple Partial_rows_log_event's,
and each Partial_rows_log_event::write() call writes its portion of the
row data buffer to disk. However, this happens before compression ever
has a chance to take place, and thereby, an event that should be
compressed, never is.

MDEV-39762 added event structure validation for compressed events, and
discovered that these events carry the compressed event type, but are
not actually compressed. Event validation thereby fails.

This patch adds a workaround to override the event type of the
fragmented row events to be regular row events, to be consistent with
the actual on-disk content. The underlying problem still needs to be
addressed though, and is tracked by MDEV-40851.

Signed-off-by: Brandon Nesterenko <[email protected]>
Yuchen Pei
MDEV-40805 Do not call lock_rec_convert_impl_to_expl if a table S-lock is held

lock_clust_rec_read_check_and_lock() skipped the implicit-to-explicit
conversion only under a table LOCK_X. When a table LOCK_S is held the
conversion is equally pointless: no other transaction can hold an
implicit X-lock on the record, because modifying a row requires a
table LOCK_IX and LOCK_IX is incompatible with our LOCK_S.
lock_table_has() matches stronger modes, so testing LOCK_S subsumes
the old LOCK_X test.
PranavKTiwari
bug fix.

modified logic.

modified logic.
PranavKTiwari
bug fix.
Sergei Golubchik
MDEV-40445 TLS 1.3 early data (0-RTT)

Assisted-By: Claude:claude-5-opus
Alexander Barkov
MDEV-39563 Implement UPDATE ... RETURNING ... INTO

Adding support for UPDATE .. RETURNING .. INTO queries.

For example:

  UPDATE t1 SET a=10,b=20 RETURNING a,b INTO va,vb;
  UPDATE t1 SET a=10,b=20 RETURNING a,b INTO @a,@b;

Limitations:
1. These types of queries:
  - REPLACE .. RETURNING .. INTO
  - DELETE .. RETURNING .. INTO
  - INSERT .. RETURNING .. INTO
  do not work - they return an error.
  They will be implemented separately, when needed.

2. UPDATE..RETURNING..INTO with --binlog_format=statement is not allowed
  and an error is raised.

3. Using OLD_VALUE(col) inside UPDATE..RETURNING..INTO is not allowed
  and an error is raised.

Notes:

1. ANALYZE and EXPLAIN
  Both
    ANALYZE UPDATE .. RETURNING .. INTO ..
    EXPLAIN UPDATE .. RETURNING .. INTO ..
  return this error:
    'RETURNING..INTO' is not allowed in this context

2. Behavior on no data

  a. If the updated table contains no rows, no errors are raised.

  b. In case of degenerated plans (WHERE 1=0, LIMIT 0),
    no errors are raised.

  c. If there are some rows, but non of them match the WHERE condition,
    then this error is raised:
      No data - zero rows fetched, selected, or processed

  d. If some rows where found but none of them actually
    got changed by the SET, still this error is raised:
      No data - zero rows fetched, selected, or processed
    The error message might be misleading. However, if we read
    it as "zero rows [that required updates] fetched", it looks OK.
    Let's don't introduce a new error message for now.
Sergei Golubchik
MDEV-40445 TLS session resumption (wolfssl)

* enable HAVE_SESSION_TICKET to allow resumption
* enable OPENSSL_ALL+KEEP_PEER_CERT to keep peer cert in the ticket,
  otherwise REQUIRE SUBJECT doesn't work after resumption
* disable OPENSSL_EXTRA which was enabled as a replacement
  when OPENSSL_ALL was disabled in 136e8661197
* disable NO_WOLFSSL_STUB to get SSL_CTX_sess_hits/etc stubs
* but they're stubs, always return 0, so add a wolfssl combination to
  the test that checks for these values
Sergei Golubchik
MDEV-40445 TLS session resumption

in fact it was already on in the server, so this only
enables statistics to see it in SHOW STATUS, adds tests,
and updates C/C to match.

Assisted-By: Claude:claude-5-opus