Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Sergei Petrunia
Cleanup in Optimizer_context_recorder

- Introduce init_optimizer_context_recorder_if_needed() which is called
at query start. We n longer create the recorder lazily.

- Recorder functions no longer take MEM_ROOT to store data on as an
  argument. The recorder knows the MEM_ROOT it should use.
Daniel Black
MDEV-39031 remove Docs/README-wsrep

The contents of this file where not current
and better and more current forms of documentation
related to Galera are on mariadb.com/docs.
Thirunarayanan Balathandayuthapani
MDEV-19194  ASAN use-after-poison in fk_prepare_copy_alter_table upon dropping FK

Problem:
========
The issue occurs when ALTER TABLE contains duplicate DROP FOREIGN KEY
operations (e.g., "DROP FOREIGN KEY f1, DROP FOREIGN KEY f1").
In fk_prepare_copy_alter_table(), Removes from the list all foreign
keys which are to be dropped. But iterator continues to accessing the
removed entry when it encounters duplicate foreign key.

Solution:
=========
mysql_prepare_alter_table(): Add duplicate detection logic to handle
cases where the same foreign key is specified multiple times in a
single ALTER TABLE statement's DROP statement.

fk_prepare_copy_alter_table(): fk_parent_key_list and fk_child_key_list
contain all existing foreign keys where the table acts as parent or child
respectively, and they are populated through storage engine calls.
After this phase, the code iterates through these collections comparing
each foreign key against alter_info->drop_list using case-insensitive
string matching for constraint names and database/table names.

- When matches are found, instead of removing them immediately,
the matching FOREIGN_KEY_INFO objects are added to the
keys_to_remove collection, which acts as a temporary holding area.
Once all matching keys are collected in keys_to_remove, use it to
remove the key entries from original lists.
Kristian Nielsen
MDEV-38734: Virtual columns wrongly included in binlog_row_image=MINIMAL

The original symptom of this was an assertion 'marked_for_read()' failing in
RBR with unique blobs and binlog_row_image=MINIMAL. The problem was that the
hidden DB_ROW_HASH_1 virtual column was included in the after-image of the
update, but the underlying blob column was not being updated, so it was not
in the read_set/write_set.

It seems clearly wrong to include the DB_ROW_HASH_1 in the after-image when
the underlying blob isn't even being updated. The cause of this is the
following commit:

Author: Monty <[email protected]>
Date:  Wed May 23 22:42:29 2018 +0300

    MDEV-15243 Crash with virtual fields and row based binary logging

That patch removed a check for if the underlying fields of a virtual column
were being updated, and just added them unconditionally. This seems wrong.

So revert that part of the commit, restoring the logic to only add a virtual
column if any underlying field is actually in the write_set.

Also fix a typo in that commit where a code reformat accidentally reversed a
condition.

Also fix an assertion when InnoDB goes to update secondary indexes: If any
part of the primary key is being updated, then add all virtual columns that
are part of secondary indexes to the read_set.

Signed-off-by: Kristian Nielsen <[email protected]>
sjaakola
MDEV-21935 Mariabackup file limits set incorrectly

Fixed a build time regression happening in x86-debian-12 platform
Marko Mäkelä
Merge 10.11 into 11.4
Marko Mäkelä
MDEV-29445 fixup: Remove a bogus assertion

buf_buddy_shrink(): Remove a redundant assertion
ut_ad(buf_pool.will_be_withdrawn(bpage->zip.data, size))
that is using an incorrect size parameter. In the only caller
buf_pool_t::shrink(size_t size) we already checked with the
correct size that the ROW_FORMAT=COMPRESSED page frame will be
in the being-trimmed portion of the buffer pool. If
p >= memory + size held for a larger size, it would also trivially
hold with the smaller (and incorrect) size.

This code is covered by the test innodb.innodb_buffer_pool_resize,
which uses ROW_FORMAT=COMPRESSED tables.
Rucha Deodhar
MDEV-38835: json path regression

Analysis:
JSON_VALUE should not allow wildcard in the path since it is not supposed to
return multiple values (as per json standards). However, JSON_QUERY()
should.

Fix:
Make JSON_VALUE() return multiple error on wildcard in the path.
Marko Mäkelä
MDEV-38958: Core dump contains buffer pool in release builds

buf_pool_t::create(), buf_pool_t::resize(): After
my_virtual_mem_commit() successfully invoked mmap(MAP_FIXED), invoke
ut_dontdump() so that the buffer pool will continue to be excluded
from any core dump as expected on platforms that implement this
functionality.

This was manually tested on Linux and FreeBSD by executing
killall -ABRT mariadbd
and checking the size of the core dump file, while the following
test case was executing:
--source include/have_innodb.inc
set global innodb_buffer_pool_size=10737418240;
sleep 3600;

This fixes up the following changes:
commit b6923420f326ac030e4f3ef89a2acddb45eccb30 (MDEV-29445)
commit 072c7dc774e7f31974eaa43ec1cbb3b742a1582e (MDEV-38671)
Vladislav Vaintroub
Revert "MDEV-38989 main.ctype_utf16le SEGV in Ubuntu 26.04 (x86-64-v3)"

This reverts commit 6318d5001597fff28ea7ba7f957d66edd7e3e954.

uint2korr is now fixed in MDEV-37788, thus partial workarounds are not necessary anymore.
Kristian Nielsen
Merge 10.11 to 11.4
Marko Mäkelä
MDEV-38968 Redundant FILE_CHECKPOINT writes

Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records that refer to the same checkpoint. Let us simplify the logic
by making the dedicated buf_flush_page_cleaner() thread responsible
for checkpoints.

log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which will replace the data member log_sys.next_checkpoint_lsn.

log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner()
thread will write checkpoints or initiate page writes.

log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.

buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().

lot_t::write_buf(): Remove a call set_check_for_checkpoint(false)
that commit 7443ad1c8a8437a761e1a2d3ba53f7c0ba0dd3bb (MDEV-32374)
had added. The flag should only be cleared when the checkpoint has
advanced far enough.

log_make_checkpoint(): Simply wrap buf_flush_sync_batch(0, true).

buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.

buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.

buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.

buf_flush_buffer_pool(): Remove.

buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.

logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.

fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.

innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().

srv_undo_tablespaces_reinit(): Simplify the logic and remove the
fault injection point after_reinit_undo_abort, which would cause
a failure on Microsoft Windows. Changing innodb_undo_tablespaces is
not fully crash-safe.

Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Saahil Alam
Raghunandan Bhat
MDEV-36678: Various crashes upon stored procedure querying view with concat/group_concat

Problem:
  A stored procedure querying a view with `CONCAT`/`GROUP_CONCAT` could
  crash due to a NULL pointer dereference during query optimization.

  During condition pushdown, `Item_direct_ref_to_item::deep_copy`
  creates a clone where the clone's `ref` pointer incorrectly remains
  pointing to the original object's `m_item`, rather than its own.

  Because the clone is tethered to the original object, subsequent
  transformations happen on the original item instead of the clone.
  Calling `fix_fields` on malformed `Item_direct_ref_to_item` clone
  resolves to `Item_field::fix_fields` instead of `Item_ref::fix_fields`
  leaving the member `Item_ref::ref` uninitialized (NULL). When this is
  dereferenced in `Item_ref::const_item`, server crashes.

Fix:
  The `Item_direct_ref_to_item::set_item` method is made to update the
  item and reference, making the clone complete. This results in correct
  `fix_fields` resolution, making `Item_ref::ref` safe to access.
kjarir
MDEV-38507: Reduce fadvise() overhead on pipes in Mariabackup

Currently, posix_fadvise(..., 0, 0, POSIX_FADV_DONTNEED) is executed
repeatedly after each my_write. In case of pipes, it evaluates to
ESPIPE and fails. This results in millions of redundant syscalls,
introducing massive overhead.

As proposed by Daniel Black, call posix_fadvise with a length/size
of 0 once when the file is opened, rather than repetitively after
every write. This change moves the fadvise call to the respective
_open functions for ds_local, ds_stdout and ds_tmpfile datasinks.
It fails fast on pipes early on and eliminates the redundant
syscall overhead for subsequent write operations.
Marko Mäkelä
Merge 10.11 into 11.4
Kristian Nielsen
MDEV-38734: Virtual columns wrongly included in binlog_row_image=MINIMAL

The original symptom of this was an assertion 'marked_for_read()' failing in
RBR with unique blobs and binlog_row_image=MINIMAL. The problem was that the
hidden DB_ROW_HASH_1 virtual column was included in the after-image of the
update, but the underlying blob column was not being updated, so it was not
in the read_set/write_set.

It seems clearly wrong to include the DB_ROW_HASH_1 in the after-image when
the underlying blob isn't even being updated. The cause of this is the
following commit:

Author: Monty <[email protected]>
Date:  Wed May 23 22:42:29 2018 +0300

    MDEV-15243 Crash with virtual fields and row based binary logging

That patch removed a check for if the underlying fields of a virtual column
were being updated, and just added them unconditionally. This seems wrong.

So revert that part of the commit, restoring the logic to only add a virtual
column if any underlying field is actually in the write_set.

Also fix a typo in that commit where a code reformat accidentally reversed a
condition.

Also fix an assertion when InnoDB goes to update secondary indexes: If any
part of the primary key is being updated, then add all virtual columns that
are part of secondary indexes to the read_set.

Signed-off-by: Kristian Nielsen <[email protected]>
Thirunarayanan Balathandayuthapani
MDEV-38993 Assertion `trx->undo_no == 1' fails upon ALTER IGNORE

Problem:
========
During ALTER TABLE ... IGNORE, a partial rollback on duplicate key
error resets trx->undo_no to 0. The subsequent insert then enters
the undo rewrite block with undo_no == 0, hitting the assertion
that expected undo_no == 1.

Solution:
=========
Partial rollback which truncates the last insert undo record
via trx_undo_truncate_end(), which rewrites TRX_UNDO_PAGE_FREE
on the page. By checking trx->undo_no as part of the rewrite
predicate, InnoDB correctly skips the rewrite logic after partial
rollback.

trx_undo_report_row_operation(): Pre-compute the full predicate
(clear_ignore) before trx_undo_assign_low(), since old_offset
and top_offset are not modified by that call.

trx_undo_rewrite_ignore(): Extract the rewrite body into a
separate ATTRIBUTE_COLD ATTRIBUTE_NOINLINE static function.
Marko Mäkelä
Fix GCC-16 -Wunused-but-set-variable
Georgi (Joro) Kodinov
MDEV-38673: Focus the pick-a-task wording in CONTRIBUTING.md
The wording on how to pick a task in CONTRIBUTING.md is a bit weak.
A more focused message is needed to highlight the tasks that
are available for people to work on.
Vladislav Vaintroub
MDEV-37788 fix uintNkorr/intNstore macros to avoid unaligned access

Replaced macros with optimized inline functions, that use memcpy
instead of unaligned access (translates to single instruction
on architectures we support). Used compiler byteswap intrinsics
where appropriate.

Removed byte_order_generic.h as well as Intel specific headers
byte_order_generic_{x86|x86_64}.h

Added some casts to places where intNstore writes into "const char*".
Kristian Nielsen
Merge 10.11 to 11.4
Kristian Nielsen
MDEV-38734: Virtual columns wrongly included in binlog_row_image=MINIMAL

The original symptom of this was an assertion 'marked_for_read()' failing in
RBR with unique blobs and binlog_row_image=MINIMAL. The problem was that the
hidden DB_ROW_HASH_1 virtual column was included in the after-image of the
update, but the underlying blob column was not being updated, so it was not
in the read_set/write_set.

It seems clearly wrong to include the DB_ROW_HASH_1 in the after-image when
the underlying blob isn't even being updated. The cause of this is the
following commit:

Author: Monty <[email protected]>
Date:  Wed May 23 22:42:29 2018 +0300

    MDEV-15243 Crash with virtual fields and row based binary logging

That patch removed a check for if the underlying fields of a virtual column
were being updated, and just added them unconditionally. This seems wrong.

So revert that part of the commit, restoring the logic to only add a virtual
column if any underlying field is actually in the write_set.

Also fix a typo in that commit where a code reformat accidentally reversed a
condition.

Also fix an assertion when InnoDB goes to update secondary indexes: If any
part of the primary key is being updated, then add all virtual columns that
are part of secondary indexes to the read_set.

Signed-off-by: Kristian Nielsen <[email protected]>
Vladislav Vaintroub
Revert "MDEV-38989 main.ctype_utf16le SEGV in Ubuntu 26.04 (x86-64-v3)"

This reverts commit 6318d5001597fff28ea7ba7f957d66edd7e3e954.

uint2korr is now fixed in MDEV-37788, thus partial workarounds are not necessary anymore.
Marko Mäkelä
Merge 10.6 into 10.11
Marko Mäkelä
MDEV-38989 main.ctype_utf16le SEGV in Ubuntu 26.04 (x86-64-v3)

my_utf16le_uni(), my_lengthsp_utf16le(): Instead of wrongly claiming
aligned access by invoking uint2korr(), inform the compiler of
unaligned access by invoking memcpy(), which will be optimized away.
Marko Mäkelä
MDEV-37412/MDEV-38289 fixup: Allow more FILE_CHECKPOINT

The test was occasionally failing on Microsoft Windows, with the message
saying 12514, which was exactly 16 more than the maximum value that our
regular expression tolerated. Let us add some more slack, up to 12690.
Marko Mäkelä
Fix GCC-16 -Wunused-but-set-variable
Marko Mäkelä
MDEV-38958 fixup: Correct the address ranges

buf_pool_t::create(): Invoke ut_dontdump() on the entire
buf_pool.memory_unaligned.

buf_pool_t::resize(): Invoke ut_dontdump() on the grown allocation.

Thanks to Alessandro Vetere for pointing this out.
Marko Mäkelä
Fix GCC-16 -Wmaybe-uninitialized
Kristian Nielsen
Fix compile error in prior merge

Signed-off-by: Kristian Nielsen <[email protected]>
Daniel Black
MDEV-39015 Debian - remove libboost-system-dev dependency

In all supported Debian versions, which are all 1.69+,
this package only provided a library stub. The header
files where moved into libboost-dev when upstream removed
the need for a library.
Hemant Dangi
MDEV-38916: Galera test failure on galera_3nodes.MDEV-36360

Issue:
wsrep_slave_FK_checks is deprecated(MDEV-38787) and has no effect,
while applier threads are forced to run with FK checks enabled.
The MDEV-36360 test expects that no FK constraint failures appear
in the error log, which does not happen because of
wsrep_slave_FK_checks has no effect and so test fails.

Solution:
Remove the assert_grep.inc block so the test remains as a check
for table-level X-lock / deadlock behavior only.
Vladislav Vaintroub
MDEV-37788 fix uintNkorr/intNstore macros to avoid unaligned access

Replaced macros with optimized inline functions, that use memcpy
instead of unaligned access (translates to single instruction
on architectures we support). Used compiler byteswap intrinsics
where appropriate.

Removed byte_order_generic.h as well as Intel specific headers
byte_order_generic_{x86|x86_64}.h

Added some casts to places where intNstore writes into "const char*".
Thirunarayanan Balathandayuthapani
Merge branch 10.6 into 10.11
Thirunarayanan Balathandayuthapani
MDEV-38928 Assertion `undo_no <= 1' failed in trx_t::reset_and_truncate_undo()

Problem:
========
1) ALTER IGNORE TABLE...ALGORITHM=COPY on an InnoDB temporary table
triggers an assertion failure in trx_t::reset_and_truncate_undo().
The IGNORE_UNDO mode rewrites the insert undo log to retain only
the latest record, but temporary tables undo logs are not processed
by the purge thread. Applying this optimization
to a temporary table is incorrect.

2) For partitioned tables with ALTER IGNORE...ALGORITHM=COPY, the
IGNORE_UNDO in trx_undo_report_row_operation() resets trx->undo_no
to 0, but trx->mod_tables retains entries from earlier partitions.
When a different partition (dict_table_t) is inserted into, its
mod_tables entry is created with first=1. After the rewinding, it
writes the undo record with top_undo_no=0, time.valid(0) fails
because first(1) > 0.

3) The savepoint for partial rollback also did not account for
IGNORE_UNDO mode, where trx->undo_no is continuously reset to 0.

Solution:
=========
ha_innobase::extra(): Set skip_alter_undo=IGNORE_UNDO only for
non-temporary tables during HA_EXTRA_BEGIN_ALTER_IGNORE_COPY.
Temporary tables keep NORMAL_UNDO since purge won't process
write temporary table undo logs.

trx_t::reset_and_truncate_undo(): Remove the table parameter
because same transaction could modify multiple tables in
case of partition.

trx_undo_report_row_operation(): After rewinding trx->undo_no to 0
in IGNORE_UNDO mode, clear mod_tables and re-emplace the
table with first=0.

row_insert_for_mysql(): Use savept=0 for IGNORE_UNDO mode. So that
partial rollback target the last inserted undo record.

Change TRX_DML_BULK=2, TRX_DDL_BULK=3 so that bit 1 is set for all
bulk insert modes and for dict_table_t::IGNORE_UNDO. This allows
to replace the savept conditional with a single bit test:
(trx->bulk_insert | table->skip_alter_undo) & 2
Marko Mäkelä
Merge 10.6 into 10.11
Marko Mäkelä
Merge 10.6 into 10.11
Varun Deep Saini
MDEV-38873 JSON_EXTRACT truncates result through derived tables

Item_func_json_extract::fix_length_and_dec() underestimated
max_length. The result is passed through json_nice() with LOOSE
formatting which expands separators (e.g. ["a",1] -> ["a", 1]),
but max_length was computed from the input size alone. When the
result flows through a materialized derived table, the field is
created with this too-small max_length and the output gets truncated.

Fix: multiply by 2 to account for LOOSE expansion, add array
framing overhead for multi-path extractions, and use
fix_char_length_ulonglong() for proper character-to-byte
conversion and overflow handling.

Signed-off-by: Varun Deep Saini <[email protected]>
Abhishek Bansal
MDEV-37640: Crash at val_str in Item_func_json* functions

This also fixes MDEV-33984. Item_func_json_normalize::val_str()
and Item_func_json_keys::val_str failed to initialize the character
set of the result buffer. In certain contexts, the buffer can be a
zero-initialized String object with a NULL charset. This led to a null
pointer dereference in String::append(), which relies on the charset
information.

Fixed by explicitly setting the buffer's charset to the item's
collation before appending the normalized JSON string.
Marko Mäkelä
MDEV-39040 log_sys.latch performance lost to PERFORMANCE_SCHEMA

log_sys.latch: Remove the PERFORMANCE_SCHEMA instrumentation.
We already know that this is a very busy latch. All code paths
where a shared log_sys.latch is being held should already be
highly optimized. The few paths where an exclusive log_sys.latch
is being held are known to be potentially problematic, but rare.

Removing the PERFORMANCE_SCHEMA instrumentation for this latch
is expected to improve performance. On a quick Sysbench run with

performance_schema_instrument=wait/synch/rwlock/%=on

I observed an 2.9% improvement on throughput. It could be more in
other test scenarios.