Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Rex Johnston
MDEV-29360 Crash when pushing condition with always false IS NULL into derived

When a condition containing an always FALSE range part is pushed
down into a derived table, an enclosed outer reference can cause
a null pointer crash.  An example

SELECT * FROM (SELECT id  FROM t1 GROUP BY id) dt1, (SELECT id FROM t2) dt2, t3
  WHERE dt2.id = t3.id AND dt1.id BETWEEN 0 AND (dt2.id IS NULL);

is considered as one that can be pushed into the derived table dt1
because the expression (dt2.id IS NULL) is identified as always false.
On the on other hand the condition still contains a reference to a
column of the table dt2. When pushing the condition to the where clause
of the derived table dt1 a copy of it is created and this copy is
transformed to be used in the new where clause. When the transformation
procedure encounters a reference dt2.id it crashes the server as only
the references to the columns used in the group by list of the
specification of dt1 or those equal to them are expected in the
condition that can be pushed into the where clause of the derived table.

Based on patch by Igor Babaev.
Daniel Black
MDEV-39031 remove Docs/README-wsrep

The contents of this file where not current
and better and more current forms of documentation
related to Galera are on mariadb.com/docs.
Thirunarayanan Balathandayuthapani
MDEV-19194  ASAN use-after-poison in fk_prepare_copy_alter_table upon dropping FK

Problem:
========
The issue occurs when ALTER TABLE contains duplicate DROP FOREIGN KEY
operations (e.g., "DROP FOREIGN KEY f1, DROP FOREIGN KEY f1").
In fk_prepare_copy_alter_table(), Removes from the list all foreign
keys which are to be dropped. But iterator continues to accessing the
removed entry when it encounters duplicate foreign key.

Solution:
=========
mysql_prepare_alter_table(): Add duplicate detection logic to handle
cases where the same foreign key is specified multiple times in a
single ALTER TABLE statement's DROP statement.

fk_prepare_copy_alter_table(): fk_parent_key_list and fk_child_key_list
contain all existing foreign keys where the table acts as parent or child
respectively, and they are populated through storage engine calls.
After this phase, the code iterates through these collections comparing
each foreign key against alter_info->drop_list using case-insensitive
string matching for constraint names and database/table names.

- When matches are found, instead of removing them immediately,
the matching FOREIGN_KEY_INFO objects are added to the
keys_to_remove collection, which acts as a temporary holding area.
Once all matching keys are collected in keys_to_remove, use it to
remove the key entries from original lists.
Daniel Black
MCOL-6313: test submodule update for MCOL-6313
Rucha Deodhar
MDEV-38835: json path regression

Analysis:
JSON_VALUE should not allow wildcard in the path since it is not supposed to
return multiple values (as per json standards). However, JSON_QUERY()
should.

Fix:
Make JSON_VALUE() return multiple error on wildcard in the path.
bsrikanth-mariadb
MDEV-38805: Tests refactoring

1. store round_cost() in a separate .inc file so that it can be
  reused across multiple tests
2. Change the usage of "--error 1290" with
  "--error ER_OPTION_PREVENTS_STATEMENT"
Marko Mäkelä
MDEV-38958: Core dump contains buffer pool in release builds

buf_pool_t::create(), buf_pool_t::resize(): After
my_virtual_mem_commit() successfully invoked mmap(MAP_FIXED), invoke
ut_dontdump() so that the buffer pool will continue to be excluded
from any core dump as expected on platforms that implement this
functionality.

This was manually tested on Linux and FreeBSD by executing
killall -ABRT mariadbd
and checking the size of the core dump file, while the following
test case was executing:
--source include/have_innodb.inc
set global innodb_buffer_pool_size=10737418240;
sleep 3600;

This fixes up the following changes:
commit b6923420f326ac030e4f3ef89a2acddb45eccb30 (MDEV-29445)
commit 072c7dc774e7f31974eaa43ec1cbb3b742a1582e (MDEV-38671)
kjarir
MDEV-38998: Assertion failure in Diagnostics Area with IGNORE and CHECK failure.

When evaluating a CHECK constraint during INSERT/UPDATE IGNORE, an evaluation error (like ER_DATA_OUT_OF_RANGE) can be set in the Diagnostics Area.
If the failure is ignored (downgraded to a warning), verify_constraints() was not clearing the error state from the DA before returning.
This led to a crash in my_ok() because it asserts that no error is set in the DA.

Fixed by calling clear_error() inside verify_constraints() when ignore_failure is true, ensuring the DA is ready for the subsequent OK packet.
Yuchen Pei
MDEV-24813 Fix avoid_deadlock_with_blocked?

Perhaps there is a race condition.

Prior art (tests with --error 0,ER_LOCK_WAIT_TIMEOUT):

- innodb.deadlock_detect (nondeterministic error added by MDEV-13262
  innodb.deadlock_detect failed in buildbot)
- innodb.import_tablespace_race
- innodb.innodb-lock-inherit-read_commited
- innodb.innodb-lock
Marko Mäkelä
MDEV-38968 Redundant FILE_CHECKPOINT writes

Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records that refer to the same checkpoint. Let us simplify the logic
by making the dedicated buf_flush_page_cleaner() thread responsible
for checkpoints.

log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which will replace the data member log_sys.next_checkpoint_lsn.

log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner()
thread will write checkpoints or initiate page writes.

log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.

buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().

lot_t::write_buf(): Remove a call set_check_for_checkpoint(false)
that commit 7443ad1c8a8437a761e1a2d3ba53f7c0ba0dd3bb (MDEV-32374)
had added. The flag should only be cleared when the checkpoint has
advanced far enough.

log_make_checkpoint(): Simply wrap buf_flush_sync_batch(0, true).

buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.

buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.

buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.

buf_flush_buffer_pool(): Remove.

buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.

logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.

fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.

innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().

srv_undo_tablespaces_reinit(): Simplify the logic and remove the
fault injection point after_reinit_undo_abort, which would cause
a failure on Microsoft Windows. Changing innodb_undo_tablespaces is
not fully crash-safe.

Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Saahil Alam
Raghunandan Bhat
MDEV-36678: Various crashes upon stored procedure querying view with concat/group_concat

Problem:
  A stored procedure querying a view with `CONCAT`/`GROUP_CONCAT` could
  crash due to a NULL pointer dereference during query optimization.

  During condition pushdown, `Item_direct_ref_to_item::deep_copy`
  creates a clone where the clone's `ref` pointer incorrectly remains
  pointing to the original object's `m_item`, rather than its own.

  Because the clone is tethered to the original object, subsequent
  transformations happen on the original item instead of the clone.
  Calling `fix_fields` on malformed `Item_direct_ref_to_item` clone
  resolves to `Item_field::fix_fields` instead of `Item_ref::fix_fields`
  leaving the member `Item_ref::ref` uninitialized (NULL). When this is
  dereferenced in `Item_ref::const_item`, server crashes.

Fix:
  The `Item_direct_ref_to_item::set_item` method is made to update the
  item and reference, making the clone complete. This results in correct
  `fix_fields` resolution, making `Item_ref::ref` safe to access.
Kristian Nielsen
MDEV-38731: Wrong index usage when row-based replication and no PK

This affects row-based replication for UPDATE/DELETE on tables with no
primary key (and no non-NULL unique index).

There was a typo / incorrect merge from the following commit:

commit 3abce27e9d45282084aa8d0972ffb6721bac16f5
Author: unknown <[email protected]>
Date:  Mon Dec 27 22:37:37 2010 +0100

    Merge Percona patch row_based_replication_without_primary_key.patch into MariaDB.

Instead of referencing the index selected for applying the event, part
of the code was referencing the first index (in internal order) in the
table. This is code that checks if an index lookup is unique, or if it
requires a range scan. This could lead the code to incorrectly do a
unique lookup instead of comparing the pre-image against each row in a
range scan, thus applying the event to the wrong row and causing
replication to diverge.

Signed-off-by: Kristian Nielsen <[email protected]>
Kristian Nielsen
Merge 10.6 -> 10.11
kjarir
MDEV-38507: Reduce fadvise() overhead on pipes in Mariabackup

Currently, posix_fadvise(..., 0, 0, POSIX_FADV_DONTNEED) is executed
repeatedly after each my_write. In case of pipes, it evaluates to
ESPIPE and fails. This results in millions of redundant syscalls,
introducing massive overhead.

As proposed by Daniel Black, call posix_fadvise with a length/size
of 0 once when the file is opened, rather than repetitively after
every write. This change moves the fadvise call to the respective
_open functions for ds_local, ds_stdout and ds_tmpfile datasinks.
It fails fast on pipes early on and eliminates the redundant
syscall overhead for subsequent write operations.
Marko Mäkelä
Merge 10.11 into 11.4
Kristian Nielsen
MDEV-38734: Virtual columns wrongly included in binlog_row_image=MINIMAL

The original symptom of this was an assertion 'marked_for_read()' failing in
RBR with unique blobs and binlog_row_image=MINIMAL. The problem was that the
hidden DB_ROW_HASH_1 virtual column was included in the after-image of the
update, but the underlying blob column was not being updated, so it was not
in the read_set/write_set.

It seems clearly wrong to include the DB_ROW_HASH_1 in the after-image when
the underlying blob isn't even being updated. The cause of this is the
following commit:

Author: Monty <[email protected]>
Date:  Wed May 23 22:42:29 2018 +0300

    MDEV-15243 Crash with virtual fields and row based binary logging

That patch removed a check for if the underlying fields of a virtual column
were being updated, and just added them unconditionally. This seems wrong.

So revert that part of the commit, restoring the logic to only add a virtual
column if any underlying field is actually in the write_set.

Also fix a typo in that commit where a code reformat accidentally reversed a
condition.

Also fix an assertion when InnoDB goes to update secondary indexes: If any
part of the primary key is being updated, then add all virtual columns that
are part of secondary indexes to the read_set.

Signed-off-by: Kristian Nielsen <[email protected]>
Thirunarayanan Balathandayuthapani
MDEV-38993 Assertion `trx->undo_no == 1' fails upon ALTER IGNORE

Problem:
========
During ALTER TABLE ... IGNORE, a partial rollback on duplicate key
error resets trx->undo_no to 0. The subsequent insert then enters
the undo rewrite block with undo_no == 0, hitting the assertion
that expected undo_no == 1.

Solution:
=========
Partial rollback which truncates the last insert undo record
via trx_undo_truncate_end(), which rewrites TRX_UNDO_PAGE_FREE
on the page. By checking trx->undo_no as part of the rewrite
predicate, InnoDB correctly skips the rewrite logic after partial
rollback.

trx_undo_report_row_operation(): Pre-compute the full predicate
(clear_ignore) before trx_undo_assign_low(), since old_offset
and top_offset are not modified by that call.

trx_undo_rewrite_ignore(): Extract the rewrite body into a
separate ATTRIBUTE_COLD ATTRIBUTE_NOINLINE static function.
Marko Mäkelä
Fix GCC-16 -Wunused-but-set-variable
Kristian Nielsen
MDEV-38776 [ERROR] Slave worker thread retried transaction 10 time(s) in vain, giving up

Partially revert this commit:

commit 6a1cb449feb1b77e5ec94904c228d7c5477f528a
Author: Sergei Golubchik <[email protected]>
Date:  Mon Jan 18 18:02:16 2021 +0100

    cleanup: remove slave background thread, use handle_manager thread instead

This restores running the parallel replication deadlock killing in its own
dedicated thread, not in the manager thread shared with other unrelated
processing.

When a parallel replication conflict is detected, multiple threads can be
waiting for each other, potentially in a loop. It is critical for
correctness (as well as performance) that the blocking thread is killed
immediately to allow other threads to continue. If one of the threads being
blocked was the manager thread itself in some unrelated job, the kill could
end up being blocked indefinitely, causing replication to hang, usually
eventually timing out on innodb_lock_wait_timeout and failing replication
with an error like:

[ERROR] Slave worker thread retried transaction 10 time(s) in vain, giving up

Signed-off-by: Kristian Nielsen <[email protected]>
Georgi (Joro) Kodinov
MDEV-38673: Focus the pick-a-task wording in CONTRIBUTING.md
The wording on how to pick a task in CONTRIBUTING.md is a bit weak.
A more focused message is needed to highlight the tasks that
are available for people to work on.
Aquila Macedo
systemd: Add low risk hardening to unit templates

Harden mariadb.service and [email protected] with low regression systemd
sandboxing defaults (kernel/cgroup protections, restrict namespaces,
disable realtime, lock personality, overrideable via drop-in).

Avoid historically problematic knobs (NoNewPrivileges, capability
bounding, PrivateDevices). Refs: MDEV-10404, MDEV-19878, MDEV-36591,
MDEV-36681
Kristian Nielsen
Merge 10.11 to 11.4
Marko Mäkelä
Merge 10.6 into 10.11
Yuchen Pei
MDEV-24813 Signal full scan to storage engines.

When starting to do a full table/index scan without a WHERE or JOIN
condition, tell the storage engine so and the corresponding
ulong-truncated LIMIT.

Include an innodb implementation: added an innodb switch
table_lock_on_full_scan, so that when the switch is on, on receiving
the full scan signal from the sql layer, if the truncated LIMIT is
ULONG_MAX (likely no LIMIT), attempt to acquire a table lock.

Updated tests that have different results with the switch on.

(Comment and code edited by Sergei Petrunia)
Marko Mäkelä
MDEV-38989 main.ctype_utf16le SEGV in Ubuntu 26.04 (x86-64-v3)

my_utf16le_uni(), my_lengthsp_utf16le(): Instead of wrongly claiming
aligned access by invoking uint2korr(), inform the compiler of
unaligned access by invoking memcpy(), which will be optimized away.
Yuchen Pei
MDEV-24813 Fix avoid_deadlock_with_blocked?

Perhaps there is a race condition.

Prior art (tests with --error 0,ER_LOCK_WAIT_TIMEOUT):

- innodb.deadlock_detect (nondeterministic error added by MDEV-13262
  innodb.deadlock_detect failed in buildbot)
- innodb.import_tablespace_race
- innodb.innodb-lock-inherit-read_commited
- innodb.innodb-lock
Yuchen Pei
MDEV-24813 Fix avoid_deadlock_with_blocked?

Perhaps there is a race condition.

Prior art (tests with --error 0,ER_LOCK_WAIT_TIMEOUT):

- innodb.deadlock_detect (nondeterministic error added by MDEV-13262
  innodb.deadlock_detect failed in buildbot)
- innodb.import_tablespace_race
- innodb.innodb-lock-inherit-read_commited
- innodb.innodb-lock
Marko Mäkelä
MDEV-37412/MDEV-38289 fixup: Allow more FILE_CHECKPOINT

The test was occasionally failing on Microsoft Windows, with the message
saying 12514, which was exactly 16 more than the maximum value that our
regular expression tolerated. Let us add some more slack, up to 12690.
Marko Mäkelä
Fix GCC-16 -Wunused-but-set-variable
Marko Mäkelä
MDEV-38958 fixup: Correct the address ranges

buf_pool_t::create(): Invoke ut_dontdump() on the entire
buf_pool.memory_unaligned.

buf_pool_t::resize(): Invoke ut_dontdump() on the grown allocation.

Thanks to Alessandro Vetere for pointing this out.
Marko Mäkelä
Fix GCC-16 -Wmaybe-uninitialized
Fariha Shaikh
MDEV-39028 DROP PARTITION, CONVERT OUT require both ALTER and DROP privileges

ALTER TABLE ... DROP PARTITION requires both ALTER and DROP privileges,
which is inconsistent with TRUNCATE PARTITION that only requires DROP.
This prevents fine-grained privilege separation since users who need to
drop partitions must also be granted ALTER, allowing them to perform
any other DDL changes on the table.

Change DROP PARTITION to require only DROP privilege, consistent with
TRUNCATE PARTITION. Users allowed to drop the table should also be
allowed to drop partitions without needing ALTER rights.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
Daniel Black
MDEV-39015 Debian - remove libboost-system-dev dependency

In all supported Debian versions, which are all 1.69+,
this package only provided a library stub. The header
files where moved into libboost-dev when upstream removed
the need for a library.
Hemant Dangi
MDEV-38916: Galera test failure on galera_3nodes.MDEV-36360

Issue:
wsrep_slave_FK_checks is deprecated(MDEV-38787) and has no effect,
while applier threads are forced to run with FK checks enabled.
The MDEV-36360 test expects that no FK constraint failures appear
in the error log, which does not happen because of
wsrep_slave_FK_checks has no effect and so test fails.

Solution:
Remove the assert_grep.inc block so the test remains as a check
for table-level X-lock / deadlock behavior only.
Thirunarayanan Balathandayuthapani
MDEV-38928 Assertion `undo_no <= 1' failed in trx_t::reset_and_truncate_undo()

Problem:
========
1) ALTER IGNORE TABLE...ALGORITHM=COPY on an InnoDB temporary table
triggers an assertion failure in trx_t::reset_and_truncate_undo().
The IGNORE_UNDO mode rewrites the insert undo log to retain only
the latest record, but temporary tables undo logs are not processed
by the purge thread. Applying this optimization
to a temporary table is incorrect.

2) For partitioned tables with ALTER IGNORE...ALGORITHM=COPY, the
IGNORE_UNDO in trx_undo_report_row_operation() resets trx->undo_no
to 0, but trx->mod_tables retains entries from earlier partitions.
When a different partition (dict_table_t) is inserted into, its
mod_tables entry is created with first=1. After the rewinding, it
writes the undo record with top_undo_no=0, time.valid(0) fails
because first(1) > 0.

3) The savepoint for partial rollback also did not account for
IGNORE_UNDO mode, where trx->undo_no is continuously reset to 0.

Solution:
=========
ha_innobase::extra(): Set skip_alter_undo=IGNORE_UNDO only for
non-temporary tables during HA_EXTRA_BEGIN_ALTER_IGNORE_COPY.
Temporary tables keep NORMAL_UNDO since purge won't process
write temporary table undo logs.

trx_t::reset_and_truncate_undo(): Remove the table parameter
because same transaction could modify multiple tables in
case of partition.

trx_undo_report_row_operation(): After rewinding trx->undo_no to 0
in IGNORE_UNDO mode, clear mod_tables and re-emplace the
table with first=0.

row_insert_for_mysql(): Use savept=0 for IGNORE_UNDO mode. So that
partial rollback target the last inserted undo record.

Change TRX_DML_BULK=2, TRX_DDL_BULK=3 so that bit 1 is set for all
bulk insert modes and for dict_table_t::IGNORE_UNDO. This allows
to replace the savept conditional with a single bit test:
(trx->bulk_insert | table->skip_alter_undo) & 2
Kristian Nielsen
MDEV-37133: Fix parallel replication stalls due to missing wait report

In certain situations, while one transaction was waiting for a row lock,
InnoDB could grant another transaction a lock that conflicted with the waiting
lock, but not with other already granted locks. Then if the page was
reorganised, the granted lock could be moved to the head of the lock queue.
This could end up with a situation where the waiting lock is now waiting for
the granted lock, and this wait was never reported to replication. This could
cause parallel replication to hang for --innodb-lock-wait-timeout if the
waiting transaction is before the granted one.

This patch fixes the issue by adding logic, so that when a lock is granted and
added to a lock queue, a wait is reported against this lock for any already
waiting locks. (This supplements the wait reports made when a waiting lock
is added to the queue).

Signed-off-by: Kristian Nielsen <[email protected]>
Marko Mäkelä
Merge 10.6 into 10.11
Varun Deep Saini
MDEV-38873 JSON_EXTRACT truncates result through derived tables

Item_func_json_extract::fix_length_and_dec() underestimated
max_length. The result is passed through json_nice() with LOOSE
formatting which expands separators (e.g. ["a",1] -> ["a", 1]),
but max_length was computed from the input size alone. When the
result flows through a materialized derived table, the field is
created with this too-small max_length and the output gets truncated.

Fix: multiply by 2 to account for LOOSE expansion, add array
framing overhead for multi-path extractions, and use
fix_char_length_ulonglong() for proper character-to-byte
conversion and overflow handling.

Signed-off-by: Varun Deep Saini <[email protected]>
Abhishek Bansal
MDEV-37640: Crash at val_str in Item_func_json* functions

This also fixes MDEV-33984. Item_func_json_normalize::val_str()
and Item_func_json_keys::val_str failed to initialize the character
set of the result buffer. In certain contexts, the buffer can be a
zero-initialized String object with a NULL charset. This led to a null
pointer dereference in String::append(), which relies on the charset
information.

Fixed by explicitly setting the buffer's charset to the item's
collation before appending the normalized JSON string.
Daniel Black
systemd: remove NoNewPrivileges

NoNewPrivileges was added in MDEV-10404 due to a kernel bug correcte
in [1] in Linux 4.14. The only kernel version earlier than this
in a supported distro in RHEL7.

The RHEL 7 latest kernel (linux-3.10.0-1160.119.1.el7) and it has
the patch[1] and therefore selinux isn't the blocker any more.

TODO: There are SUID pam helpers still to check in CI:

[1] https://github.com/torvalds/linux/commit/af63f4193f9fbbbac50fc766417d74735afd87ef