Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Tuukka Pasanen
MDEV-30953: Add package for Galera server for RPM

Following the previous commit, we apply the same
to RPM.

Add new package Mariadb-server-galera to RPM build
recude mariadb-server package dependencies.
Galera cluster scripts and libraries are currently
in package mariadb-server. Move Galera cluster
scripts and libraries to new package mariadb-server-galera
as most of the people does not need them and
it allows to slim down mariadb-server package

The MariaDB-test package unlike its Debian
equal, didn't depend on the MariaDB-server
(and now it does), except if WITH_WSREP is
specified in which case it depends on the
MariaDB-server-galera package

Also from a865a7c23bd8c06bf1de0a759703f898f6c180c2
lsof, socat and pv are RECOMMENDS.
* lsof/socat - when selinux is enabled, these aren't used
* pv - the galera sst scripts will silently ignore its absence

Updated/Edited by: Daniel Black <[email protected]>
sjaakola
MDEV-21935 Mariabackup file limits set incorrectly

Fixed a build time regression happening in x86-debian-12 platform
Thirunarayanan Balathandayuthapani
MDEV-36436  Assertion "unexpected references" == 0 after ALTER IGNORE TABLE

Problem:
=========
An assertion failure occurs in InnoDB during consecutive
ALTER TABLE operations using the COPY algorithm. The crash
happens during the table rename phase because the
source table (dict_table_t::n_ref_count) is non-zero, despite
the thread holding an exclusive Metadata Lock (MDL).

Reason:
========
When ALTER IGNORE TABLE is executed via the COPY algorithm,
it generates undo logs for every row inserted into the
intermediate table (e.g., #sql-alter-...). The background Purge
Thread, responsible for cleaning up these undo logs, attempts
to take an MDL on the table to prevent the table from
being dropped while in use.

Race condition:
==================
First ALTER: Creates #sql-alter-, copies data, and renames it to t1.

Purge Activation: The Purge thread picks up the undo logs from step 1.
It takes an MDL on the temporary name (#sql-alter-) and increments
the table's n_ref_count.

Identity Shift: InnoDB renames the physical table object to t1, but
the Purge thread still holds a reference to this object.

Second ALTER: Starts a new copy process. When it attempts to rename
the "new" t1 to a backup name, it checks if n_ref_count == 0.
Because the Purge thread is still "pinning" the object to
clean up logs from the first ALTER, the count is > 0,
triggering the assertion failure.

Solution:
========
ALTER IGNORE TABLE needs row-level undo logging to easily
roll back the last inserted row in case of duplicate key errors.
By discarding the last undo log record after inserting each row,
purge will not process any log records generated by
ALTER IGNORE TABLE, preventing unexpected access from the purge
subsystem during subsequent DDL operations.

Make skip_alter_undo (1-bit) to (2-bit enum)
to support different ALTER operation modes:

- NO_UNDO (0): Normal mode with standard undo logging
- SKIP_UNDO (1): ALTER mode that skips undo logging
- IGNORE_UNDO (2): ALTER IGNORE mode that rewrites undo blocks

trx_undo_report_row_operation(): Add ALTER IGNORE undo
rewriting logic. Store old undo record info before writing
new records for IGNORE_UNDO mode. Reset undo top_offset
to maintain only latest insert undo

row_ins_clust_index_entry_low(): Prevent ALTER IGNORE from
using bulk insert optimization. since it requires to write
undo log for each row operation that's incompatible with
existing bulk operation.
Mohammad Tafzeel Shams
MDEV-38079: Crash recovery fails after ALTER TABLE…PAGE_COMPRESSED=1

Issue:
- Instant enabling of page_compressed may cause crash recovery failure.
- FSP_SPACE_FLAGS may not match the actual page format during recovery.

Root Cause:
- Table t has pending writes.
- ALTER TABLE t PAGE_COMPRESSED=1 is executed.
- Some pending pages are written in compressed format, but the server may
  crash before all pages are rewritten.
- During recovery, FSP_SPACE_FLAGS may indicate non-compressed while some
  pages are compressed which results in space ID mismatch.

Fix:
- We go for the simple fix of requiring the table to be rebuilt when the
  page_compressed attribute is changed, because:
  - The page_compressed option is used very rarely.
  - The only motivation of page_compressed=1 is to save space, and space may
    only be saved when all pages of the table are rewritten (table is rebuilt).

- alter_options_need_rebuild(): Enabling page_compressed will require a
  full table rebuild.
Marko Mäkelä
MDEV-29445 fixup: Remove a bogus assertion

buf_buddy_shrink(): Remove a redundant assertion
ut_ad(buf_pool.will_be_withdrawn(bpage->zip.data, size))
that is using an incorrect size parameter. In the only caller
buf_pool_t::shrink(size_t size) we already checked with the
correct size that the ROW_FORMAT=COMPRESSED page frame will be
in the being-trimmed portion of the buffer pool. If
p >= memory + size held for a larger size, it would also trivially
hold with the smaller (and incorrect) size.

This code is covered by the test innodb.innodb_buffer_pool_resize,
which uses ROW_FORMAT=COMPRESSED tables.
Sergei Golubchik
MDEV-38587 slow_query_log missing from debian conf

add a line to actually enable slow query log, not only to
configure it
Sergei Golubchik
MDEV-38778 mariadbcheck --silent

silence non-errors from admin commands, as it's what --silent
is documented to do
forkfun
MDEV-26112 STR_TO_DATE should work with lc_time_names
nada
MDEV-38624: disallow skipping ER_CONNECTION_KILLED

Prevent ER_CONNECTION_KILLED (1927) from being added to the slave_error_mask bitmap in init_slave_skip_errors(), even when explicitly listed in --slave-skip-errors or when using --slave-skip-errors=all. A warning is emitted in the error log when either is attempted.
ParadoxV5
MDEV-38825 Slave delays when there is "nothing" to replicate

Delayed Replication does not delay on certain metadata events,
but not every type of “non-user” event.
GTID List and Binlog Checkpoint are two of them,
and because the primary sends them after a replica connects,
this replica delays upon START SLAVE even when it’s already up-to-date.

This commit fixes this oddity by changing this condition
set to check the entire non-group event category.
Thirunarayanan Balathandayuthapani
MDEV-38928 Assertion `undo_no <= 1' failed in trx_t::reset_and_truncate_undo()

Problem:
========
1) ALTER IGNORE TABLE...ALGORITHM=COPY on an InnoDB temporary table
triggers an assertion failure in trx_t::reset_and_truncate_undo().
The IGNORE_UNDO mode rewrites the insert undo log to retain only
the latest record, but temporary tables undo logs are not processed
by the purge thread. Applying this optimization
to a temporary table is incorrect.

2) For partitioned tables with ALTER IGNORE...ALGORITHM=COPY, the
IGNORE_UNDO in trx_undo_report_row_operation() resets trx->undo_no
to 0, but trx->mod_tables retains entries from earlier partitions.
When a different partition (dict_table_t) is inserted into, its
mod_tables entry is created with first=1. After the rewinding, it
writes the undo record with top_undo_no=0, time.valid(0) fails
because first(1) > 0.

3) The savepoint for partial rollback also did not account for
IGNORE_UNDO mode, where trx->undo_no is continuously reset to 0.

Solution:
=========
ha_innobase::extra(): Set skip_alter_undo=IGNORE_UNDO only for
non-temporary tables during HA_EXTRA_BEGIN_ALTER_IGNORE_COPY.
Temporary tables keep NORMAL_UNDO since purge won't process
write temporary table undo logs.

trx_t::reset_and_truncate_undo(): Remove the table parameter
because same transaction could modify multiple tables in
case of partition.

row_insert_for_mysql(): Use savept=0 for IGNORE_UNDO mode. So that
partial rollback target the last inserted undo record.

trx_undo_report_row_operation(): After rewinding trx->undo_no to 0
in IGNORE_UNDO mode, clear mod_tables and re-emplace the
table with first=0.
Thirunarayanan Balathandayuthapani
MDEV-38928 Assertion `undo_no <= 1' failed in trx_t::reset_and_truncate_undo()

Problem:
========
1) ALTER IGNORE TABLE...ALGORITHM=COPY on an InnoDB temporary table
triggers an assertion failure in trx_t::reset_and_truncate_undo().
The IGNORE_UNDO mode rewrites the insert undo log to retain only
the latest record, but temporary tables undo logs are not processed
by the purge thread. Applying this optimization
to a temporary table is incorrect.

2) For partitioned tables with ALTER IGNORE...ALGORITHM=COPY, the
IGNORE_UNDO in trx_undo_report_row_operation() resets trx->undo_no
to 0, but trx->mod_tables retains entries from earlier partitions.
When a different partition (dict_table_t) is inserted into, its
mod_tables entry is created with first=1. After the rewinding, it
writes the undo record with top_undo_no=0, time.valid(0) fails
because first(1) > 0.

3) The savepoint for partial rollback also did not account for
IGNORE_UNDO mode, where trx->undo_no is continuously reset to 0.

Solution:
=========
ha_innobase::extra(): Set skip_alter_undo=IGNORE_UNDO only for
non-temporary tables during HA_EXTRA_BEGIN_ALTER_IGNORE_COPY.
Temporary tables keep NORMAL_UNDO since purge won't process
write temporary table undo logs.

trx_t::reset_and_truncate_undo(): Remove the table parameter
because same transaction could modify multiple tables in
case of partition.

trx_undo_report_row_operation(): After rewinding trx->undo_no to 0
in IGNORE_UNDO mode, clear mod_tables and re-emplace the
table with first=0.

row_insert_for_mysql(): Use savept=0 for IGNORE_UNDO mode. So that
partial rollback target the last inserted undo record.

Change TRX_DML_BULK=2, TRX_DDL_BULK=3 so that bit 1 is set for all
bulk insert modes and for dict_table_t::IGNORE_UNDO. This allows
to replace the savept conditional with a single bit test:
(trx->bulk_insert | table->skip_alter_undo) & 2
bsrikanth-mariadb
MDEV-38805:Store optimizer_context into a IS table

Currently, optimizer context is written as JSON sub-element in the
Optimizer Trace.

In this task, we separate it out from the optimizer trace, and instead
store it in optimizer_context Information Schema table.

The structure of the context is changed to look like below: -
----------------------------------
SET var_name1=val1;

SET var_name2=val2;
.
.
.

CREATE TABLE t1 ( ... );
-- in case it is a constant table
REPLACE INTO t1 VALUES (...);

CREATE TABLE t2 ( ... );
...

set @context='{ JSON with all the captured calls }';
set @optimizer_context='context';

... the original query;
----------------------------------

The IS can be used to read the current stored context, as well as to
dump it to a sql file which can later be replayed in a different
environment.

It is done like below: -
--------------------------------------
set optimizer_record_context=ON;
set optimizer_trace=1

-- sample query

select into outfile '/tmp/captured-context.sql' context from information_schema.OPTIMIZER_CONTEXT;
---------------------------------------

All the existing tests are modified to query OPTIMIZER_CONTEXT IS table
Daniel Black
deb: format debian/control of videx
Raghunandan Bhat
MDEV-38864: Use mmap for MEMORY engine allocations

MEMORY engine's data and indexes blocks are allocated using `my_malloc`.
This created internal fragmentation within the system allocator, causing
gradual memory growth and leading to OOM server crash.

This patch introduces memory mapping for MEMORY engine's allocations.
- Implements `my_vmalloc` and `my_vmrelease` functions.
- These functions wrap `my_virtual_mem_alloc` & `my_virtual_mem_release`
  to provide error handling, interface with Performance Schema(PSI) and
  global memory accounting.
sjaakola
MDEV-21935 Mariabackup file limits set incorrectly

When a cluster donor node executes mariabackup SST, it will use same
approximation for max open files limit as was set for the mariadbd
process during the server startup. This may be a problem for installation
where mariabackup execution would need higher open file count, and might
crash for exceeding the too tight open file limit.

The reason for this behavior is that when mariadbd server calculates the
expected max open files count, it will record this open file count approximation
as system ulimit value, both as soft and hard limit. Later, when the node
operates as SST donor, it spawns mariabackup executable, which now inherits the
ulmit setting used for the mariadbd process. Mariabackup tries to enforce
open_files_limit variable value configured in [mariabackup] group in the my.cnf
fle, but this will fail if hard ulimit value is smaller.

The fix in this commit records the approximated max open file count only as soft
ulimit value. If hard ulimit is higher or unlimited, there remains head room for
the mariabackup to use higher open_files_limit configuration.
Sergei Golubchik
MDEV-38868 Assertion `*str != '\0'' failed in my_message_sql on CREATE TABLE

avoid empty error messages in failed CONNECT assisted discovery
Marko Mäkelä
MDEV-37412 fixup: Allow more FILE_CHECKPOINT writes

The test was occasionally failing on Microsoft Windows, with the message
saying 12514, which was exactly 16 more than the maximum value that our
regular expression tolerated. Let us add some more slack.
Georgi Kodinov
Merge branch '10.11' into 10.6-mdev-38673
bsrikanth-mariadb
MDEV-38805:Store optimizer_context into a IS table

Currently, optimizer context is written as JSON sub-element in the
Optimizer Trace.

In this task, we separate it out from the optimizer trace, and instead
store it in optimizer_context Information Schema table.

The structure of the context is changed to look like below: -
----------------------------------
SET var_name1=val1;

SET var_name2=val2;
.
.
.

CREATE TABLE t1 ( ... );
-- in case it is a constant table
REPLACE INTO t1 VALUES (...);

CREATE TABLE t2 ( ... );
...

set @context='{ JSON with all the captured calls }';
set @optimizer_context='context';

... the original query;
----------------------------------

The IS can be used to read the current stored context, as well as to
dump it to a sql file which can later be replayed in a different
environment.

It is done like below: -
--------------------------------------
set optimizer_record_context=ON;
set optimizer_trace=1

-- sample query

select into outfile '/tmp/captured-context.sql' context from information_schema.OPTIMIZER_CONTEXT;
---------------------------------------

All the existing tests are modified to query OPTIMIZER_CONTEXT IS table
Vladislav Vaintroub
MDEV-14443 DENY statement

Implements DENY/REVOKE DENY and associated tasks.
Sergei Golubchik
MDEV-38827 Assertion `str[strlen(str)-1] != '\n'[ || strlen(str) == 512-1]' failed in my_message_sql

remove trailing '\n' from CONNECT error messages
Thirunarayanan Balathandayuthapani
MDEV-38882 Assertion in diagnostics area on DDL stats

- This issue shares the same root cause of MDEV-38667. This
patch fixes the remaining caller of HA_EXTRA_END_ALTER_COPY by
temporarily setting abort_on_warning to false that prevents
the warning to error escalation.
Marko Mäkelä
MDEV-37412/MDEV-38289 fixup: Allow more FILE_CHECKPOINT

The test was occasionally failing on Microsoft Windows, with the message
saying 12514, which was exactly 16 more than the maximum value that our
regular expression tolerated. Let us add some more slack, up to 12690.
Marko Mäkelä
MDEV-38832 test case fixup

The test innodb.stat_tables would fail with a result difference
when using non-default values of the global variables.
Georgi Kodinov
Merge branch '10.11' into 10.6-mdev-38673
Tuukka Pasanen
MDEV-30953: Add package for Galera server for RPM

Following the previous commit, we apply the same
to RPM.

Add new package Mariadb-server-galera to RPM build
recude mariadb-server package dependencies.
Galera cluster scripts and libraries are currently
in package mariadb-server. Move Galera cluster
scripts and libraries to new package mariadb-server-galera
as most of the people does not need them and
it allows to slim down mariadb-server package

The MariaDB-test package unlike its Debian
equal, didn't depend on the MariaDB-server
(and now it does), except if WITH_WSREP is
specified in which case it depends on the
MariaDB-server-galera package

Also from a865a7c23bd8c06bf1de0a759703f898f6c180c2
lsof, socat and pv are RECOMMENDS.
* lsof/socat - when selinux is enabled, these aren't used
* pv - the galera sst scripts will silently ignore its absence

Updated/Edited by: Daniel Black <[email protected]>
Sergei Golubchik
MDEV-38811 crash in information_schema.table_constraints when --skip-grant-tables

acl_get_all3() wasn't expecting --skip-grant-tables
Thirunarayanan Balathandayuthapani
Merge branch 10.6 into 10.11
Marko Mäkelä
Merge 10.6 into 10.11
Marko Mäkelä
MDEV-38671: Default to large innodb_buffer_pool_size_max (except on AIX)

my_large_virtual_alloc(): If large page allocation fails, clear
my_use_large_pages. Outside large page allocation (and IBM AIX),
invoke mmap(2) with PROT_NONE so that no page table entries will
have to be allocated. Also, fix the error reporting.

my_virtual_mem_commit(): Do not set the Linux specific flag MAP_POPULATE
because it would introduce a performance regression on startup.
There is a better, system-wide way to disable the overcommit:

sudo sysctl vm.overcommit_memory=2

HAVE_UNACCESSIBLE_AFTER_MEM_DECOMMIT: Remove. We will always make
the virtual memory unaccessible, so that the operating system
can deallocate the page table entries.

buf_pool_t::create(): Skip the my_virtual_mem_commit() on AIX only.

buf_pool_t::page_guess(): Unconditionally check if the pointer is
within the currently allocated buffer pool.

innodb_buffer_pool_size_max: On 64-bit systems other than IBM AIX,
we will default to 8 TiB.

On 32-bit systems, we retain the default innodb_buffer_pool_size_max=0,
that is, by default, SET GLOBAL innodb_buffer_pool_size=... will be
unable to increase from the initial value.

There is no functional change to the startup logic that adjusts
innodb_buffer_pool_size_max to be at least innodb_buffer_pool_size
rounded up to the extent size (2 MiB on 32-bit, 8 MiB on 64-bit).
Tuukka Pasanen
MDEV-30953: Add package for Galera server (Deb)

MDEV-38744 removed the galera as a server dependency,
and we've decided to put it back.

While we put it back and there's a fresh release
series, lets incorporate some packaging improvements.

Historicly, since 10.1 when Galera became part
of the server the MariaDB-server package included
all the dependencies of Galera, its SST scripts,
in case the user wanted the. There where many
cases where this wasn't the case. As we
separate out to a separate mariadb-server-galera
package, we give the option if they want a
galera install and have its dependencies, or
a more minimal installation.

To facilitate this:

Add new package mariadb-server-galera to Debian to
recude mariadb-server package dependencies.
Galera cluster scripts and libraries are currently
in package mariadb-server. Move Galera cluster
scripts and libraries to new package mariadb-server-galera
as most of the people does not need them and
it allows to slim down mariadb-server package

More explictly:

As such this moves the following files that were previously
in the MariaDB-server packgage to the MariaDB-server-galera package:
* README-wsrep
* Galera SST scripts
* /etc/systemd/system/[email protected]/use_galera_new_cluster.conf

The dependencies of the Galera SST scripts are moved to
the dependencies of the MariaDB-server-galera package.

As the installation of this package is a key indication that
galera is required 60-galera.cnf in the configuration has
explict galera required configuration items set. The
wsrep_cluster_address will however be required to be set by the
user.

The galera-4 is also a dependency of MariaDB-server-galera package
rather than the MariaDB-server package.

MariaDB users previously using galera installed instance, that are
doing a major upgrade will need to be aware that the new package
is required.

Non-galera users during a major upgrade are going to get a slightly
smaller install, and a bunch of previous installed dependencies that
are no longer required and can be removed.

The mariadb-test package change its dependency from mariadb-server
to mariadb-server-galera so all the requirement of the server
are there.

Updated/Edited by: Daniel Black <[email protected]>