Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Marko Mäkelä
MDEV-39142 InnoDB fails to start up with CONFIG_ARM64_VA_BITS_39=y

Some 64-bit ARM targets, such as the Raspberry Pi 4, use a Linux kernel
build option that limits the width of virtual addresses to 39 bits,
corresponding to 512 GiB. This would prevent a reservation of
the default innodb_buffer_pool_size_max=8t.

buf_pool_t::create(): If we fail to allocate virtual address space
for the default innodb_buffer_pool_size_max=8TiB on a 64-bit system,
let us retry with 128 GiB or innodb_buffer_pool_size,
whichever is greater.

buf_pool.size_in_bytes_max_default, buf_pool.size_in_bytes_max_minimum:
These constexpr size_t replace innodb_buffer_pool_size_max_default and
innodb_buffer_pool_size_max_minimum, respectively.

my_large_virtual_alloc(): Do not report EE_OUTOFMEMORY becaues we are
not actually running out of memory but only virtual address space.
buf_pool_t::create() will already report an appropriate error message.

On AMD64, POWER and SPARC, the size of the virtual address space
should always be at least 47 bits.

For 64-bit ARM, Linux also offers CONFIG_ARM64_VA_BITS_36, which would
limit the virtual address space to 16 GiB. This setting should be rare
even on embedded systems, such as OpenWrt.

For RISC-V, the minimum appears to be CONFIG_VA_BITS_SV39.

On MIPS, the virtual address space could be 40 bits or less, but we
don't know how much less. On the MIPS based LoongArch, the Linux
kernel will read the VALEN bits from the first cpucfg register.

We will assume that 39 bits is a sensible least common denominator for
64-bit ARM, RISC-V, MIPS, and LoongArch.

If this logic fails and InnoDB fails to start up on some 64-bit platform,
it will always be possible to specify a smaller value than the default
innodb_buffer_pool_size_max=8t.

Reviewed by: Thirunarayanan Balathandayuthapani
Marko Mäkelä
MDEV-39142 InnoDB fails to start up with CONFIG_ARM64_VA_BITS_39=y

Some 64-bit ARM targets, such as the Raspberry Pi 4, use a Linux kernel
build option that limits the width of virtual addresses to 39 bits,
corresponding to 512 GiB. This would prevent a reservation of
the default innodb_buffer_pool_size_max=8t.

buf_pool_t::create(): If we fail to allocate virtual address space
for the default innodb_buffer_pool_size_max=8TiB on a 64-bit system,
let us retry with 128 GiB or innodb_buffer_pool_size,
whichever is greater.

buf_pool.size_in_bytes_max_default, buf_pool.size_in_bytes_max_minimum:
These constexpr size_t replace innodb_buffer_pool_size_max_default and
innodb_buffer_pool_size_max_minimum, respectively.

my_large_virtual_alloc(): Do not report EE_OUTOFMEMORY becaues we are
not actually running out of memory but only virtual address space.
buf_pool_t::create() will already report an appropriate error message.

On AMD64, POWER and SPARC, the size of the virtual address space
should always be at least 47 bits.

For 64-bit ARM, Linux also offers CONFIG_ARM64_VA_BITS_36, which would
limit the virtual address space to 16 GiB. This setting should be rare
even on embedded systems, such as OpenWrt.

For RISC-V, the minimum appears to be CONFIG_VA_BITS_SV39.

On MIPS, the virtual address space could be 40 bits or less, but we
don't know how much less. On the MIPS based LoongArch, the Linux
kernel will read the VALEN bits from the first cpucfg register.

We will assume that 39 bits is a sensible least common denominator for
64-bit ARM, RISC-V, MIPS, and LoongArch.

If this logic fails and InnoDB fails to start up on some 64-bit platform,
it will always be possible to specify a smaller value than the default
innodb_buffer_pool_size_max=8t.
Nikita Malyavin
MDEV-38468 GTT: use-after-free after failed BINLOG event in XA mode

Deny setting pseudo_slave_mode for GTT even when rli_fake is set.

Parent issue:
MDEV-35915 Implement Global temporary tables
Thirunarayanan Balathandayuthapani
MDEV-39081 InnoDB: tried to purge non-delete-marked record, assertion fails in row_purge_del_mark_error

Reason:
=======
Following the changes in MDEV-38734, the server no longer marks
all indexed virtual columns during an UPDATE operation.
Consequently, ha_innobase::update() populates the upd_t vector
with old_vrow but omits the actual data for these virtual columns.

Despite this omission, trx_undo_page_report_modify() continues to
write metadata for indexed virtual columns into the undo log. Because
the actual values are missing from the update vector, the undo log
entry is recorded without the historical data for these columns.

When the purge thread processes the undo log to reconstruct a
previous record state for MVCC, it identifies an indexed virtual
column but finds no associated data.

The purge thread incorrectly interprets this missing data as a NULL
value, rather than a "missing/unrecorded" value. The historical
record is reconstructed with an incorrect NULL for the virtual column.
This causes the purge thread to incorrectly identify and purge
records that are not actually delete-marked, leading to abort
of server.

Solution:
=========
ha_innobase::column_bitmaps_signal(): Revert the column-marking
logic to the state prior to commit a4e4a56720c, ensuring all
indexed virtual columns are unconditionally marked during an UPDATE.

- The previous "optimization" attempted to manually detect indexed
column changes before marking virtual columns. The manual check
for indexed column modifications is redundant. InnoDB already
provides the UPD_NODE_NO_ORD_CHANGE flag within row_upd_step().
This flag is being used in trx_undo_page_report_modify() and
trx_undo_read_undo_rec() should log or read virtual column values.

- For INSERT and DELETE (outside of Online DDL contexts), the
function now exits early. Since the MySQL server layer manages
virtual column computation in TABLE::mark_virtual_columns_for_write()
for inserts, handler-level re-marking provided no value.
Christian Hesse
MDEV-35969 wsrep: add more details in service manager status (10.11)

Let's show the difference between donor and joiner.

Signed-off-by: Julius Goryavsky <[email protected]>
Nikita Malyavin
MDEV-38603 GTT: Failing assertion prebuilt->template_type == ROW_MYSQL_WHOLE_ROW

FLUSH TABLES WITH READ LOCK differs from FLUSH TABLES in that the latter
doesn't open the tables. FLUSH TABLES works fine with temporary tables,
while FLUSH TABLES WITH READ LOCK results in ER_NO_SUCH_TABLE.

So we also shouldn't allow FLUSH TABLES WITH READ LOCK open a child GTT.
Doing so would result in many unpredicted problems, like this assertion.

Particularly: m_prebuilt->select_lock_type stays LOCK_S after
FLUSH TABLE WITH READ LOCK, and then is not reset by
handler::external_lock or handler::start_stmt. However, innodb would
expect LOCK_X in SERIALIZABLE isolation.

In this commit:
Always submit parent GTT to FLUSH TABLES. Since FLUSH TABLES doesn't
open tables, there'll be no difference for it, and FLUSH TABLES WITH
READ LOCK will use a parent table.

Parent issue:
MDEV-35915 Implement Global temporary tables
Rex Johnston
MDEV-29360 Crash when pushing condition with always false IS NULL into derived

When a condition containing an always FALSE range part is pushed
down into a derived table, an enclosed outer reference can cause
a null pointer crash.  An example

SELECT * FROM (SELECT id  FROM t1 GROUP BY id) dt1, (SELECT id FROM t2) dt2, t3
  WHERE dt2.id = t3.id AND dt1.id BETWEEN 0 AND (dt2.id IS NULL);

is considered as one that can be pushed into the derived table dt1
because the expression (dt2.id IS NULL) is identified as always false.
On the on other hand the condition still contains a reference to a
column of the table dt2. When pushing the condition to the where clause
of the derived table dt1 a copy of it is created and this copy is
transformed to be used in the new where clause. When the transformation
procedure encounters a reference dt2.id it crashes the server as only
the references to the columns used in the group by list of the
specification of dt1 or those equal to them are expected in the
condition that can be pushed into the where clause of the derived table.

The fix here is transform the argument to the function Item_func_isnull
from something that may be a column reference, but cannot be null to
an Item_literal that also cannot be null (1).

Based on patch by Igor Babaev.
Oleg Smirnov
MDEV-38728 Improve join size estimation for ref access

When estimating number of rows produced by a join after `ref` access,
the optimizer assumes all driving table values will find matches
in the inner table. This causes overestimation when the driving
table has more distinct values than the inner table's key.

Fix: use number of distinct values (NDV) for columns in the
join predicate to calculate match probability:
  match_prob = min(1.0, NDV(inner) / NDV(driving))
The expected number of records after `ref` access is then multiplied
by match probability to provide more accurate estimate.

Limitations:
- EITS must be available for both columns in the join predicate
- both columns must be real table fields
- only single-column ref access is supported
- only first key part of the inner table's index is used

TODO:
- WHERE filter on the driving table may reduce NDV and affect estimation.
  Currently, it is handled by evaluating of partial joins cardinality,
  but it is an approximation. Can it be done more precisely?

This commit overwrites only those test results which have been verified,
i.e. provided better join size estimation. Other failing tests are not
yet verified.
Mohammad Tafzeel Shams
MDEV-22186: Add innodb_buffer_pool_in_core_dump to exclude InnoDB buffer pool from cores

Problem:
There is no control to include or exclude large InnoDB buffer pool from
core files, which can lead to unnecessarily large core dumps or prevent
capturing useful memory state.

Solution:
Introduce a new global dynamic system variable innodb_buffer_pool_in_core_dump
that allows excluding InnoDB buffer pool from core dumps using MADV_DONTDUMP
where supported. When enabled the server marks relevant memory regions to be
omitted from core files. The madvise state is dynamically updated when the
variable changes.

This variable is only available on Linux & FreeBSD.

Additionally, a test helper script is introduced to inspect
/proc/<pid>/smaps and verify whether large memory regions are marked
for trimming.

- innodb_buffer_pool_in_core_dump
  Global boolean system variable controlling whether InnoDB buffer
  pool should be excluded from core dumps.
  Default: OFF on non-debug builds with MADV_DONTDUMP support,
  ON otherwise.

- innodb_buffer_pool_in_core_dump_update()
  Update hook that sets innodb_buffer_pool_in_core_dump and triggers runtime
  updates of madvise state for the buffer pool.

- buf_pool_t::core_dump_trim()
  Advises MADV_DONTDUMP or MADV_DODUMP according to latest
  innodb_buffer_pool_in_core_dump value.

- buf_pool_t::create()
  Updates madvise flag to the buffer pool memory based on
  innodb_buffer_pool_in_core_dump.

- buf_pool_t::resize()
  Updates the madvise flag for buffer pool to keep it in sync with
  the innodb_buffer_pool_in_core_dump.

- check_core_dump_trim.inc
  Test helper that inspects /proc/<pid>/smaps to detect memory regions
  with the dd VmFlag and determine whether the core file is expected
  to be trimmed based on buffer pool and log buffer sizes.
Nikita Malyavin
MDEV-38937 GTT: 1944 Error executing row event with trigger replication

Row events prelock tables for triggers even if those triggers are not
accessed.

If the table in trigger doesn't exist though, the error wouldn't be
triggered.

Make the same for GTT:
ignore its open denials on slave if it's meant so.

Parent issue:
MDEV-35915 Implement Global temporary tables
Nikita Malyavin
MDEV-38481 GTT: Assertion failed in ha_maria::start_stmt on ANALYZE

Assertion 'file->trn == trn' failed in ha_maria::start_stmt on
ANALYZE TABLE, also assertion 'trn' failed in another test.

file->trn is NULL for a newly opened table and is normally assigned by
st_thr_lock::start_trans in mysql_lock_tables.

Same for thd->ha_data[maria_hton->slot].ha_ptr aka 'trn', which is
NULLed when the old table is dropped as part of OR REPLACE action, if
that was the only table in the aria locked tables. Similarly, it is
restored back on st_thr_lock::start_trans in mysql_lock_tables.

Since ANALYZE is registered as operating on the parent table,
open_only_one_table opens and locks the latter. lock_tables doesn't
issue a table lock in locked tables mode, but issues
table->file->start_stmt(...), which asserts for ha_maria.

Solution: lock the parent GTT table as well in
select_create::create_table_from_items. It will be unlocked altogether
within the lock saved in select_create as m_plock.

Parent issue:
MDEV-35915 Implement Global temporary tables
Nikita Malyavin
MDEV-37660 Make Global Temporary Tables DDL errors more specific

Add ER_TABLE_IN_USE code.
Use it for ALTER, DROP and RENAME when a child Global temporary table
exists in the same connection (effectively blocking the statement)

Parent issue:
MDEV-35915 Implement Global temporary tables
bsrikanth-mariadb
MDEV-39159: Disable --view-protocol for main.opt_context_store_sys_vars

Result file mismatch is noticed when the opt_context_sys_vars test is run
with "view protocol" option. With the option, a view is added on top of the
existing query, and that gets stored in the sql dump file.

In the test, dump file gets loaded and read back. But, because the view query
is different from the stored query in the result file, result mismatch
is reported.

We need to disable the test to run with --view option.
Rex Johnston
MDEV-29360 Crash when pushing condition with always false IS NULL into derived

When a condition containing an always FALSE range part is pushed
down into a derived table, an enclosed outer reference can cause
a null pointer crash.  An example

SELECT * FROM (SELECT id  FROM t1 GROUP BY id) dt1, (SELECT id FROM t2) dt2, t3
  WHERE dt2.id = t3.id AND dt1.id BETWEEN 0 AND (dt2.id IS NULL);

is considered as one that can be pushed into the derived table dt1
because the expression (dt2.id IS NULL) is identified as always false.
On the on other hand the condition still contains a reference to a
column of the table dt2. When pushing the condition to the where clause
of the derived table dt1 a copy of it is created and this copy is
transformed to be used in the new where clause. When the transformation
procedure encounters a reference dt2.id it crashes the server as only
the references to the columns used in the group by list of the
specification of dt1 or those equal to them are expected in the
condition that can be pushed into the where clause of the derived table.

The fix here is transform the argument to the function Item_func_isnull
from something that may be a column reference, but cannot be null to
an Item_literal that also cannot be null (1).

Based on patch by Igor Babaev.
Nikita Malyavin
MDEV-38937 GTT: 1944 Error executing row event with trigger replication

Row events prelock tables for triggers even if those triggers are not
accessed. Write_row event is known to trigger any kind of row operation:

uint8 Write_rows_log_event::get_trg_event_map() const
{
  return trg2bit(TRG_EVENT_INSERT) | trg2bit(TRG_EVENT_UPDATE) |
          trg2bit(TRG_EVENT_DELETE);
}
So no surprise that AFTER UPDATE trigger is prepared for insert event.

If the table in trigger doesn't exist though, the error wouldn't be
triggered.

Make the same for GTT:
ignore its open denials on slave if it's meant so.

Parent issue:
MDEV-35915 Implement Global temporary tables
Nikita Malyavin
MDEV-37896 global_temporary_table tests are not stable on 2nd run

replace con_id and purge logs

Parent issue:
MDEV-35915 Implement Global temporary tables
Christian Hesse
MDEV-35969 wsrep: set new status for service manager (10.11 backport)

On WSREP state transfer the status in service manager changes to:

    WSREP state transfer (role ...) ongoing...

This status was not changed after state transfer was complete. Let's
change it again to reflect now situation:

    WSREP state transfer (role ...) comleted.

Signed-off-by: Julius Goryavsky <[email protected]>
Nikita Malyavin
Revert "MDEV-38683 SIGSEGV (dbg), SIGABRT or ER_EMPTY_QUERY when using ROWS EXAMINED with log_output=TABLE"

This reverts commit 5d26d515fc316c1509e22b00eb5453aa288ee7ad.
Rex Johnston
MDEV-29360 Crash when pushing condition with always false IS NULL into derived

When a condition containing an always FALSE range part is pushed
down into a derived table, an enclosed outer reference can cause
a null pointer crash.  An example

SELECT * FROM (SELECT id  FROM t1 GROUP BY id) dt1, (SELECT id FROM t2) dt2, t3
  WHERE dt2.id = t3.id AND dt1.id BETWEEN 0 AND (dt2.id IS NULL);

is considered as one that can be pushed into the derived table dt1
because the expression (dt2.id IS NULL) is identified as always false.
On the on other hand the condition still contains a reference to a
column of the table dt2. When pushing the condition to the where clause
of the derived table dt1 a copy of it is created and this copy is
transformed to be used in the new where clause. When the transformation
procedure encounters a reference dt2.id it crashes the server as only
the references to the columns used in the group by list of the
specification of dt1 or those equal to them are expected in the
condition that can be pushed into the where clause of the derived table.

The fix here is transform the argument to the function Item_func_isnull
from something that may be a column reference, but cannot be null to
an Item_literal that also cannot be null (1).

Based on patch by Igor Babaev.
Nikita Malyavin
MDEV-38448 assertion ...tdc->ref_count > 0 failed on LOCK gtt+open error

If open_global_temporary_table failed, a goto was to an incorrect place.
In locked_tables_mode, an extra acquire didn't happen, so a release on
error shouldn't have happened as well (i.e. goto err_lock).
That is, in locked_tables_mode, just return immediately on error.

The easiest way to reproduce it was with DML while pseudo_slave_mode=1,
but actually any error inside open_global_temporary_table would do.

Parent issue:
MDEV-35915 Implement Global temporary tables
Marko Mäkelä
MDEV-39170 innodb.mem_pressure,32bit fails with wrong result

In commit 36eba98817339f53cc57f1d4884ace3efb38db8d (MDEV-19123)
the test was changed but the result was not adjusted for
32-bit targets accordingly.
Nikita Malyavin
MDEV-38937 GTT: 1944 Error executing row event with trigger replication

Row events prelock tables for triggers even if those triggers are not
accessed. Write_row event is known to trigger a wider set of row
operations. In IDEMPOTENT mode it is

  trg2bit(TRG_EVENT_INSERT) | trg2bit(TRG_EVENT_DELETE);

So no surprise that AFTER DELETE trigger is prepared for insert event.

If the table in trigger doesn't exist though, the error wouldn't be
triggered.

Make the same for GTT:
ignore its open denials on slave if it's meant so.

Parent issue:
MDEV-35915 Implement Global temporary tables
Nikita Malyavin
MDEV-38480 GTT: Aria gets error HA_ERR_CRASHED_ON_USAGE after TRUNCATE

Do not call HA_EXTRA_PREPARE_FOR_DROP for the parent table

Parent issue:
MDEV-35915 Implement Global temporary tables
Marko Mäkelä
MDEV-39142 InnoDB fails to start up with CONFIG_ARM64_VA_BITS_39=y

Some 64-bit ARM targets, such as the Raspberry Pi 4, use a Linux kernel
build option that limits the width of virtual addresses to 39 bits,
corresponding to 512 GiB. This would prevent a reservation of
the default innodb_buffer_pool_size_max=8t.

buf_pool_t::create(): If we fail to allocate virtual address space
for the default innodb_buffer_pool_size_max=8TiB on a 64-bit system,
let us retry with 128 GiB or innodb_buffer_pool_size,
whichever is greater.

On AMD64, POWER and SPARC, the size of the virtual address space
should always be at least 47 bits.

For 64-bit ARM, Linux also offers CONFIG_ARM64_VA_BITS_36, which would
limit the virtual address space to 16 GiB. This setting should be rare
even on embedded systems, such as OpenWrt.

For RISC-V, the minimum appears to be CONFIG_VA_BITS_SV39.

On MIPS, the virtual address space could be 40 bits or less, but we
don't know how much less. On the MIPS based LoongArch, the Linux
kernel will read the VALEN bits from the first cpucfg register.

We will assume that 39 bits is a sensible least common denominator for
64-bit ARM, RISC-V, MIPS, and LoongArch.

If this logic fails and InnoDB fails to start up on some 64-bit platform,
it will always be possible to specify a smaller value than the default
innodb_buffer_pool_size_max=8t.
Nikita Malyavin
MDEV-37872 SIGSEGV on some GTT operations under pseudo_thread_id changed

Deny pseudo_thread_id changing when there are some GTTs are open.
pseudo_thread_id is only used for replication testing. Since GTT child
tables are never opened on replicas, it's fine to do so -- there is no
case to test.

Parent issue:
MDEV-35915 Implement Global temporary tables
Nikita Malyavin
MDEV-37934 Assertion `thd->transaction->stmt.is_empty()...' in GRANT

...on GTT failed

Open a parent handle for GTT on GRANT.
ON COMMIT PRESERVE ROWS does not register handlerton, so it will work
fine.

Parent issue:
MDEV-35915 Implement Global temporary tables
Nikita Malyavin
MDEV-37941 Assertion !thd->rgi_slave failed on INSERT w/ parallel slave

Replicas can't crash on the incorrect/unexpected data. It should react
with an error.

Change the assertion to if and return error if the relay log contains
query with GTT access.

Parent issue:
MDEV-35915 Implement Global temporary tables
Nikita Malyavin
MDEV-37956 use-after-free in mysql_ha_close_table on DROP DATABASE

Drop global temporary tables after dropping handlers.

Parent issue:
MDEV-35915 Implement Global temporary tables
Marko Mäkelä
Merge 12.3 into main
Nikita Malyavin
MDEV-37958 SIGSEGV in ha_mroonga::storage_create_foreign_key on INSERT

More a mroonga bug, but doesn't reproduce otherwise.
Use a correct alter_info, not the one stored in lex.

Parent issue:
MDEV-35915 Implement Global temporary tables
forkfun
MDEV-36183 mariadb-dump backup corrupt if stored procedure ends with comment

When a stored procedure, trigger, or event ends with a line comment (#comment
or --comment), mariadb-dump would append the custom delimiter (;;) on the same line.
This caused the comment to "comment out" the delimiter, leading to syntax errors
during restoration.

Fixed by ensuring a newline is inserted before the delimiter in the dump output.
Nikita Malyavin
MDEV-38567 GTT Context corruption on repeated SQL when autocommit=0

Always reset temporary_tables->committed when
drop_on_commit_delete_tables is called. Do it right in that function.

Parent issue:
MDEV-35915 Implement Global temporary tables
Nikita Malyavin
MDEV-37898 No release GTT MTR tests due to debug_sync injection

Parent issue:
MDEV-35915 Implement Global temporary tables
Mohamed
MDEV-38601 require FEDERATED ADMIN for SHOW CREATE SERVER

SHOW CREATE SERVER was accessible to any user with a global
privilege, without requiring FEDERATED ADMIN. Add a privilege
check before mysql_show_create_server() so that only users
with FEDERATED ADMIN can view server definitions.
Nikita Malyavin
MDEV-37957 Assertion lock_type >= TL_READ_SKIP_LOCKED failed on HANDLER

...READ

Don't set lock_type to TL_IGNORE: open_temporary_table intentionally
sets it to TL_WRITE to "simulate locking".

Parent issue:
MDEV-35915 Implement Global temporary tables
Marko Mäkelä
On POSIX-like platforms, pass a target directory (or stream) handle
Yewon Kwak
MDEV-30354 Fix JSON escaping in optimizer trace

The expanded_query field in optimizer trace could produce invalid JSON when
queries contained special characters. Use st_append_escaped() to properly
handle quotes, backslashes and other special characters in the output.
If st_append_escaped() fails, emit an error message in the expanded_query.

Example:
Before: "expanded_query": "select length('a') AS `LENGTH("a")`"
After:  "expanded_query": "select length('a') AS `LENGTH(\"a\")`"

The output is now properly escaped and valid JSON while maintaining
readability.

Added test cases to mysql-test/main/opt_trace.test to verify the fix.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license
Nikita Malyavin
MDEV-38568 GTT: Assertion thd->mdl_context.is_lock_owner failed in CoR

The crash happens on the junction of GTT, CREATE OR REPLACE...SELECT,
LOCK TABLES, and error during select.

1. The parent table should be dropped with drop_open_table, not
quick_rm_table, because it should first be closed.
2. In select_create::abort_result_set, drop the parent table as well.
3. Store parent parent GTT in select_create -- this improves readability
  and access hugely.

Parent issue:
MDEV-35915 Implement Global temporary tables