Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Sergei Golubchik
MDEV-40186 MEMORY tables incorrectly restart index scan on DELETE

remember the last found key and restart the search (if needed)
from it not from the original one.
Sergei Golubchik
MDEV-40337 store user vars in a dedicated memroot

user_var_entry objects and their names have a connection lifetime,
they exist until the connection ends (or is reset), and then they're
all deleted at once. This is exactly the use case for MEM_ROOT,
let's store them there.

Additionally, let's set MY_ROOT_USE_MPROTECT flag to keep this memroot
off the general heap where user_var_entry values are stored
and where heap buffer overflows can happen.

The latter makes memory allocations for the MEM_ROOT about 10x more
expensive, so let's always start with an empty memroot (= zero overhead
if no user variables are used) and on THD cleanup let's retain one
memroot block (= zero overhead if the next connection takes THD from the
cache and uses user variables up to one block size).

Assisted-By: Claude:claude-4.8-opus
Sergei Golubchik
MDEV-40186 MEMORY tables incorrectly restart index scan on DELETE

remember the last found key and restart the search (if needed)
from it not from the original one.
Sergei Golubchik
MDEV-40341 store read-only sysvars in a read-only root

Store all READ_ONLY sysvar values in the read_only_root

Even though READ_ONLY sysvars are protected, for string variables
it usually means that the pointer cannot be changed. The value it
points to - the string itself - still can be. Let's store all
values of string READ_ONLY sysvars in the read_only_root.

sysvars that point directly into argv are copied to read_only_root.
sysvars that have their values calculated and allocated now
must be explicitly marked with PREALLOCATED to let it know they
have to be free()-d.

Assisted-By: Claude:claude-4.8-opus
Sergei Golubchik
MDEV-40337 store user vars in a dedicated memroot

user_var_entry objects and their names have a connection lifetime,
they exist until the connection ends (or is reset), and then they're
all deleted at once. This is exactly the use case for MEM_ROOT,
let's store them there.

Additionally, let's set MY_ROOT_USE_MPROTECT flag to keep this memroot
off the general heap where user_var_entry values are stored
and where heap buffer overflows can happen.

The latter makes memory allocations for the MEM_ROOT about 10x more
expensive, so let's always start with an empty memroot (= zero overhead
if no user variables are used) and on THD cleanup let's retain one
memroot block (= zero overhead if the next connection takes THD from the
cache and uses user variables up to one block size).

Assisted-By: Claude:claude-4.8-opus
Sergei Golubchik
MDEV-40217 restore error handling

get_table_structure() needs to distinguish between
the error condition and 0 fields to dump
Sergei Golubchik
MDEV-40186 MEMORY tables incorrectly restart index scan on DELETE

remember the last found key and restart the search (if needed)
from it not from the original one.
Sergei Golubchik
cleanup: main.view test
Sergei Golubchik
cleanup: only include my_compare.h into heap code as needed

to avoid name conflict on `get_key_length`
Sergei Golubchik
MDEV-40411 SFORMAT ignores max_allowed_packet

do our own allocator that implements std::allocator interface
but uses our memory accounting (my_malloc) and limits max allocation
(e.g. to max_allowed_packet).

use it for fmt::vformat_to.
later can be used for various std:: stuff too.
drrtuy
fix: extra try-catch during plugin init.
Sergei Golubchik
cleanup: main.view test
Sergei Golubchik
MDEV-23086 Error codes/messages reveal information about table structure

if a user tries to access a table or a database they have no priivleges
on, the error is alwaus "access denied", independently from whether
the object exists or not. Do the same for columns.
Sergei Golubchik
MDEV-40411 SFORMAT ignores max_allowed_packet

do our own allocator that implements std::allocator interface
but uses our memory accounting (my_malloc) and limits max allocation
(e.g. to max_allowed_packet).

use it for fmt::vformat_to.
later can be used for various std:: stuff too.
Sergei Golubchik
MDEV-40217 mariadb-dump --complete-insert emits INSERT statements for tables with only generated columns

post-push fix. if all columns are generated, don't dump any values,
use INSERT INTO `t5` () VALUES (), (), ();
Sergei Golubchik
MDEV-40341 store read-only sysvars in a read-only root

Store all READ_ONLY sysvar values in the read_only_root

Even though READ_ONLY sysvars are protected, for string variables
it usually means that the pointer cannot be changed. The value it
points to - the string itself - still can be. Let's store all
values of string READ_ONLY sysvars in the read_only_root.

sysvars that point directly into argv are copied to read_only_root.
sysvars that have their values calculated and allocated now
must be explicitly marked with PREALLOCATED to let it know they
have to be free()-d.

Assisted-By: Claude:claude-4.8-opus
Rex Johnston
MDEV-19941 "Duplicate column name" while using VALUES table constructor

Copy mysql column name scheme for TVCs (column_[0-9999]).
Arcadiy Ivanov
MDEV-40523 Versioned UPDATE on a HEAP table with blobs corrupts the history row

A system-versioned `UPDATE` of a blob column on a `HEAP` table stored garbage
in the history row, and an `AFTER UPDATE` trigger reading `OLD.<blob>` saw the
same garbage.  Both values are durable: the history row is what
`SELECT ... FOR SYSTEM_TIME ALL` returns, and both reach replicas through the
row-based binlog image.  No ASAN build is needed to reproduce either.

`heap_update()` and `heap_delete()` do not free the old blob chain outright.
They park it, because the SQL layer keeps reading the pre-update row out of
`record[1]` after `ha_update_row()` returns -- `binlog_log_row()` builds the
before-image from it, and an `AFTER UPDATE` trigger reads `OLD.<blob>` from it.
Those are zero-copy pointers straight into `HP_BLOCK`, so freeing the chain
would make them dangle.

`heap_write()` redeemed that parking unconditionally, before allocating.  For a
system-versioned `UPDATE` that is exactly the wrong moment:
`vers_insert_history_row()` does `restore_record(table, record[1])`, blob data
pointer included, so the row being written sources its blob from the chain the
update just parked.  The free put those records on the delete list, where the
allocation immediately below handed them straight back as the history row's own
chain -- with `hp_push_free_block()`'s free-list links already scribbled
through the payload.  Source and destination of the blob copy overlapped, and
`record[1]` was left pointing at reused memory for the rest of the statement.

Triggering statements are every `vers_insert_history_row()` caller reaching a
`HEAP` table whose record buffer was filled by a read: single-table `UPDATE`,
multi-table `UPDATE` (both the on-the-fly and the deferred `do_updates()`
path), `INSERT ... ON DUPLICATE KEY UPDATE`, and the row-based replication
applier.  A versioned `UPDATE` that does not change the blob column is
unaffected -- `heap_update()` keeps the chain and parks nothing.

Three consumers then read corrupted data, all of them out of `record[1]` after
the history-row write has recycled the chain:

- the history row itself, as returned by `SELECT ... FOR SYSTEM_TIME ALL`;
- the row-based binlog before-image, so the corruption reaches replicas;
- `AFTER UPDATE` triggers reading `OLD.<blob>`, so whatever the trigger does
  with that value -- typically writing it to an audit table -- stores wrong
  data, and that write is itself replicated.

The trigger case is the easiest to miss, because `LENGTH(OLD.<blob>)` is still
correct: the length lives in the record buffer and survives, and only the
payload has been recycled.  A `BEFORE UPDATE` trigger on the same table reports
the correct value, which localises the damage to the history-row write that
happens between the two.

The parked chain already holds exactly the bytes the history row needs -- it is
a verbatim copy of the same record.  So instead of freeing it and allocating a
duplicate, the new row adopts it:

- `hp_flush_unaliased_blob_free()` redeems every parked chain **except** ones
  the record being written still sources blob data from.
- `hp_write_blobs()` takes those over: the stored row points at the parked
  chain and no new chain is written.  The pending slot is cleared only once
  every column has succeeded, so the rollback path can tell an adopted chain
  from an allocated one and leaves it parked rather than freeing it.

Both record buffers stay valid, because the chain's contents are never
disturbed.  Adoption also needs no space at all, which matters at
`max_heap_table_size`: the history row previously had to find room for a second
copy of a blob that was already resident, and the parked chain was often the
only reclaimable space.

Aliasing is detected by exact pointer equality against the parked chain head.
`hp_read_blobs()` hands out zero-copy pointers of exactly two forms -- the
chain head, or `chain + recbuffer` -- and reassembles a multi-run chain into
`info->blob_buff`, which can never alias, so the two comparisons in
`hp_blob_sources_chain()` are complete and need no walk of the `HP_BLOCK` tree.
Matching is per blob column, since `pending_blob_chains[i]` is the chain parked
for column `i`.

`REPLACE` was never affected: `HA_EXTRA_WRITE_CAN_REPLACE` makes
`hp_read_blobs()` copy rather than hand out zero-copy pointers.  Internal
temporary tables never park -- they free their chains outright and do not
allocate the array -- so `hp_write_blobs()` guards on it.

`storage/heap/hp_test_blob_alias-t.c` drives the sequence at the `heap_write()`
API for a single-record chain, a zero-copy run and a multi-run chain, and
checks both the stored row and the caller's buffer.  The multi-run case is
reassembled into `info->blob_buff` and so cannot alias; the test asserts which
layout it got, so the coverage cannot silently degrade.

`blob_vers_trigger` covers `OLD.<blob>` in triggers across single-table
`UPDATE`, multiple blob columns, `ON DUPLICATE KEY UPDATE` and multi-table
`UPDATE`, with `BEFORE UPDATE` alongside `AFTER UPDATE` on one table, and
non-versioned `HEAP` and versioned `MyISAM` controls.  `blob_versioning`,
`blob_vers_odku`, `blob_vers_multi`, `blob_vers_repl` and `blob_vers_full`
cover the history row itself for every `vers_insert_history_row()` caller,
replication of the before-image, and the at-capacity case.
Sergei Golubchik
MDEV-39821 heap-use-after-free in heap_rnext with tree indexes

heap_update() forgot to update key_changed
Sergei Golubchik
MDEV-23086 Error codes/messages reveal information about table structure

if a user tries to access a table or a database they have no priivleges
on, the error is alwaus "access denied", independently from whether
the object exists or not. Do the same for columns.
Sergei Golubchik
MDEV-40411 SFORMAT ignores max_allowed_packet

do our own allocator that implements std::allocator interface
but uses our memory accounting (my_malloc) and limits max allocation
(e.g. to max_allowed_packet).

use it for fmt::vformat_to.
later can be used for various std:: stuff too.
Sergei Golubchik
MDEV-39821 heap-use-after-free in heap_rnext with tree indexes

heap_update() forgot to update key_changed
Sergei Golubchik
MDEV-23086 Error codes/messages reveal information about table structure

if a user tries to access a table or a database they have no priivleges
on, the error is alwaus "access denied", independently from whether
the object exists or not. Do the same for columns.
Sergei Golubchik
cleanup: only include my_compare.h into heap code as needed

to avoid name conflict on `get_key_length`
drrtuy
fix: clean DeltaAppender after at rollback or disconnect.
Sergei Golubchik
MDEV-39821 heap-use-after-free in heap_rnext with tree indexes

heap_update() forgot to update key_changed
Sergei Golubchik
fixup! cleanup: only include my_compare.h into heap code as needed
Sergei Golubchik
MDEV-40341 store read-only sysvars in a read-only segment

Protect all READ_ONLY sysvars from run-time changes.

Put them into a separate ELF section with a special attribute
and mprotect() this section after the server is fully initialized.

Verify that they're all protected in the sys_var constructor.

One exception: opt_noacl (--skip-grant-tables) can be changed
from 1 to 0 on FLUSH PRIVILEGES. Let's briefly drop the
protection for this 1->0 change. It can be needed only once
in a server lifetime and only if it was started with --skip-grant-tables

On shutdown the protection is removed, making variables writable
again because shutdown resets some of them during the cleanup

Assisted-By: Claude:claude-4.8-opus
Rex Johnston
MDEV-19941 "Duplicate column name" while using VALUES table constructor

Copy mysql column name scheme for TVCs (column_[0-9999]).
Rex Johnston
MDEV-19941 "Duplicate column name" while using VALUES table constructor

Copy mysql column name scheme for TVCs (column_N).
Sergei Golubchik
MDEV-40341 store read-only sysvars in a read-only segment

Protect all READ_ONLY sysvars from run-time changes.

Put them into a separate ELF section with a special attribute
and mprotect() this section after the server is fully initialized.

Verify that they're all protected in the sys_var constructor.

One exception: opt_noacl (--skip-grant-tables) can be changed
from 1 to 0 on FLUSH PRIVILEGES. Let's briefly drop the
protection for this 1->0 change. It can be needed only once
in a server lifetime and only if it was started with --skip-grant-tables

On shutdown the protection is removed, making variables writable
again because shutdown resets some of them during the cleanup

Assisted-By: Claude:claude-4.8-opus
Thirunarayanan Balathandayuthapani
MDEV-32067 InnoDB linear read-ahead had better be logical

The traditional linear read-ahead, enabled by innodb_read_ahead_threshold,
only helps when consecutively accessed pages have adjacent page numbers.
That is rarely true for B-tree leaf pages: after splits, merges and page
reuse, the logical order of leaves has nothing to do with their
physical page numbers. So for a scan of scattered leaves the
old read-ahead either did nothing or read the wrong pages.

This replaces linear read-ahead of index pages with a logical
read-ahead that prefetches the actual leaf pages a scan is
going to visit, discovered from the B-tree during traversal.
After this change, the exact nonzero value of innodb_read_ahead_threshold
matters only for the read-ahead of undo log pages.

records_in_range() reports the first and last leaf page of a range,
and the optimizer forwards that extent to the engine through
handler::advise_page_range() so read-ahead can be sized and
bounded to the scan.

buf_read_ahead_undo(): Renamed from buf_read_ahead_linear(). Now invoked
only for undo log pages, whose page numbers are sequential. No longer called
on BLOB pages or index pages.

buf_read_ahead_one(): Read ahead a single leaf page. Cannot be disabled.

buf_read_ahead_pages(): Read ahead a set of known B-tree leaf pages.

buf_read_ahead_random(): Split into a decision part (which extent to read,
in buf0buf.cc) and an action part (issue the reads).

The pending-read throttle now compares os_aio_pending_reads_approx() with
buf_pool.curr_size() (a page count) instead of curr_pool_size() (bytes),
which had made the throttle a no-op. buf_read_ahead_pages() releases the
tablespace reference on a failed/corrupted page.

btr_read_ahead_t: A context which records the level-1 page and the
child page number of the last node pointer collected,
so read-ahead can be resumed later.

btr_read_ahead_collect(): scan the node pointers of a PAGE_LEVEL=1
page from a given record in scan order, appending child page numbers.

btr_cur_t::search_leaf(), btr_cur_t::open_leaf(): At PAGE_LEVEL=1, after the
descent has located the child to follow, harvest leaf page numbers starting
at that child in the scan direction. These are exactly the
leaves the cursor will visit, so the prefetch is precise regardless of
physical page numbers.

btr_read_ahead_resume_rec(): Locate the record from which read-ahead
should resume, by re-finding the last harvested child by value. It walks only
genuine records, so a concurrent page reorganization cannot cause an invalid
read (a stored byte offset could).

btr_pcur_move_to_next_page(), btr_pcur_move_backward_from_page(): Invoke
buf_read_ahead_one() on the following/preceding sibling.

btr_copy_blob_prefix(): Simplified; no longer relies on FIL_PAGE_PREV/NEXT.

advise_page_range(): Records the scan's estimated leaf extent.
Used only to size and bound read-ahead; it never
gates the pages the scan actually reads.
mrr_readahead_from_scan_range() derives the read-ahead ceiling from
the advised extent, falling back to a LIMIT-based estimate.
The window ramps up from small number of pages, doubling on each
refill toward the ceiling, so single-row probes such as index_first() for
MIN()/MAX() prefetch only a few leaves while long scans reach full depth.

start_readahead(): For a just-positioned scan, prefetch the collected batch
and set up the rolling cursor.

readahead_refill(): As the scan advances (general_fetch()), resume the
level-1 harvest under an index S-latch, chain across level-1 siblings, and
prefetch the next batch bounded by the advised last leaf, or to the end of
the index when the extent is unknown.

Read-ahead is started from index_read() (range and full index scans),
rnd_init()/general_fetch() (full table scans), and, at one-page depth, from
row_merge_read_clustered_index() (OPTIMIZE/ALTER rebuild). general_fetch()
sets active_handler_stats before issuing read-ahead so prefetched pages are
attributed to the query.

A truncation race is closed with a new STOPPING_READS tablespace flag:
mtr_t::commit_shrink() sets it before shrinking and clears it after, and
buf_read_ahead_undo() re-checks space->is_stopping().

trx_undo_get_prev_rec(), trx_undo_get_prev_rec_from_prev_page():
take the trx_undo_t object instead of a long parameter list
and always latch shared.

handler::multi_range_read_info_const() and DsMrr_impl::dsmrr_info_const()
Passed a page_range parameter; the default MRR implementation aggregates
the per-range leaf extents.

opt_range.cc carries it through the chosen QUICK_RANGE_SELECT, and
QUICK_RANGE_SELECT::reset() calls advise_page_range() before
multi_range_read_init().

main.analyze_stmt_prefetch_count: corrected for double counting now that
pages_read_count includes waits on prefetched pages.
Sergei Golubchik
disable connect.odbc_sqlite3 in ASAN builds

LSAN complains about ~400 bytes lost after dlclose of ha_connect.so
that uses unixODBC that uses libsqlite3.so. No leak if connect is
compiled statically or if there's no dlclose. The issue _seems_
to be in one of these libraries that connect loads, so we cannot
fix it.
Sergei Golubchik
MDEV-40341 store read-only sysvars in a read-only root

Store all READ_ONLY sysvar values in the read_only_root

Even though READ_ONLY sysvars are protected, for string variables
it usually means that the pointer cannot be changed. The value it
points to - the string itself - still can be. Let's store all
values of string READ_ONLY sysvars in the read_only_root.

sysvars that point directly into argv are copied to read_only_root.
sysvars that have their values calculated and allocated now
must be explicitly marked with PREALLOCATED to let it know they
have to be free()-d.

Assisted-By: Claude:claude-4.8-opus
Sergei Golubchik
MDEV-40337 store user vars in a dedicated memroot

user_var_entry objects and their names have a connection lifetime,
they exist until the connection ends (or is reset), and then they're
all deleted at once. This is exactly the use case for MEM_ROOT,
let's store them there.

Additionally, let's set MY_ROOT_USE_MPROTECT flag to keep this memroot
off the general heap where user_var_entry values are stored
and where heap buffer overflows can happen.

The latter makes memory allocations for the MEM_ROOT about 10x more
expensive, so let's always start with an empty memroot (= zero overhead
if no user variables are used) and on THD cleanup let's retain one
memroot block (= zero overhead if the next connection takes THD from the
cache and uses user variables up to one block size).

Assisted-By: Claude:claude-4.8-opus
KhaledR57
MDEV-36552 mariadb-dump fails with error 1370 for a view that uses a function

SHOW COLUMNS on a view checked EXECUTE on every function the view
uses. Reading column metadata does not run the function, so
EXECUTE should not be needed. SHOW VIEW is enough.

Check EXECUTE only when the view is created, not when it is opened
just to read its columns.
Sergei Golubchik
MDEV-40341 store read-only sysvars in a read-only segment

Protect all READ_ONLY sysvars from run-time changes.

Put them into a separate ELF section with a special attribute
and mprotect() this section after the server is fully initialized.

Verify that they're all protected in the sys_var constructor.

One exception: opt_noacl (--skip-grant-tables) can be changed
from 1 to 0 on FLUSH PRIVILEGES. Let's briefly drop the
protection for this 1->0 change. It can be needed only once
in a server lifetime and only if it was started with --skip-grant-tables

On shutdown the protection is removed, making variables writable
again because shutdown resets some of them during the cleanup

Assisted-By: Claude:claude-4.8-opus
Sergei Golubchik
cleanup: main.view test