Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
fabdd6631364...
fabdd6631364a483f4fdcb3a50395aa106d38f85
Rex Johnston
MDEV-40012 PQ: refuse a plan that reads a table through a join buffer

A worker has no join buffer. setup_worker_jointabs() stripped the cache off
every worker JOIN_TAB and joined a row at a time, and pwt_table_conds() handed
the worker the buffer's scan filter (cache_select->cond) as well as
select_cond so that it still rejected what the buffer would have. The result
was correct, but it was not the plan the optimizer costed: BNL, BNLH, BKA and
BKAH were chosen for scanning the inner table once per buffer-full of outer
rows, and a worker scanned it once per driving row instead. For a full-scan
inner table that can be orders of magnitude more work than the serial plan
the parallel one is meant to beat.

So the gate now refuses the plan, as the prototype did. The check sits in
can_run_query_in_workers() beside the other per-table refusals, on
tab->cache or tab->use_join_cache, both of which make_join_readinfo() has
settled by the time parallel_join_check() runs in optimize_stage2(). The
refusal names the table in the optimizer trace.

The worker-side handling of a buffered table goes with it:

- pwt_table_conds() becomes pwt_table_cond() and no longer reports
  cache_select->cond. remove_redundant_bnl_scan_conds() only moves conjuncts
  out of select_cond when the tab has a BNL/BNLH cache, so for every tab the
  gate now admits select_cond is already the whole condition. A tab can still
  carry a cache_select without a buffer, because make_join_select() builds the
  scan filter before check_join_cache_usage() decides, but its cond is then a
  copy of conjuncts select_cond still holds. The gate no longer asks whether
  that copy is worker-safe, since no worker evaluates it.
- pwt_clone_table_conds(), which cloned both halves and ANDed them, is gone;
  setup_worker_jointabs() clones the one condition with pwt_clone_rebind().
- setup_worker_jointabs() no longer clears cache, use_join_cache and
  jbuf_tracker on the worker's copy; pwt_assert_tab_inert() asserts that the
  manager's tab has no cache instead. The copy's cache_select is still
  cleared, as it can be set on a tab without a buffer.

parallel_query_join: the BNL case now expects the query to run serially,
with the same rows. The full-scan inner table case (d3) was getting a BNL
buffer under the default join_cache_level, so it runs with
join_cache_level=0 to keep covering a worker-side full scan of an inner
table. parallel_query_why pins the new refusal reason.

This commit was prepared with Claude Code (Opus 5.5), which wrote the gate
check, removed the worker-side join buffer handling it made unreachable,
updated the tests, reconfigured and rebuilt the worktree, and ran the
parallel_query tests in normal and --ps-protocol modes.
f6da36723ee2...
f6da36723ee2a75bb1410db2d7d1e91431ac1940
Sergei Golubchik
MDEV-41655 caching_sha2_password does not limit the password length

let's tighten the limits to be closer to actual OpenSSL limits.

Encrypting with RSA, 2048-bit key and PKCS1 OAEP padding *always*
produces the ciphertext of 256 bytes. And to fit in this, the plaintext
cannot be longer than 214 bytes. Assuming the user can generate her
own keys, 512 bytes would be enough for 4096-bit key. Larger, 8192-bit
keys don't work anyway (don't fit in public_key[] buffer).

And also let's reject too long passwords that can never authenticate.
ed7ef6c67a01...
ed7ef6c67a01db5bc26c787bf888699d7df32632
Rex Johnston
PQ: report per worker what the scan cost it, not just how much it read

ANALYZE FORMAT=JSON says how many rows each worker read, which answers how the
chunks divided the work but not why a worker finished when it did. Two figures
the engine already keeps answer that, once they are kept per worker rather than
summed: how long each worker was busy inside the engine, and how much of that
was waiting for pages to come off disk.

    "r_rows_per_worker": [181967, 93080, 159138, 165815],
    "r_engine_time_per_worker_ms": [69.577, 74.895, 72.507, 69.902],
    "r_pages_read_time_per_worker_ms": [0.276, 0.229, 0.285, 0.283],
    "r_chunks": 18,
    "r_chunks_resplit": 1,

The pair is what makes a straggler readable. Rows without time is a worker that
was handed more to do; time without rows is one that was blocked. Above, four
workers within seven per cent of each other on time despite a two-to-one spread
in rows says the spread is the pull queue's doing, not the disk's. The read
times sum to the pages_read_time_ms already reported in r_engine_stats, so the
array is a decomposition of that total rather than a second measurement of it.

Reading is the wait worth attributing this way: a parallel scan takes no row
locks, because the gate declines the scan when the read is a locking one, so
there is no lock wait to divide. The figures are undercounts wherever the
pre-fetcher did its job -- a page somebody else fetched costs its reader
nothing here -- so a low read time is not proof that nobody waited.

Two things had to be fixed before the numbers existed rather than merely being
hidden.

The page read time was being collected per worker all along and thrown away at
the merge: each worker resets its handler stats when it opens its tables and
snapshots them into tab_hstats before closing them, and quiesce_workers() added
those into the manager's handler without keeping the split. It now keeps it.

Engine time was always zero for a worker, for two reasons. It is folded into
the handler stats from the handler's Exec_time_tracker by close_thread_table(),
which a worker's tables never go through, so snapshot_table_stats() now does
that fold itself. Under that, the worker's handlers had no tracker at all to
fold: set_time_tracker() is called by the optimizer on the plan's tables, and a
worker's copies are opened after the plan exists, so nothing was timing them.
Each worker now owns one Exec_time_tracker per table, installed as the table is
opened.

Both arrays are omitted when the engine recorded no time, so a scan that never
touched the disk does not carry an array of zeroes; this is why the read-time
array appears in some plans and not others. Timings and a per-worker split are
both non-deterministic, so analyze-format.inc masks them, and the three .result
files that show a parallel scan are re-recorded.

This commit was prepared with Claude Code, which found that the read time was
already being collected and discarded, and traced the always-zero engine time
to the missing tracker on the worker's handlers.

Added, r_peak_to_average_ratio for worker engine time.
ea0fff7f4174...
ea0fff7f4174813c0214a73d2248f9df7ea2fa05
Sergei Golubchik
caching_sha2_password: add test for max password length

as enforced by RSA with 2048-bit key and RSA PKCS1 OAEP padding
e39d02fe0907...
e39d02fe0907ff0951e943a2494e39ea9751fdf9
Sergei Golubchik
MDEV-41520 Unescaped XID bytes in SESSION_TRACK_TRANSACTION_CHARACTERISTICS

* always print gtrid and bqual in hex
* don't try to be smart to print as few fields as possible
  - it was broken and could skip bqual, but still print formatID
* don't try to predict how many bytes it'll take. Print into a local
  buffer, then memcpy. Robust and costs just one memcpy.
e21185185d1f...
e21185185d1f557a587e339e190b7ebb16df7637
Oleksandr Byelkin
Merge branch 'bb-11.8-release' into bb-12.3-release
da13f8b70f0a...
da13f8b70f0a1d14d4a5dcd9d1e1a1234b14ee7e
Rex Johnston
MDEV-41251 Wrong result with split materialized query

A query like

  SELECT * FROM t0 WHERE t0.b IN
    (SELECT t.n2 FROM t1, (SELECT n1, n2 FROM t1 GROUP BY n1, n2) t
      WHERE t.n1 = t1.n1 AND t1.c1 = 'a' UNION SELECT -1)

returned only the row of t0 that was read first, when the derived table t
was split-materialized.

The IN->EXISTS transformation wraps t0.b in an Item_cache owned by the
Item_in_optimizer, and injects t.n2 = <cache>(t0.b) into the subquery. The
optimizer then pushes an equality on n2 into the derived table, so that
the table only has to be filled with the matching rows.
TABLE::add_splitting_info_for_key_field() built the right side of that
equality as a deep copy of <cache>(t0.b). A copy of a cache is a second
cache: it read t0.b for the first row of t0 and kept that value, because
only the Item_in_optimizer stores new values, and it stores them into the
original cache. Every later refill of the derived table was filtered on
the first row's value.

The right side is an expression of the outer select used inside the select
that specifies the derived table. Add a copy hierarchy for that purpose.
Item::new_select_copy_with_checks(thd, item_select) returns a copy for use
in a select nested in item_select, the select the item belongs to. In the
nested select the copy is an outer reference to item_select:

- An Item_field is copied and its depended_from set to item_select. This is
  what add_splitting_info_for_key_field() did before with
  set_fields_as_dependent_processor, except that depended_from now names
  the select the column belongs to instead of the nested select.
- An Item_direct_view_ref, a column of a merged view or derived table, is
  also copied like a column: the copy refers to a new_select_copy() of the
  item the view column stands for. A constant view column keeps its constant
  underlying item, which keeps a column of a constant NULL-complemented
  table constant in the pushed equality (the MDEV-30081 test in
  main.derived_cond_pushdown still shows ref "const" for its LATERAL DERIVED
  table). If the copy still depends on a table of item_select, which happens
  for a constant column of a view on the NULL-complemented side of an outer
  join, since used_tables() then names that table, the copy gets its
  depended_from set as well.
- Any other item that depends on tables of item_select is not copied. The
  copy is an Item_direct_ref to the original item, with depended_from set
  to item_select. So the nested select reads the cache, aggregate or
  subquery that its owner keeps up to date, instead of a copy of it that
  nobody refreshes. Item_default_value, Item_insert_value and
  Item_trigger_field are treated this way too, as they are not plain
  columns.
- An item that depends on no table of item_select is shared. This includes
  items whose deep_copy() returns nullptr, such as stored routine
  variables, which therefore no longer prevent splitting.

The copy is fixed, and only refers to tables outside the select it is used
in, which the wrapper asserts. add_splitting_info_for_key_field() now uses
it for the right side of the pushed equality, and no longer walks the copy
with set_fields_as_dependent_processor or calls update_used_tables() on
it.

A second test, taken from Dave Gosselin's MDEV-41251 commit, pushes a
comparison, dt.a <=> (t1.b = 1), and its row form. The deep copy of the
comparison was never fixed again. Its Arg_comparator still pointed at the
original, which sets null_value of the original, and a copied row
comparison had comparators set to NULL. The row of t1 with a NULL b was
missing from the result, and the row form crashed in
Arg_comparator::compare_row(). Now the comparison is not copied: the right
side refers to the original.

A third test pushes dt.a <=> v.one where v is a merged view with the
constant column one, on the NULL-complemented side of an outer join. It
covers the view column copy that depends on a table only through its
NULL-complemented table.

This commit was prepared with Claude Code (Opus 5), which designed and
wrote the new_select_copy() hierarchy, switched the split equality to it,
wrote the three-row test from the earlier rr analysis, added the
comparison test, confirmed in a build of the parent commit that both tests
fail without the change, and ran main and the SQL-layer suites in normal
and --ps-protocol mode. Claude Code (Fable 5.1) then reviewed the commit,
found and reproduced the NULL-complemented view column case, wrote its fix
and test, and reran the derived and SQL-layer tests.
c9c58dd85192...
c9c58dd85192ccfe3c924a7f7761b0ff5df485c9
Oleksandr Byelkin
Merge branch 'bb-13.1-release' into bb-13.2-release
c4477addd92f...
c4477addd92fe95f0a4058bc70f651d89fe046fe
Rex Johnston
MDEV-40012 PQ: refuse DEFAULT() in an expression a worker evaluates

DEFAULT(col) was evaluated against the worker's row. Item_default_value is
an Item_field whose field is a private copy made by make_default_field(),
reading the share's default_values but carrying the column's own table.
Its deep_copy() shares that field, and Pwt_field_rebinder, seeing the
manager's table, repointed it at the worker's real column. DEFAULT(col)
then read the current row's value, and for an expression default
calculate() called set_default() and wrote the default into the row --
wrong results in a release build, a marked_for_write assertion in a debug
one. It shows only when the expression also names the column some other
way; alone, DEFAULT(col) is table-independent and evaluated at optimize
time.

The gate now refuses any expression holding a DEFAULT(), found with
check_func_default_processor, which only Item_default_value answers.

Found by running the main suite with --parallel-worker-threads=4, where
type_temporal_innodb lost a "Truncated incorrect BOOLEAN value" warning
because the worker's WHERE evaluated to NULL. parallel_query_clone covers
a lost row, a gained row and the expression default that crashed, and
checks that none of them ran in the workers.

This commit was prepared with Claude Code (Opus 5.5), which traced the
wrong result from a main-suite sweep with workers enabled to the rebinder,
wrote the gate refusal and the test, confirmed the test fails without the
refusal, and ran the parallel_query tests in normal and --ps-protocol
modes.
c3e3a30f302a...
c3e3a30f302aaa2909a37f683b3d76e04d86818e
Sergei Golubchik
MDEV-41029 AddressSanitizer: use-after-poison - "install soname ha_rocksdb"

rocksdb tried to modify a read-only sysvar (ignore_db_dirs). It was
kind of fine on startup, even if hackish, but at INSTALL PLUGIN time
it meant violating the contract, read-only sysvar cannot be modified
at run-time.

let's ignore any ill-conceived attempts to modify ignore_db_dirs
at run-time. as a result, if RocksDB is installed with INSTALL PLUGIN
command, new #rocksdb directory will not be excluded from SHOW DATABASES
list until server restart.
c10e9b135b91...
c10e9b135b914778122199c7d4b95014e578a411
Rex Johnston
MDEV-40012 PQ: end a worker's parallel scan before close() frees m_prebuilt

A worker's handler was destroyed with its parallel scan still open.
exec() and scan_only() jump to their exit labels when
parallel_init_worker() fails, and the commonest failure is
HA_ERR_END_OF_FILE: no chunk was left for this worker, which happens all
the time on small tables. InnoDB has allocated m_pscan_worker by then, so
the scan was left for ~ha_innobase() -> parallel_scan_free(), which runs
after close() has freed m_prebuilt. parallel_scan_free() says it drops the
worker without end() for exactly that reason, but ~Parallel_scan_worker()
called end() anyway, and end() writes m_prebuilt->pscan_chunk_clamp: a
heap-use-after-free that ASAN reports on every such worker. In a build
without ASAN the write lands in freed memory, and it is the likely cause of
the intermittent SIGSEGV in free() inside pthread_create() when a later
worker reused a cached thread stack whose DTV had been overwritten.

Three changes close it. Both workers now end the scan at their exit label
instead of after the loop, so every way out ends it while the handler is
still open; ending a scan that was never begun is harmless.
ha_innobase::close() ends any worker or coordinator scan still open before
row_prebuilt_free(), so a path that misses it is ended properly rather than
left for the destructor; with the SQL layer put back as it was, this alone
keeps ASAN clean. And ~Parallel_scan_worker() no longer calls end(), so
parallel_scan_free(), now reached only for a handler destroyed without
close(), cannot write through m_prebuilt either. The close() hook and the
single call site are taken from a patch by a co-worker.

Two debug assertions state the rule the defect broke, so it no longer
takes ASAN to see it. ha_innobase::close() asserts that the SQL layer has
already ended a worker's scan, and parallel_scan_free() asserts that no
worker survives close(). The injection pwt_leave_scan_for_close makes the
SQL layer leave the scan of a worker that holds no cursor -- one handed no
chunk -- for close() to end, so the backstop is exercised on purpose and
the first assertion is told to allow it. pwt_end_worker_scan() holds the
injection and ends the scan at both exit labels.

parallel_query_no_chunk (debug only) runs a full scan and a range scan of a
one-page table with four workers, so three of them get no chunk, twice
each, through the normal and the scan-only path, with and without the
injection, checking the rows and that the workers ran every query. In a
plain debug build, removing the exit-label calls fails the first assertion
and removing the close() backstop fails the second.
~Parallel_scan_worker() no longer calling end() is defence for release
builds and is not reachable while the backstop is there.

This commit was prepared with Claude Code (Opus 5.5), which reproduced the
pthread_create() crash under ASAN and traced it, wrote the fix, folded in
the co-worker's close() hook and single call site, added the debug
assertions, the injection and the test, confirmed in a plain debug build
that the test fails with each of the two fixes removed, and ran the
parallel_query tests in normal and --ps-protocol modes and the main suite
with --parallel-worker-threads=4 under ASAN.
bf2aa7f6a7a0...
bf2aa7f6a7a06762d8e35fd6cea414274d1d8271
Thirunarayanan Balathandayuthapani
MDEV-32067 InnoDB linear read-ahead had better be logical

The traditional linear read-ahead, enabled by
innodb_read_ahead_threshold, only helps when consecutively
accessed pages have adjacent page numbers.

That is rarely true for B-tree leaf pages: after splits,
merges and page reuse, the logical order of leaves has
nothing to do with their physical page numbers. So for a
scan of scattered leaves the old read-ahead either did nothing
or read the wrong pages.

This replaces linear read-ahead of index pages with a logical
read-ahead that prefetches the actual leaf pages a scan is
going to visit, discovered from the B-tree during traversal.
After this change, the exact nonzero value of
innodb_read_ahead_threshold matters only for the read-ahead of undo
log pages.

buf_read_ahead_undo(): Renamed from buf_read_ahead_linear(). Now
invoked only for undo log pages, whose page numbers are
sequential. It is no longer called on BLOB pages or index pages.

buf_read_ahead_one(): Read ahead a single leaf page.

buf_read_ahead_pages(): Read ahead a set of known B-tree leaf pages.
Stops at the first page that is already resident or fails.

buf_read_ahead_random(): Split into a decision part(which extent
to read) and action part (issue the reads)

The pending read throttle now compares the
os_aio_pending_reads_approx() with buf_pool.curr_size() instead of
current pool bytes, which had made the throttle a no-op.

buf_read_ahead_pages() releases the tablespace reference
on a failed/corrupted page.

btr_ra_latch: A caller-owned buffer of leaf page numbers, which also
records the level-1 page and the child page number of the last node
pointer collected, so read-ahead can be resumed later.

btr_ra_collect(): Scan the node pointers of PAGE_LEVEL=1 page from
the given record in scan order, appending child page numbers

btr_cur_t::search_leaf(), btr_cur_t::open_leaf(): At PAGE_LEVEL=1,
after the descent has located the child to follow, harvest leaf page
numbers starting at that child in the scan direction. These are
exactly the leaves the cursor will visit, so prefetch is precise
regardless of physical page numbers.

btr_ra_resume_rec(): Locate the record from which read-ahead should
resume, by re-finding the last harvested child by value. It walks only
genuine records, so a concurrent page reorganization cannot cause
an invalid read

btr_pcur_move_to_next_page(): Prefetch the sibling after the
one being entered, with buf_read_ahead_one().

btr_pcur_move_backward_from_page(): Add a no-wait fast path that
latches the previous page if it is resident
(validating it against the current page), and prefetch
the one before it.

btr_copy_blob_prefix(): Simplified; no longer relies on
FIL_PAGE_PREV/NEXT, which are not initialized on BLOB pages.

records_in_range() reports the first and last leaf page of a
range, and the optimizer forwards that extent to the engine through
handler::advise_page_range().

multi_range_read_info_const() takes a page_range parameter;
the default MRR implementation aggregates the
per-range extents, and opt_range.cc carries the result through the
chosen QUICK_RANGE_SELECT, whose reset() calls advise_page_range()
before multi_range_read_init(). The extent only sizes and bounds
read-ahead; it never gates the pages the scan actually reads.
The LIMIT is converted in multi_range_read_info_const() to a
leaf count (m_ra_limit_pages).

mrr_readahead_from_scan_range() derives the read-ahead ceiling
from the advised extent, falling back to that LIMIT-based estimate.
The ceiling is capped by init_readahead_window() at 1/256 of
the buffer pool (but at least 4 pages), because every prefetched
page occupies a buffer pool frame as soon as its read is issued.

The window ramps up: a positioning read harvests at most 4 leaves
and resets the window to 4, and readahead_refill() doubles it
toward the ceiling. Single-row probes such as index_first()
for MIN()/MAX(), and every range of a multi-range read,
therefore prefetch only a few leaves while long scans reach
full depth.

search_with_readahead(): Position the cursor while collecting the leaf
pages, then call start_readahead(). It is used by
index_read() and by the first general_fetch() only when
read-ahead is armed, so the other paths carry no read-ahead locals.

start_readahead(): For a just-positioned scan, prefetch the collected
batch and set up the rolling cursor.

readahead_refill(): As the scan advances (general_fetch()),
resume the level-1 harvest under an index S-latch, chain across
level-1 siblings, and prefetch the next batch bounded by the
advised last leaf, or to the end of the index when the extent
is unknown. The reads are issued after the latches are released.

Read-ahead is started from index_read() (range and full index scans),
rnd_init()/general_fetch() (full table scans), and, at one-page depth,
from row_merge_read_clustered_index() (OPTIMIZE/ALTER rebuild).
general_fetch() sets active_handler_stats before issuing
read-ahead so prefetched pages are attributed to the query.

dict_stats_analyze_index_for_n_prefix(): Choose the records to dive
below first, prefetch their child pages in one batch with
buf_read_ahead_pages(), and then dive below the same records.

Undo/system tablespace shrinking holds fil_space_t::latch exclusively
(x_lock_space()), and buf_read_ahead_undo() holds it shared,
via the new fil_space_t::s_lock_try() that never waits,
while it checks the size and issues the reads.
If the latch is not available, the read-ahead is skipped.

trx_undo_get_prev_rec(), trx_undo_get_prev_rec_from_prev_page():
take the trx_undo_t object instead of a long parameter list and
always latch shared.
b764a55d09e8...
b764a55d09e8dba9722f3160db266ad754edf797
Oleksandr Byelkin
Merge branch 'bb-12.3-release' into bb-13.1-release
b25a4f31f1da...
b25a4f31f1daae19cd732a8ec76ac167a38fccb0
Rex Johnston
PQ: divide a multiple-range scan by the work it holds, not a range at a time

Each range of a multiple-range scan divided itself: it compared its own chunk
count against the worker count, found itself short, tagged what it had for
re-splitting, and the splits that followed were per range as well. Ten ranges
did that ten times over and the queue filled with chunks the scan never needed,
without dividing the work any more evenly than a single range does.

So for a scan of more than one range the work is measured first and divided
afterwards. Measuring means walking one level below the sub-trees a level-0
walk finds, which gives a boundary per page: a page is the unit the measure
needs, and the sub-trees are not -- a range narrower than one of those yields a
single boundary whether it holds ten rows or a hundred thousand, which is why
the two rules built on those counts both failed. With every range measured the
same way, the chunks the scan should have -- twice the worker count -- are
shared out in proportion to pages, and each range coalesces its own boundaries
down to its share. Nothing is tagged: the division was made knowing the sizes,
so there is nothing left for a worker to correct at run time.

A scan of one range keeps the division it had. There is only one thing to
divide, it already gets the whole worker count, and leaving it alone keeps this
change off the common path.

Measured with r_chunks in ANALYZE FORMAT=JSON. Ten ranges whose sizes go up by
the golden ratio -- 253, 410, 663, 1073, 1736, 2809, 4545, 7355, 11900, 19255
rows -- against a million-row table. Sizes in that progression because no two
of them are a whole multiple of each other, so nothing can come out balanced by
accident:

    workers      chunks          pages          rows on the busiest worker
              before after    before after        before after
      4        97    18        854  582        14044  14893
      8        169    25      1142  610          7686  8116
      12        216    34      1330  646          5856  5518
      16        261    41      1510  674          4880  4392

Five to six times fewer chunks, a third to a half fewer pages, and the spread
across workers no worse -- better from eight workers up, where the busiest
worker now has less to do than it did. The same ten ranges at equal sizes go
from 103, 223, 432 and 432 chunks to 10, 20, 30 and 50; one large range beside
nine tiny ones goes from 28, 36, 44 and 54 to 18, 26, 34 and 44, with the
per-worker spread between 1.01 and 3.00 throughout. A single range is
unchanged, which is the point of leaving it alone.

Not yet measured, and the reason to: the deeper walk reads the pages one level
down for every range, before the scan starts, where those reads used to happen
inside a re-split and only for the chunks that were tagged. For these range
sets it pays for itself several times over. For a range set much larger than
these it might not, and that is a cost question rather than a correctness one.

This commit was prepared with Claude Code (Opus 5), which took the measurements
above, and wrote and discarded two earlier rules -- one counting chunks, one
counting whole sub-trees -- before finding that neither count measures work.
aaf5d46fec07...
aaf5d46fec078c2e349670729c914b4b9cafa7b9
Sergei Golubchik
MDEV-41123 bool show_role_grants(THD *, const char *, ACL_USER_BASE *, char *, size_t): Assertion `pair' failed

GRANT to a role that exists in memory but has no row in
mysql.global_priv took the "new role" path and called acl_insert_role(),
adding a second ACL_ROLE with the same name to acl_roles. DROP ROLE then
removed one of them, while user role_grants kept pointing to the other,
whose grant pair was already gone.

Reject the mismatch between the table and memory with
ER_PASSWORD_NO_MATCH, like replace_user_table() already does for users.

Also, remove repeatable role hash lookups, one is enough.

Assisted-by: Claude:claude-opus-5-5
a9db325e8267...
a9db325e826708d4a3e77556c37cab11637db7f6
Rex Johnston
MDEV-40012 PQ: a worker must not end its container's scan after handing it over

A pre-aggregating worker accumulates into its own container, a temporary
table, and hands that container to the manager with sink->flush(). Just
before the hand-off, execute_and_handoff() closes the container's index,
because the manager starts its own scan of the container immediately after
the hand-off.

The tidy-up at exec_exit then closed the container a second time. Its
comment said this was only for the paths that reach exec_exit without a
hand-off, but the code ran on every path, including the path with a
successful hand-off. By that point the container may already belong to the
manager: claim_next_result() has seen it marked as done, and
pwt_tmp_table_source::next_row() has rebound it to the manager's thread and
called ha_rnd_init() on it. The worker's ha_index_or_rnd_end() therefore
ends the manager's scan. The manager's next ha_rnd_next() then fails
"inited == RND" in a debug build, and in a release build it reads from a
table with a closed scan.

The window between the flush and the second close is only a few
instructions wide, so the failure is rare: main.ps_3innodb hit it once in a
main-suite run with --parallel-worker-threads=4, on
SELECT ... UNION DISTINCT SELECT sum(a) ... GROUP BY b. The second close now
runs only on the paths that skipped the first close, as recorded by a flag
set next to the first one.

parallel_query_handoff makes the race certain. With pwt_handoff_sync set, a
worker that has handed over its container waits until the manager reaches
the new sync point pwt_manager_scanning_container, and the manager keeps its
scan open until the worker has finished its tidy-up. The test uses one
worker, so there is a single container and no risk of crossed signals. It
checks the rows and that the query ran in the worker. Without the fix it
fails the inited == RND assertion on every run.

This commit was prepared with Claude Code (Opus 5.5), which found the crash
in a main-suite sweep with workers enabled, traced it from the core dump to
the second close, wrote the fix, the sync points and the test, and confirmed
that the test fails without the fix in five runs out of five and passes with
it in normal and --ps-protocol modes.
a5fbf0b9be8b...
a5fbf0b9be8b7af9d91d66041bb53604fb359179
Oleg Smirnov
Rename Parallel_coordinator, some other refactorings and clean-up
a565972a30a0...
a565972a30a04e6a123006d125febcc0a5e9f6cb
Sergei Petrunia
Make declaration order match handler.h.
a1ce78edad4e...
a1ce78edad4e366b3f3c26d7ac1cf9b3a33bc395
Rex Johnston
MDEV-40012 PQ: merge AVG, MIN and MAX partials after the manager's table spills

The manager merges a worker's partial for a group through the plan's own
terminal. direct_add() hands the partial to each aggregate, and
end_update() either opens the group with reset_field() or folds the
partial into it with update_field(). Once the aggregation table spills
from HEAP to disk, the terminal becomes end_unique_update(), which calls
reset_field() for every row, before the write shows whether the group is
new. For a group already in the table it then restores the stored row
and calls update_field() with the same partial.

reset_field() consumes the partial, so update_field() has to find it
again. Item_sum_sum and Item_sum_count keep their direct values for that
case and mark it with direct_reseted_field. Two aggregates lacked that:

- Item_sum_avg::reset_field() zeroed direct_count, so update_field()
  added the partial's sum with a count of zero, and the average of such a
  group came out too high.
- Item_sum_min_max::reset_field() cleared direct_added and had no
  direct_reseted_field, so update_field() compared the stored value with
  args[0], the manager's own argument over a base-table row the manager
  never fills, instead of with the partial.

Item_sum_avg::reset_field() now leaves direct_count for update_field(),
which already clears it. Item_sum_min_max gains direct_reseted_field, set
by reset_field() and read and cleared by update_field(). A directly added
item also bypasses the FILTER test in update_field(), as in
Item_sum_sum, since the partial already reflects the filter. A serial
plan never sets direct_added, so it never reaches the new branches.

The grouped case already in main.parallel_query_spill has one row per
group, so each group gets a single partial and the branch for a group
already in the table never ran. The new case groups 20000 rows into 3000
groups, so each worker sends a partial for nearly every group, and the
64K heap limit makes the manager's table spill. Without either of the two
changes, its parallel answer differs from the serial one.

This commit was prepared with Claude Code (Opus 5.5), which found the
wrong averages while letting the workers pre-aggregate on a wide GROUP BY
column, traced them and the wrong MIN and MAX values to the reset_field()
and update_field() pair under end_unique_update(), and showed with the new
test that the AVG change and the MIN/MAX change each fix a failure of
their own.
97d71fa17bd7...
97d71fa17bd77b445146a8706c1a6f63b7966d37
Sergei Golubchik
MDEV-41655 caching_sha2_password does not limit the password length

let's tighten the limits to be closer to actual OpenSSL limits.

Encrypting with RSA, 2048-bit key and PKCS1 OAEP padding *always*
produces the cyphertext of 256 bytes. And to fit in this, the plaintext
cannot be longer than 214 bytes.

Let's check both.

Note that decryption buffer, strangely enough, must be at least 257 bytes,
even though it'll never hold more than 214.
92fea8a82470...
92fea8a82470516e8c570ab039ce8a52d4516eac
Oleg Smirnov
Refactor for better object names and more accurate comments
8a2f93b18c39...
8a2f93b18c397c6ba05ad0a9b8fb0695b822b08e
Oleg Smirnov
InnoDB: fix unlatched right-move in the parallel scan partitioner
8950556ab44e...
8950556ab44ed1da86f09d33ac28f0f1e76edf27
Sergei Golubchik
MDEV-41491 parsec ASAN crash on manually modified mysql.global_priv table

the server didn't invoke auth plugin's preprocess_hash() for empty
passwords, thus it didn't check whether empty passwords were acceptable
and plugin's auth() method reliance on the binary hash being correctly
preprocessed was broken.
85d095e004e4...
85d095e004e482631d262befddfd7eb4811a989b
Oleg Smirnov
Extract parallel logic from ha_innobase to separate files
8475c9c17739...
8475c9c177397e03e2eee8de6fb06209f95b6533
Rex Johnston
PQ: the workers pre-aggregate a query that has no GROUP BY

SELECT COUNT(*), SUM(b) FROM t ran serially. The workers could pre-aggregate a
GROUP BY, but aggregation over the whole table was refused, because the shape
the grouped path merges through is not there: with no group the server
materialises nothing, and aggregates in end_send_group() straight off the last
join tab -- no aggregation table, no group key, nothing for a partial to be
merged into.

It turns out neither end needs new machinery, and this commit is mostly the
finding of that.

The manager already merges these. end_send_group() reaches Item_sum::add(), and
add() honours a value left by direct_add() for every aggregate the gate accepts
-- COUNT adds a count instead of incrementing, AVG takes a partial sum together
with the number of rows behind it. So the drain primes the aggregates and hands
the row to the plan's own terminal here exactly as it does through end_update()
for a GROUP BY, and the drain needed no case of its own.

The workers accumulate into the grouped machinery with a single group in it.
The layout gives this shape a group of one constant column, and from there it
is the grouped path: a worker keeps one partial in a keyed container, merges
into it through end_update(), and ships one row. Beyond costing a column per
shipped row -- of which there is one per worker -- that is also what keeps
Item_sum_avg::create_tmp_field() packing the count beside the sum, which it does
only for a container that has a group. An average built any other way would have
arrived without its count.

So what is actually new is the gate. The aggregate tests -- the five that can be
merged, no DISTINCT, one argument, an argument a worker can evaluate -- are the
same for both shapes and move out to pwt_aggregates_can_be_merged(). What is
left is the shape question, and which shape to ask about is the query's own
grouping rather than the plan's: a query that names a GROUP BY stays the grouped
question whatever the optimizer then did with it, so a plan with no group key to
merge through is a refusal of that query and not an invitation to read it as
aggregating over everything, which would answer something else. Only a query
that never grouped reaches the second question.

Refused there: a select-list item reading a column outside an aggregate. Its
value is whichever row the terminal happened to hold, and a worker holds a
different one.

An empty result needed nothing. No rows means no groups means no partials
shipped, and end_send_group() still sends its one row at end of records:
COUNT(*) is 0, SUM is NULL, one row and not one per worker. The new test checks
each of those rather than assuming them.

main.parallel_query_aggregate_whole is that test: serial against parallel for
COUNT, COUNT(col), SUM, MIN, MAX and AVG over integer, decimal and double
columns; the empty case; that exactly one row is sent; and that a DISTINCT
aggregate, STD, GROUP_CONCAT and a bare column outside an aggregate are still
refused and run serially. main.parallel_query_aggregate gains the fall-back
case, where the container turns out to have no group index: the constant group
reaches the same fall-back, ships plain rows, and the manager aggregates them a
row at a time.

Known and accepted: adding doubles is not associative, so SUM and AVG over DOUBLE
or FLOAT can differ from the serial answer, and between runs, in the last
significant bit -- which worker summed which chunk is scheduling. Decimal and
integer aggregates are exact and unaffected. The test rounds well clear of the
last bit rather than leaving floats uncovered.

Under a forced-worker sweep of the main suite (--parallel-worker-threads=4)
three more tests differ than before, all of them EXPLAIN text where a query of
this shape now says ALL_parallel or range_parallel: main.costs,
main.func_group_innodb and main.innodb_full_scan. No value changes in any of
them.

This commit was prepared with Claude Code (Opus 5), which established that
Item_sum::add() already honours a directly added partial -- the fact the whole
change rests on -- and found that a container without a group would have
dropped an average's count.
83f072649af5...
83f072649af5c3fab0852f72f9ac3494e03c496b
Sergei Golubchik
MDEV-41655 caching_sha2_password does not limit the password length

let's tighten the limits to be closer to actual OpenSSL limits.

Encrypting with RSA, 2048-bit key and PKCS1 OAEP padding *always*
produces the ciphertext of 256 bytes. And to fit in this, the plaintext
cannot be longer than 214 bytes. Assuming the user can generate her
own keys, 512 bytes would be enough for 4096-bit key. Larger, 8192-bit
keys don't work anyway (don't fit in public_key[] buffer).
80225ba763da...
80225ba763da3cf3b6e61ca0fff1c060c7970878
Sergei Golubchik
MDEV-41480 Failed ADD PARTITION with a #mysql50# partition name removes the table

mysql_write_frm(WFRM_WRITE_SHADOW) declared a local `error` shadowing
the function-level one, so a failed ha_create_partitioning_metadata()
returned success and ALTER continued to install the shadow frm.

Assisted-by: Claude:claude-5.5-opus
77e7b9bdf8f6...
77e7b9bdf8f6e8f9ac876da05bd6a731704701b3
Sergei Golubchik
MDEV-41504 COM_FIELD_LIST Integer Underflow Leading to Heap Over-read

reject COM_FIELD_LIST packets without a table name / wildcard separator.
Otherwise the wildcard started one byte past the packet end and its
length wrapped to UINT_MAX.

Assisted-by: Claude:claude-opus-5-5
755b6490631e...
755b6490631ec4ba569020c506ae5ac1a4873dd9
Rex Johnston
PQ: a pre-aggregating worker hands over its grouping table

A worker that pre-aggregates kept two temporary tables of the same shape: the
grouping table its aggregates accumulated into, and the shipping container the
manager reads. At end of records flush_groups() walked the first and wrote every
row into the second -- a memcpy and a keyed insert per group -- so that the
manager could read rows it already had, in a table it could already have been
given. The tables hold the same rows because a worker produces one row per group
and ships one row per group; there was never anything for the second table to do
but hold a copy.

So there is one table now. setup_worker_preagg() builds the shipping container
itself, from the worker's own item list, and the aggregates accumulate straight
into it; flush_groups() is gone, and so is the per-group copy and the second
table's memory. What the manager reads is the table end_update() wrote.

The batch transport is disabled, which is what makes that possible to say
plainly. It carries record images through a buffer rather than handing a table
over, so everything that goes through it is copied twice, and a worker with a
finished table still has to stream it. pwt_create_transport() now builds the
temporary-table transport unconditionally and the debug flag that selected the
other one is gone. The classes are kept, compiled out, in case the streaming
shape is wanted again.

Two things the copy had been hiding had to be fixed before the table could be
handed over.

A rebuild on disk has to be the cross-thread kind. A container is instantiated
cross-thread, because the manager reads and frees what the worker wrote, and
the transport's own spill path passes that through. But a pre-aggregating
worker's table fills while end_update() is writing it, and end_update() rebuilt
it with cross_thread=false: Aria then ties the new handle's allocations to the
worker's thread, and the manager frees them after that thread is gone. The
decision now travels on TMP_TABLE_PARAM, set where the container is created and
read where the rebuild happens. Without it the new test fails with the
allocation crossing threads, named:

    16416 bytes freed by T@3, allocated by T@71 at maria/ma_bitmap.c:242, ...
    sql/sql_select.cc:23926, sql/sql_select.cc:26598

And the memory that rebuild allocates has to change hands. The transport
already moves the charge from the worker's THD to the manager's for a spill it
performs itself, but a spill inside end_update() never passes through the
transport. pwt_end_send() now brackets put_record() and hands anything the call
left outstanding to the sink: working memory a call allocates and frees nets to
nothing there, and the rebuild is what does not.

main.parallel_query_spill covers the new path: twenty thousand groups against a
64K heap, asserting the query ran in the workers, that a row per group arrived,
that the container really was rebuilt on disk, and that the answer matches the
serial one group for group. main.parallel_query_transport loses the halves that
exercised the batch transport, and with them its dependency on a debug build.

Measured against this tree with the same forced-worker sweep of the main suite
(--parallel-worker-threads=4): the same fifty tests fail before and after, the
same fifty by name.

This commit was prepared with Claude Code (Opus 5), which found that the two
tables held the same rows, and identified both the non-cross-thread rebuild and
the unaccounted memory before either could reach a test.

The we found a bug, we need to close the container's index before handing it
to the manager

A pre-aggregating worker leaves its container's index open: AGGR_OP::put_record()
opens it on its first call and nothing since has closed it. When the copy into a
separate shipping container went away, closing it moved to the worker's tidy-up
at the end of execute_and_handoff() -- which is after sink->flush(), and flush()
is what tells the transport this producer is finished.

The manager may claim the container the moment it is marked done. So a manager
that got there before the worker reached its tidy-up called ha_rnd_init() on a
table whose index was still open, and that asserts:

    mariadbd: sql/handler.h:3770: int handler::ha_rnd_init(bool):
    Assertion `inited==NONE || (inited==RND && scan)' failed.
      handler::ha_rnd_init
      pwt_tmp_table_source::next_row
      pwt_manager::drain_and_send
    Query: SELECT col1, COUNT(col2) FROM t1 GROUP BY col1 ORDER BY NULL

Which thread wins is scheduling, so this showed up as main.parallel_query_aggregate
failing about one run in eight with the whole parallel set running at
--parallel=18, and passing every time on its own.

Closing the index is moved to before the flush, where the hand-off is. The
tidy-up still closes it, for the paths that reach the end without handing
anything over; ha_index_or_rnd_end() on a handler that is already closed does
nothing.

pwt_tmp_table_sink::flush() now asserts what the ordering is for: that nothing
this producer had open on the table is still open when the container becomes the
consumer's to read. That turns a race the manager notices only when it wins into
a condition the producer checks every time -- with the close put back after the
flush, main.parallel_query_aggregate fails on the assertion on every run rather
than on one in eight.

This commit was prepared with Claude Code (Opus 5). The crash was found by
running the parallel test set repeatedly under load rather than once; the
producer-side assertion was added so the next mistake of this shape does not
need luck to be seen.
74795f94d1a5...
74795f94d1a513e8246cab4c7f1143e4c1fa6423
Rex Johnston
PQ: separate finding a range's boundaries from cutting the scan up

add_scan() walked the tree for one range and turned that range's boundaries
into chunks in the same breath. Both halves are per range, and only the first
one should be: whether a chunk may be divided again is a question about the
scan, and a scan may hold many ranges of very different sizes.

So add_scan() now partitions and keeps the boundaries on its Scan_ctx, and the
caller adds every range it has before calling create_chunks() once. Behaviour
is unchanged -- each range still decides for itself, in
Scan_ctx::create_chunks(), exactly as it did before -- and this is the shape
that lets that decision move.

Measured with r_chunks in ANALYZE FORMAT=JSON, the same 50,000 rows of a
million-row table, reached three ways: as one range, as ten equal ranges, and
as one large range with nine tiny ones beside it.

    workers            one range  ten equal  one large + nine tiny
      4    chunks          10        103            28
            pages          458        899          511
      12    chunks          26        432            44
            pages          522      2215          575

Ten ranges cost sixteen times the chunks and four times the pages of one range
holding the same rows. The cost follows the number of ranges big enough to be
partitioned, not how unevenly the rows sit across them, which is why the
lopsided third shape is nearly as cheap as the first. Per-worker row counts are
within the usual 3:1 tail in all three, so the extra chunks buy nothing -- the
pull queue was already balancing the work.

Why the decision has not moved with the call:

It needs a measure of how much work a range holds, and the boundaries do not
carry one. A level-0 walk cuts at the children of the root, so a range narrower
than one of those -- which at this fan-out is any range under about a tenth of
the table, and so most range scans -- yields exactly one boundary whether it
covers ten rows or a hundred thousand. Two rules were built on those counts and
both were worse than what they replaced. Deciding from the total number of
chunks left the third shape's large range unsplit on a single worker while the
other three had ten rows each. Weighting by whole sub-trees, on the reasoning
that a range clamps only the first and last it touches, measured zero nearly
everywhere and serialised all three shapes.

What that rules out is worth keeping: any rule counting boundaries is deciding
without knowing the sizes. The existing per-range rule is in that class too --
it survives because its failure is to over-split, which costs pages rather than
answers. The number that does measure work is the count of sub-ranges a chunk
yields when it is split, which is already computed inside Chunk::split(), where
the piece count is decided.

This commit was prepared with Claude Code (Opus 5), which took the measurements
above, and wrote and discarded both of the rules described.
5bc5403473f3...
5bc5403473f3f96e788efa06004c7fa041872b68
Rex Johnston
PQ:  Code tidy ups and fixups of tidyups.

parallel_query_aggregate.test: fix statistics
impersonalize language in can_parallel_scan_jointab_access()
remove dead code (trace_parallel_scan_options)
add an assert to ensure we do not use Item_sum_avg without two packed fields
minor comment fixups
remove enum element NESTED_LOOP_DECLINED and use a local bool instead
fix silly code in pwt_row_layout::build_aggregates (throwing away a pointer
  then searching for it again later); 'one' starts as nullptr, because GCC
  cannot see that whole_table is unchanged between the two blocks and warns
  -Wmaybe-uninitialized in an ASAN build
remove pwt_batch* worker->manager streaming functions
tidy up Parallel_scan_partitioner::Scan_ctx::create_chunks to use
  create_chunks_unsplit

The 'one= nullptr' initialisation was added with Claude Code (Opus 5.5),
which traced the -Wmaybe-uninitialized warning to this change and folded
the fix in here.
48a08ef5338c...
48a08ef5338cd81332f6fc68231e7e11c8ef154c
Oleksandr Byelkin
fix format
44e7c4b4e11d...
44e7c4b4e11d759a5b7369334ae0797a7237857e
Sergei Golubchik
compilation failure

storage/innobase/row/row0import.cc:678:17: error: conversion from 'uint32_t' {aka 'unsigned int'} to 'uint16_t' {aka 'short unsigned int'} may change value [-Werror=conversion]
  678 |                ? 0 : FSP_FLAGS_GET_ZIP_SSIZE(m_space_flags);
3fbe59b5a181...
3fbe59b5a1818ac50c345fdaf3f6df52f1900fb5
Sergei Golubchik
MDEV-39822 MHNSW index returns wrong results for M >= 128

* number of neighbors is stored in a byte
* nodes have 2*M neighbors on the layer 0
=> M cannot be larger than 127
29bf0e74c7d2...
29bf0e74c7d2cb3b3c2a001a3871420fb1cbe922
Oleg Smirnov
Incorporate a pointer to the coordinator into worker context
23908411f195...
23908411f195f3ca76d564bc6d2f7a2bf2cdde52
Rex Johnston
MDEV-41632 UPPER(key_col) IN (constant subquery) does not use the index

Item_func_in::varchar_upper_cmp_transformer() rewrites
UPPER(key_col) IN (const, ...) into key_col IN (const, ...).

It needs a copy of the IN predicate to put the new first argument into,
it used deep_copy_with_checks(). A deep copy copies every node of the tree and
refuses as a whole if any node cannot be copied, so an IN list holding an
uncorrelated scalar subquery, which is constant but has no copy, silently
lost the rewrite: UPPER(a) IN ((SELECT MAX(c) FROM t2),'A-2') read the whole
table instead of using a range on the index on a. The rows returned were
correct; the symptom is the plan. After copying, the rewrite walked the copy
with cleanup_excluding_const_fields_processor and re-fixed it, which would
also have cleaned up a subquery had one been copied.

A copy for use in the same SELECT_LEX does not need to copy what can be
shared there. We add a separate copy hierarchy for that purpose.

Item::same_select_copy_with_checks() calls the protected virtual
Item::same_select_copy(), and cleans up every node it actually copied, so
that the caller only has to call fix_fields() on the result. fix_fields()
re-fixes the copied nodes and skips the shared ones, which are still fixed.
It does not use deep_copy() at any level; the only other copy function it
calls is shallow_copy() on the node being copied. It returns nullptr on out
of memory, and when a copied node's shallow_copy() returns nullptr; every
Item_func subclass whose shallow_copy() returns nullptr therefore overrides
same_select_copy() to share the item, and a future class in that position
must do the same or callers silently lose their rewrite.

Item default is to share the item.

Item_func, Item_cond and Item_row copy their node and recurse into their
arguments.

Item_func copies an argument that occurs more than once in args[] only once.
Item_bool_rowready_func2, Item_func_case_simple and Item_func_in detach the
comparators and value array they would otherwise share with the original,
as their deep_copy() overrides do. Item_func_interval points its row member
at the copied args[0], which it aliases.

Fields, references, constants, parameters, caches, subqueries, aggregate and
window functions are shared because they don't derive from Item_func.

Some Item_func subclasses cannot safely be copied, so they override
same_select_copy() to share the original instead:

- Functions that own memory or handles, which the copy would free a
  second time in cleanup() or its destructor: the regex functions, UDFs,
  stored functions, the JSON functions that build hashes, VEC_FromText,
  and the geometry functions with Gcalc state.
- Functions that their SELECT_LEX keeps a pointer to, so a copy would not
  be the item the SELECT_LEX uses: MATCH, ROWNUM and Item_equal.
- Functions whose copy would behave differently: RAND would restart its
  random sequence, and SET @var would assign the variable twice.
- TO_NUMBER, which refuses to be copied at all.
- Functions whose arguments point at each other: NULLIF uses args[0] a
  second time as args[2] and hides it from arg_count until cleanup(), and
  Item_in_optimizer's cache belongs to its subquery.

The rewrite now uses same_select_copy_with_checks() and no longer walks the
copy with a cleanup processor.

TABLE::add_splitting_info_for_key_field() uses it too, for the left side of
the equality it pushes into a split-materialized derived table: the item of
the derived table's select list, used in the WHERE condition of the same
select. Its real item is always a column, so the copy is the select list
item itself, where the deep copy made a second Item_field for it. Sharing
one item between the select list and the WHERE clause of a select is
intended here: state that condition processing sets on the item, such as
Item_field::item_equal, describes the same column in the same select. The
equality is fixed when it is injected.

This commit was prepared with Claude Code (Opus 5), which found the
subquery case that lost the rewrite and confirmed that the new test fails
without the change, designed and wrote the same_select_copy() hierarchy
and the list of shared Item_func classes, switched the casefold rewrite to
it, and ran main and the SQL-layer suites in normal and --ps-protocol mode.
Claude Code (Fable 5.1) then reviewed the commit, corrected the commit
message, kept Item_func_case_simple::shallow_copy() protected, added the
Item_func_interval override and the nullptr note, and re-ran the tests.
0e4a97125280...
0e4a971252801fa72d493fad55e7607a21978b8e
Rex Johnston
MDEV-40012 PQ: let the workers pre-aggregate on a GROUP BY column too wide for an on-disk key

The gate refused a grouped plan with a GROUP BY column too wide to be a
key part, or with a key too long in total, on the grounds that
create_tmp_table() would key the aggregation table by a unique
constraint, a hash of the row with nothing to look a group up in. Since
MDEV-38975 that holds only for a table that starts on disk: a HEAP table
indexes a wide or blob key part, and Create_tmp_table::start() skips the
fallback when it expects HEAP. Such a plan writes through end_update(),
so pwt_plan_group_key() finds its key, and the gate then refused it on
the per-column prediction alone.

Both predictions are now out of the gate. A plan whose aggregation table
starts on disk still has a unique constraint and no end_update()
terminal, and the gate still refuses it, with the reason it already gave
for that shape. Otherwise the containers are built the way the plan's own
table was, and pwt_row_layout::build() checks the finished container for
a group index, with plain rows shipped in place of partials if it has
none.

A worker container with a wide key fills sooner, and a worker that
rebuilds it on disk inside end_update() exposed a fault in how memory
moves between threads. The container's own MEM_ROOT was thread-specific,
so whatever a worker allocated in it was charged to the worker and freed
by the manager. pwt_end_send() moved the charge by bracketing
put_record() and handing the whole change in the worker's
local_memory_used to the manager. But that change also holds memory that
stays with the worker: ha_maria::enable_indexes() allocates its HA_CHECK
from thd->mem_root, the worker's own root, which ~THD frees later on the
worker's thread. With that block counted against the manager, the
worker's counter went negative in ~THD:

  Assertion `(longlong) thd->status_var.local_memory_used >= 0 ||
  !debug_assert_on_not_freed_memory' failed.
    THD::~THD
    destroy_background_thd
    pwt_thread::init_and_run_thread_func

A table one thread writes and another frees is not thread-specific
memory, which is what HA_OPEN_GLOBAL_TMP_TABLE already says to its
engines. Create_tmp_table::start() now gives a cross-thread table a root
without MY_THREAD_SPECIFIC, and with nothing left to hand over, the
hand-over code is removed: the bracket in pwt_end_send(), the same
accounting in pwt_tmp_table_sink::emit_row() and cleanup(), and
account_spilled_memory().

Tests:
- main.parallel_query_why now asks for the on-disk shape
  (tmp_memory_table_size=0), which keeps the unique-constraint reason. At
  the default size the same query runs in the workers.
- main.parallel_query_aggregate shows a VARCHAR(4000) GROUP BY that stays
  in HEAP and pre-aggregates in the workers, with fewer than 1000
  tmp-table writes for 20000 rows and the serial answer. Its note on the
  no-group-index fallback no longer claims a gate refusal for wide keys.
- main.parallel_query_spill shows the same kind of key at a 16M heap
  limit, where the workers rebuild their containers on disk part way
  through. Without the memory change it fails on the assertion above.

A comment that still named accumulate_group(), long since replaced by
end_update(), is corrected too.

This commit was prepared with Claude Code (Opus 5.5), which traced the
changed refusal reason to MDEV-38975, removed the two predictions, found
the negative memory counter with the new spill case and located the
HA_CHECK allocation with an rr recording, moved the container's root off
thread-specific memory, and showed by reverting the memory change on its
own that the new spill case fails on the assertion without it.
BuildBot (0.8.9) working for the MariaDB project.
Page built: Sun 01 Nov 2026 19:38:33 (UTC)