Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
fd5c55c6c3a1...
fd5c55c6c3a14f992b9a829210dee22eaede410a
Rex Johnston
MDEV-39492 Parallel Query: a re-split chunk lost its first sub-tree

Parallel_reader::Ctx::split() divides one chunk into smaller ones by calling
Scan_ctx::partition(), which descends from the index root again bounded by that
chunk's start key. It is the first caller in the engine to pass a start bound at
all: while the SQL layer asked only for whole-table scans, every partition began
before the first record and the bound was always null.

Scan_ctx::create_ranges() searched for that bound with PAGE_CUR_GE at every
level. Above the leaves a record is a node pointer keyed by the first key of the
child it points at, so the child that contains a start key is the last one whose
key is not greater than it. GE lands on the child after that one, so the
sub-tree holding the start key was never descended into, no range was created
for it, and every row in it was left out of the scan with nothing reported
anywhere. Scan_ctx::search() already makes the distinction -- PAGE_CUR_LE above
the leaves, PAGE_CUR_GE at them -- and the comment at the top of the file
describes it. create_ranges() now makes it too.

A chunk is only re-split once the tree is at least three levels deep, which no
existing test reached: they are all a few thousand narrow rows, and a table of
INTs needs some 700000 of them for a third level. The new test gets there in
5000 rows by giving the table a wide PRIMARY KEY, which shrinks the node pointer
fanout. Without the fix it answers 2591 of those 5000 rows in the workers and
5000 serially.

main.parallel_query_worker_count did have a table deep enough to be re-split,
t4, and its check that every row is read exactly once was written as a bare
aggregate -- a shape the parallel gate refuses -- so it ran serially and compared
the serial answer with itself. That scan was in fact returning 3492 of t4's 6000
rows. The check now reduces the rows outside a non-merged derived table, so what
it measures is a scan the workers ran, and it fails without this fix. t4's chunk
count per query rises from 10 to 18, which is the sub-trees a re-split no longer
skips.

This commit was prepared with Claude Code: the row loss turned up while it was
benchmarking an unrelated change, and it bisected the cause to the re-split path
by disabling re-split, identified the GE/LE descent, wrote the fix, found the
wide-PRIMARY-KEY shape that reproduces it in a table small enough for a test,
noticed that the existing completeness check was hollow, and verified that both
tests fail with the fix reverted.
fa555253c075...
fa555253c07578e922f8934fce3304118c116249
Yuchen Pei
MDEV-40467 [to-squash] Improve Y2038/2106 problem error message

When nearing the end times TIMESTAMP_MAX_VALUE, auto-partition
creation may encounter overflow when calculating new partition range.
This patch improves the message of such an overflow
f474a1addfe2...
f474a1addfe224aea76e66b88bbac942d038ebf9
Rex Johnston
MDEV-39492 Parallel Query: run LIMIT and OFFSET in the workers

The gate refused a query with LIMIT or OFFSET. It no longer needs to: since the
manager's loop ends in end_send(), the limit is applied by the server, on the
manager's single thread, and select_result_sink::send_data_with_check() applies
the offset in the same place. Reaching the limit returns NESTED_LOOP_QUERY_LIMIT,
which manager_collect_and_send() turns into a stop for the producers -- the flag
the batch handoff protocol has carried since it was written and never used. So the
number of rows delivered is exact and the scan stops early rather than draining.

Which rows arrive is another matter, and that is the reason this was left for a
decision rather than committed with the refactor. A parallel scan delivers
whichever rows the workers finish first, not the ones a serial scan would reach.
For a query with no ORDER BY there is no order for a prefix to be taken of, so
both are correct answers to the question asked; the ordered case is refused by
the gate and runs serially, unchanged. Depending on the rows an unordered LIMIT
happens to return is an antipattern, and not one this server is going to keep
supporting on the strength of a test that encodes it.

In the event nothing in the suite encoded it. The main suite with
parallel_worker_threads forced on is at the same eighteen failures as before, all
EXPLAIN and cost output, and normal runs are untouched because
parallel_worker_threads is 0 by default. The reason is worth recording: nearly
every table in the suite is small enough that the engine cannot divide it, and
"decline by the chunk count, not the worker count" refuses those outright, so
their LIMIT results were never going to move.

parallel_query_limit compares how many rows six limit and offset shapes deliver
serially and in the workers, counted through a non-merged derived table so the
figure is the query's row count. FOUND_ROWS() is not used for this: it counts
rows the executor processed, so with OFFSET it includes the skipped ones -- 8 for
LIMIT 5 OFFSET 3 -- and serial does the same, so it is consistent but not the
number the test is about. Content is asserted only where a single row can
qualify, and the ORDER BY case asserts that it did not run in the workers, so the
gate's remaining refusal cannot lapse unnoticed.

This commit was prepared with Claude Code: it verified the limit path against a
temporarily opened gate before proposing the change, established that FOUND_ROWS()
counts processed rather than delivered rows by comparing serial with parallel
rather than assuming a bug, and confirmed no test in the suite depends on the
order of an unordered prefix.
e81a1b9f6fe3...
e81a1b9f6fe3ed51fe7fcb5f7da01f561f900bb0
Rex Johnston
MDEV-39492 Parallel Query: give a worker its own JOIN

No functional change: nothing reads the worker's JOIN until its JOIN_TABs are
driven through sub_select(). Second step of replacing the worker's private nested
loop with the executor's.

A worker cannot share the manager's JOIN. sub_select() writes join->return_tab as
it descends -- the level to unwind to, set unconditionally on entry -- and the
guard that reads it compares against JOIN_TAB*, so with a JOIN_TAB array per
worker one worker's backtrack point would be compared against another's tabs:
pointers into different allocations, and a scan that can end early for no
visible reason. The stronger reason is join->thd, read at twelve sites in that
loop for the diagnostics area, the killed flag and the row counters. Sharing the
JOIN would send all of it to the manager's THD from a worker thread, which is the
bug fixed in "a worker's result table belongs to the worker", reintroduced once
per row and for every query shape. Unlike return_tab it cannot be gated away.

JOIN declares its copy constructor and assignment private and unimplemented, so
the worker's is built by JOIN's own constructor and given what it needs by name.
That constructor is JOIN::init(), a field-initialiser that allocates nothing,
plus a shallow copy of fields_list, so it is cheaper as well as better defined
than copying the bytes of a class whose author said not to. It also matches how
Join_plan_state already holds part of a JOIN: the convention here is a named list
of fields, not an object copy.

Only three fields need carrying, which is what makes a named list practical.
Walking sub_select() and evaluate_join_record() for the shapes the gate allows
leaves thd, return_tab and found_records. map2table is read only for
split_derived_to_update; join_tab_execution_startup() reads join only inside its
two semijoin-materialization branches; JOIN_TAB::preread_init() returns before
touching join->thd unless the table is a materialized derived. All three are
already asserted inert per table by pwt_assert_tab_inert().

result is left null deliberately. It is the manager's connection to the client,
and manager_collect_and_send() is the only thing that may send a row, so a worker
that reaches for it crashes rather than writing to a socket two threads share.

pwt_assert_join_inert() is the same idea as pwt_assert_tab_inert() one level up,
and none of its checks fires across the main suite with parallel_worker_threads
forced on. The sizeof(JOIN) tripwire is debug-only on purpose: JOIN carries
dbug_join_tab_array_size under #ifndef DBUG_OFF, so its size is not the same in
the two build types, and pinning both would leave two magic numbers of which only
one is ever checked by whoever changes the class.

This commit was prepared with Claude Code: it established the three-field surface
by walking the executor and the helpers it calls, found that JOIN is
deliberately non-copyable after having earlier reported the opposite, and
confirmed that main.xtradb_mrr appearing in one forced-worker sweep is a
load-dependent warning-count flake rather than a regression -- it passed six of
seven runs in isolation and a second sweep is back at the known eighteen.
e6fc36cfe3d4...
e6fc36cfe3d4831c1ffff80b8401b9d841fc89ac
Yuchen Pei
MDEV-15621 Auto add RANGE COLUMNS partitions by interval

Allow auto partitioning by interval in PARTITION BY RANGE COLUMNS

PARTITION BY RANGE COLUMNS (col_name)
INTERVAL interval [AUTO]
(
  PARTITION partition_name VALUES LESS THAN (value)
  [, PARTITION partition_name VALUES LESS THAN (value) ... ]
)

where

- col_name is the name of one column of type DATE or DATETIME or
  TIMESTAMP

- at least one partition is supplied, and the highest partition cannot
  have MAXVALUE range

- INTERVAL interval is a positive time interval. it can be mariadb
  format or oracle NUMTODSINTERVAL/NUMTOYMINTERVAL format. Like
  versioning, the smallest unit is second, i.e. no subsecond like
  microsecond.

- DATE column cannot have interval with values less than a day

- Subpartitions are allowed, but restricted to existing subpartition
  types, i.e. [LINEAR] (KEY|HASH)

When performing one of the following DML statements on such a table,
it will first add partitions by the specified interval until the
partition covers the current time:

- INSERT
- INSERT ... SELECT
- LOAD
- UPDATE
- REPLACE
- REPLACE ... SELECT

Partition addition will not cause an implicit commit like DDL normally
does.

The partitions are named pN.

Otherwise the table behaves exactly the same as a normal RANGE COLUMNS
partitioned table.

Note that TIMESTAMP is not allowed as a type for PARTITION BY RANGE
COLUMNS otherwise.

Including the following fixes:

1.

MDEV-40088 Disallow subqueries in INTERVAL clause in range interval auto partitioning

This is consistent with system time (versioning) partitioning. Also
consistent is that both allow expressions otherwise.

2.

MDEV-40048 Allow trigger and LOCK TABLES to work with range interval auto partitioning

When a range interval auto partitioned table is the target of a
trigger, the triggering statement is not necessarily one that would
cause the auto-creation of new partitions, so we need to account for
that.

Also added support for LOCK TABLES ... WRITE.

Improved tests coverage by adapting tests from versioning.partition.

3.

MDEV-40467 Improve Y2038/2106 problem error message

When nearing the end times TIMESTAMP_MAX_VALUE, auto-partition
creation may encounter overflow when calculating new partition range.
This patch improves the message of such an overflow
df31a22d3b8d...
df31a22d3b8d8c3b29e2ad235c04528cc9fe78ef
Rex Johnston
MDEV-39492 Parallel Query: decline by the chunk count, not the worker count

"start no more workers than the engine has chunks" clamped the worker count to
the chunk count and then declined the parallel scan when the result was below
two. Those are two different questions and only the first was meant. A table the
engine cannot divide is a table the serial path reads for less, which is worth
declining. One worker because parallel_worker_threads says one is the user
asking for one worker, on a table that may divide perfectly well, and it is not
even reliably slower -- the worker's scan overlaps the manager's sending of the
rows already produced, which measured as a gain rather than a loss on a join.

Test the chunk count, and clamp after.

Two tests had been exercising nothing since that commit. parallel_query_oom
injects a failure into a worker's error queue and expects the manager to surface
one ER_OUTOFMEMORY warning; it asks for one worker, so it was being declined and
no worker reached the injection. Its table also holds two rows, one leaf page and
therefore one chunk, so it needs both halves of this fix: the table grows to more
pages than one, and the decline no longer refuses its single worker.
parallel_query kills a worker mid-scan and looks for the thread group in the
processlist; its table was two rows as well, so three workers were asked for and
none started, and the test sat in wait_condition until it timed out.

Neither failure was visible because both tests require have_debug and the tree
was a release build, which skips them -- a suite that reports every test passing
while silently skipping the two that cover the thing being changed. The tables
now hold 2000 rows, enough leaf pages for the workers each test wants, and the
selects carry a WHERE so their output stays as small as it was while the whole
table is still scanned. parallel_query's ten-worker case now records the clamp
working rather than ten warnings.

This commit was prepared with Claude Code: it found both tests broken while
setting up a debug build to exercise new assertions, traced them to the decline
condition rather than to the tests, and distinguished the two questions the one
condition had been answering.
de42301f656e...
de42301f656efe84e6c22d6cdc11cd8c301f6d8f
Rex Johnston
MDEV-39492 Parallel Query: a copy must share no node, not just no field

The gate refused a copy that reached one of the original's Item_field objects.
That is the condition under which rebinding damages the original, but it is not
the only way a shared node hurts. A node that is not a leaf carries evaluation
state, and several workers evaluating one shared object at once tear it.

  CREATE TABLE t6 (d DATE);
  SELECT * FROM t6 WHERE LEAST( UTC_TIME(), d );

wraps the constant argument in an Item_cache_time. Item_cache_int::deep_copy()
is a shallow copy, so each worker got its own cache object pointing at the
manager's UTC_TIME() item, and the workers evaluated that one item together.
Time::Time() asserted on a MYSQL_TIME left half written. The shared object is an
Item_func, not an Item_field, so the field-based test did not see it.

Ask the general question instead: does the copy reach any object the original
reaches. Item::find_item_processor() already answers that for one object, so the
missing half was a way to enumerate a tree's nodes, added here as
Item::collect_all_items_processor(). Together they replace both Field_enumerator
helpers, and the check now covers the shallow-copying classes not yet met rather
than the three now known.

type_temporal_innodb no longer crashes with parallel_worker_threads forced on. It
still differs there, in the row number a warning carries: the worker's THD does
not track the manager's current row, so a relayed warning says row 0. That
belongs with the other per-session counters the workers do not share.

parallel_query_clone gains the query above, answered serially and again with
workers. Each of that test's two cases fails on its own without this commit, the
semijoin one on the in_use assertion in Field::val_int() as before, the new one
on Time::Time().

This commit was prepared with Claude Code: it traced the assertion to the shared
cache argument, generalised the check, and wrote the test.
da532baca6ee...
da532baca6eed60cdb8adeeeb4ddc31cfaa17aab
Rex Johnston
MDEV-39492 Parallel Query: fall back to serial when the engine declines

run_worker_side_join() returned 1 -- error -- when init_parallel_workers()
reported HA_ERR_UNSUPPORTED, although its own documented contract, and the
comment on that very branch, say -1 means "the engine declined, run the query
serially". No error was raised on the way out, because nothing had gone wrong:
the engine simply refused the scan. do_select() turned that 1 into
NESTED_LOOP_ERROR, so the statement failed with an empty diagnostics area and
Protocol::end_statement() hit its DA_EMPTY assertion. In a release build the
client gets a statement that neither succeeds nor reports an error.

The engine declines this way for any read that is not a consistent read:
pscan_init_coordinator()'s first check refuses the scan when
select_lock_type != LOCK_NONE. So SELECT ... FOR UPDATE,
SELECT ... LOCK IN SHARE MODE, CREATE ... AS SELECT and INSERT ... SELECT over
a parallel-scannable table all crashed a debug server as soon as
parallel_worker_threads was set -- the optimizer picks the table, then
execution has nowhere to go. Return -1 so do_select() takes its serial path,
which is what make_join_readinfo() left in place for exactly this case by
keeping the table's serial read_first_record.

parallel_query_fallback is the new test: FOR UPDATE, LOCK IN SHARE MODE,
CREATE ... AS SELECT and INSERT ... SELECT each return their rows with workers
enabled. All four reach the decline path (the DBUG_PRINT added here fires four
times), and the first of them already crashes the server without this commit.

This also un-breaks parallel_query_join and parallel_query_worker_side, which
have been failing on this branch since they were recorded: both open with a
CREATE ... AS SELECT that took the bad path. Their serial-vs-parallel
comparisons are built from CREATE ... AS SELECT pairs, so now that the fallback
works both sides run serially and those comparisons no longer say anything
about worker-side execution. The assertion in parallel_query_join that claimed
CTAS "runs worker-side" is corrected to state what the trace key actually
reports, which is the optimizer's choice. Restoring a like-for-like comparison
at scale needs a technique that does not write from a SELECT, and is left to a
follow-up.

This commit was prepared with Claude Code: it traced the DA_EMPTY assertion to
the wrong return code with a DBUG trace, reduced the repro from
CREATE ... AS SELECT to SELECT ... FOR UPDATE, and wrote the test.
d631c215f23d...
d631c215f23d703fa30e63c1a8478718d1c2d3af
Rex Johnston
MDEV-39492 Parallel Query: give worker tables their place in the join

A worker's private table copies came out of open_table_from_share() with
TABLE::map still zero, because nothing assigns it outside the optimizer's
setup_tables(). Item_field::used_tables() reads TABLE::map, so every item
rebound onto a worker copy reported used_tables() == 0 -- it looked like a
constant. pwt_clone_rebind() re-fixes the clone, and Item_cond::fix_fields()
evaluates any argument that can_eval_in_optimize(), so cloning the condition of

  SELECT a, b, c FROM t1 WHERE a % 7 = 0 AND b > 100

evaluated "b > 100" on the manager thread, against a worker record buffer that
no row had been read into. Debug builds assert in Field_long::val_int() on
marked_for_read(); release builds read that unread buffer to compute a
null-rejection cache. Any WHERE with more than one predicate was affected --
that is, most of them.

Copy map and tablenr from the manager's table when the copy is opened, so a
rebound item computes the same used_tables() as the item it was cloned from and
nothing evaluates it at clone time. Mark all columns readable there too, before
any cloning touches those tables, rather than at the start of worker_run_query()
-- the read_set then holds for the whole life of the copy, and the per-table
loop in worker_run_query() goes away.

This was invisible because no test reached it. parallel_query_join and
parallel_query_worker_side compared parallel against serial by building the
same result twice with CREATE ... AS SELECT, and writing from a SELECT is a
locking read, which makes the engine decline the parallel scan: both sides ran
serially, so the multi-predicate WHERE never reached a worker clone.

Those comparisons are rewritten to compare something that really does run in
the workers. A result set that cannot be written to a table cannot be compared
afterwards either, so each query now goes inside a non-mergeable derived table
-- the inner select runs in the workers, the outer aggregate runs in the user
thread -- and is reduced to COUNT(*), SUM(CRC32(...)) and BIT_XOR(CRC32(...)).
That fingerprint is order-independent, which a parallel result needs since its
rows arrive in batch-completion order, and it compares the whole multiset at
full scale without printing it. The shared method lives in
include/parallel_query_fingerprint.inc.

Each comparison also asserts that the workers, not the user thread, did the
scanning: both runs read the same materialized rows, but only the serial one
also scans the driving table, so the serial run's Handler_read_rnd_next must be
the larger of the two. That is what stops these comparisons from silently going
hollow again -- with the parallel half forced back to 0 workers the check flips
from 1 to 0 and the tests fail.

Both tests were checked to be load-bearing by mutation: dropping one worker
result row in 500 moves every fingerprint (713 -> 712, 5000 -> 4990,
4967 -> 4958, 6000 -> 5988).

This commit was prepared with Claude Code: it root-caused the assertion to the
zero table map, wrote the fingerprint comparison and the include, and ran the
mutation checks.
d5230c07b516...
d5230c07b516d3aed8b441dca1f8d190c214440c
Rex Johnston
MDEV-39492 Parallel Query: COUNT and SUM with no GROUP BY

The workers do not aggregate. They run the select-project query over their chunk
and ship the aggregates' arguments, one row per qualifying row, and the manager
accumulates them with the query's own Item_sum objects by calling the server's
end_send_group() -- the function a serial plan of this shape already uses. For
the length of the drain each aggregate's arguments()[0] points at the field of
result_table its value arrived in, and is put back afterwards, because the query
belongs to the statement and may run again and run serially the next time.

So no Item_sum is copied into a worker, and that is the point. Computing partial
aggregates in the workers needs a clone per worker, and Item_sum's deep_copy()
reaches the implicit copy constructor: the clone shares the Aggregator that
Item_sum::cleanup() deletes. It also needs a combine rule per kind, since a
manager that aggregated the partials with the query's own COUNT would count the
partials. Neither arises here. The manager sees every row that qualified, exactly
as it would have read them itself, and an empty result still answers COUNT 0 and
SUM NULL through end_send_group()'s own end-of-records path rather than through a
special case.

Only COUNT and SUM are accepted, because the manager's mechanism is to redirect
arguments()[0] and those are the kinds that read their argument from there each
time they are added to. Item_sum_min_max reads it through an Item_cache built
when the aggregate was fixed, so a redirected argument never reaches it and MIN
answered NULL where it should have answered a value; AVG, STD and VARIANCE are
left out until each is checked against the same question, and the DISTINCT
variants hold the set in an Aggregator keyed to the original argument. The gate
refuses those kinds, and also a select-list item that merely contains an
aggregate, so all of them run serially.

The list end_send_group() sends is join->fields, which is not always
&join->fields_list even for these plans: with no WHERE clause the planner leaves
it at &select_lex->item_list, the same items reached through another list header.
The redirect walks the list that will be sent, and checks as it goes that
position i of it holds the item the gate classified at position i, which is what
the column mapping is indexed by.

Measured on a table of 1000000 rows with four workers, a debug build, median of
five runs: COUNT(*) 3.0 times faster than serial, SUM(val) 3.2, and 4.0 for a SUM
under a selective predicate. Shipping every qualifying row in order to count it
costs less than the scan being parallelised, which is the opposite of what was
expected when this route was first weighed against per-worker partials.

This commit was prepared with Claude Code: it wrote the transport and the
manager-side redirect, established by measurement which aggregate kinds the
redirect actually reaches (MIN answering NULL is what narrowed the set to COUNT
and SUM), wrote the test, and confirmed the test catches removal of the redirect,
a one-column shift in the mapping, and end_send() substituted for
end_send_group().
d1c729097b88...
d1c729097b88ca828bb06bddfdac027a65df9c46
Rex Johnston
MDEV-39492 Parallel Query: give constant select-list items a result field

SELECT 42, a FROM t1 crashed the server whenever parallel_worker_threads was
set. create_tmp_table() does not give a constant item a field, which is right
for a query that materialises its result and can evaluate the constant once
outside the table, but the parallel result table is not that. Its layout has to
mirror the select list item for item, because a worker projects item i into
field i and ships the record image, and the manager sends one Item_field per
field to the client. With a constant in the list the table came out one field
short, and worker_emit_row() ran off the end of the field array into a NULL
Field pointer. The manager's send list was equally short, so even without the
crash the client would have been sent the wrong number of columns. Any constant
did it, a literal, a folded expression such as 1+1, or a session constant like
CONNECTION_ID().

Pass TMP_TABLE_ALL_COLUMNS when building the result table, so every item of the
select list gets a field, and assert the layout matches the list afterwards --
the transport is positional, so a mismatch from any other cause has to be
refused rather than walked over.

That exposed a second problem in the same place. create_tmp_table() overwrites
param->func_count with the number of items it actually has to copy, and
make_result_table() was called once per worker plus once for the manager from a
single TMP_TABLE_PARAM counted once by the caller. A constant needs a field but
no copy entry, so the count dropped to zero after the first table and every
later one allocated fewer fields than its layout, tripping the assertion in
Create_tmp_table::finalize(). Each result table now starts from a freshly
counted param, which is what N identical layouts from one param needs in any
case.

Session constants come out right rather than merely not crashing: the clone is
fixed on the manager's thread, so CONNECTION_ID(), USER() and DATABASE() carry
the user session's values, not the worker's. The test asserts that against
values captured outside the query.

parallel_query_worker_side covers a literal, a string literal, a folded
expression, an all-constant select list where no column of the scanned table
reaches the result, and the session constants, plus a serial-vs-parallel
fingerprint over a select list containing a constant so the case is known to run
in the workers. Every one of them crashes the server without this commit.

This commit was prepared with Claude Code: it found the crash while probing which
expressions the gate accepts, traced both causes from the core file, and wrote
the tests.
ca158c66f6eb...
ca158c66f6eba7997c76c3d9e75780899477f0a4
Yuchen Pei
MDEV-40088 [to-squash] Disallow subqueries in INTERVAL clause in range interval auto partitioning

This is consistent with system time (versioning) partitioning. Also
consistent is that both allow expressions otherwise.
c66e2171d8a0...
c66e2171d8a0146a79093fcd5902873e201e975b
Rex Johnston
MDEV-39492 Parallel Query: ship base-table columns, not projected values

A worker used to run the whole select-project query over its chunk and ship the
projected select list. It now ships the base-table columns the query reads, in
table order, and the manager copies each one back into the field it came from in
its own table instances. Then, instead of the manager choosing a terminal
function itself, it hands each row to the plan's own: the last real table's
next_select, called exactly as sub_select() would call it.

The result is that everything downstream of the driving scan reads what a serial
scan would have left in those records, with nothing redirected. That deletes
three mechanisms rather than adding one. The reference-array slice built over the
result table is gone, and HAVING resolves through the records instead. The
per-item projection list the manager sent is gone, and the select list is
evaluated by end_send() as usual. The redirect that pointed each aggregate's
arguments()[0] at a shipped column is gone, and the aggregates read their own
arguments.

Carrying columns rather than values is what the next stage needs, not merely
tidier. An aggregation temp table is filled by copy_fields(), which walks
Copy_field pairs holding raw Field pointers into the base tables, so a temp-table
stage cannot be satisfied by re-pointing Items -- the values have to be in the
base-table records. Nothing in this commit has such a stage, since the gate still
refuses one, but this is the change that makes GROUP BY reachable.

Every kind of aggregate now runs in the workers, where before only COUNT and SUM
did, and the restriction that produced that list is gone with the redirect it
described. MIN and MAX read their argument through an Item_cache built when they
were fixed, AVG and STD keep their own counters, and the DISTINCT variants keep
the set in an Aggregator; none of that is the transport's business once the record
is right. HAVING beside an aggregate works for the same reason. GROUP_CONCAT is
refused, and now for the only reason left: its value is the order the rows arrived
in, and workers finish chunks in an order that varies between runs, so it would
answer differently each time.

Two gate checks went with the design they served. The select list is no longer
tested for clonability, because nothing clones it -- and probing it was actively
harmful, since pwt_item_is_clonable() copies an item to test it and copying an
Item_sum_min_max crashes: its copy constructor leaves cmp uninitialised while its
cleanup() deletes it. The refusal of a select list with hidden items is gone too;
it existed so that reference-array slot i and result-table column i lined up, and
neither is used now.

This commit was prepared with Claude Code: it identified that copy_fields() holds
raw Field pointers and so decided the transport shape, wrote the column shipping
and the copy-back, replaced the hardcoded terminals with the plan's own,
established by test that the aggregate-kind restriction was then unnecessary,
found that the gate's clonability probe was what crashed on MIN, and updated the
tests whose recorded expectations the wider capability changed.
c2b260d4b563...
c2b260d4b56316bf3c4b39ff7cb06e03e14b8c7b
Rex Johnston
MDEV-39492 Parallel Query: bound the split, and use it where it is needed

A chunk came from a whole B-tree level, so a scan could be divided two ways and
no other. The root's fanout gave the coarse ranges, and Ctx::split() re-divided
one of those at the level above the leaves, producing a range per leaf page. On
a table of a million rows in 44MB that is a choice between 4 chunks and 2245,
where what the query wants is a few dozen. create_contexts() therefore declined
to split at all unless the tree was at least SPLIT_THRESHOLD deep, which for a
16K page means about 16GB, so in practice a table was left at its root fanout
however lopsided that was. Four chunks measured 36.0, 36.0, 18.0 and 10.0 per
cent of the rows: the largest alone held the scan open for a third of its
serial duration, and no number of workers changed that.

A range ends where the next one begins, so leaving out a start point folds that
sub-tree into the range before it and loses no rows. create_ranges() now takes
a bound on how many ranges a page may yield and strides its records to meet it,
skipping the descent into the sub-trees it folds -- not descending is the
saving. Ctx::split() asks for as many pieces as there are threads. The bound is
what makes splitting cheap enough to rely on: each piece costs a page traversal,
a context and a deep-copied tuple, built under the index S-latch, and at leaf
granularity that outweighed the balance it bought on any scan short enough to
notice.

So the depth test goes. What replaces it is structural rather than a heuristic:
a range is split unless there is nothing under the root to divide it by, which
is m_depth below 2, meaning the ranges already end at leaf pages. Those are the
trees of one or two levels, and they have at most about a thousand leaf pages,
which are their chunks already -- more than enough, and leaving them alone also
keeps the count final so init_parallel_workers() can still size its pool by it.

Measured on a release build, six cores pinned, medians of 27 samples with the
worker counts interleaved so no configuration gets a turbo window. A scan of a
million rows returning one, which is the case that suffers most from a coarse
chunk because it has no other work to hide behind, goes from 2.10 to 5.49 times
serial. The same scan joined to two dimension tables goes from 1.76 to 3.16.
Splitting the same table without the bound reached 1.93 and 2.61 and made the
scan-only case erratic, its inter-quartile range 27 per cent against 1 to 2 for
the others. A join over a thousand-row driving table is untouched, its tree
being two levels deep. COUNT, SUM and an order-independent CRC over both
million-row tables agree with the serial answer at 1, 3, 5, 12 and 50 workers,
so the folded ranges cover every row exactly once.

Parallel_scan_chunks reports what the table was divided into, which is what
makes the bound testable rather than only measurable: the new case in
parallel_query_worker_count is a table whose root holds two records and whose
tree has a level between root and leaves. It starts all eight workers asked for
where before it started two, and it divides into ten chunks. Removing the bound
divides the same table into 56, one per leaf page, and restoring the depth test
starts two workers again.

Not addressed: a split can still produce fewer pieces than there are threads,
if the page it strides has fewer records than that. The pool has already been
sized by then and the count is not knowable earlier, so a worker can still come
away with nothing. It costs a THD rather than a wrong answer.

This commit was prepared with Claude Code: it established that chunk boundaries
are B-tree level boundaries and measured the resulting chunk sizes by counting
rows per chunk in pscan_get_next_row(), wrote the striding, took the
measurements above, and confirmed by mutation that the test detects both the
loss of the bound and the loss of the split.
c1268646aa0c...
c1268646aa0c085fa71525cb6a656cbf4b0aa67c
Sergei Petrunia
Remove incorrectly added sql/opt_sum.cc.orig
bbdf55b7ac7e...
bbdf55b7ac7e01656969ad619616867a3372c379
Yuchen Pei
MDEV-40048 [to-squash] Allow trigger and LOCK TABLES to work with range interval auto partitioning

When a range interval auto partitioned table is the target of a
trigger, the triggering statement is not necessarily one that would
cause the auto-creation of new partitions, so we need to account for
that.

Also added support for LOCK TABLES ... WRITE.

Improved tests coverage by adapting tests from versioning.partition.
a729130356ee...
a729130356ee190b7f1f6d498a64b29d82407fc3
Rex Johnston
MDEV-39492 Parallel Query: check every table the worker will open

A worker opens its own copy of every non-const table of the join, not only the
one it scans in chunks, but the gate tested only the driving table. An internal
tmp table -- a materialized derived table or subquery sitting as an inner table
-- has a share built in memory rather than read from a .frm, so
open_table_from_share() walked off the end of it and the server died in
open_worker_tables(). One statement was enough,

  SET optimizer_switch='derived_merge=off';
  SELECT ta.a, d.n FROM ta, (SELECT a AS k, COUNT(*) AS n FROM tb GROUP BY a) d
    WHERE d.k = ta.a;

and it accounted for the largest group of crashes when the whole main suite was
run with parallel_worker_threads forced on: subselect_sj2_mat, subselect_sj2,
subselect_sj2_jcl6, derived_split_innodb, subselect-crash_15755 and
group_min_max_innodb.

Apply table_can_be_parallel_scanned() to every table in the join, and refuse a
join tab with no table at all. The function name reads oddly for a table the
worker only looks rows up in, but each condition it tests is one a worker-read
table needs: an internal tmp table cannot be copied, blob payloads live outside
the record buffer and do not survive the row transport whichever table they come
from, a partitioned table cannot be opened as a plain copy, and the engine flag
is also what tells us the engine can hand the worker the manager's snapshot.
That last one closes the other half of this hole, a join whose inner table is in
an engine that cannot share a snapshot and would have been read outside the
manager's.

The six tests above no longer crash. What is left in them is the plan and
EXPLAIN churn that the 1/N cost discount causes in forced mode, plus, in the
semijoin ones, a wrong result that this crash was hiding: the worker implements
none of the semijoin duplicate-elimination strategies. The following commit
refuses those plans.

parallel_query_excluded gains the derived-table case, which asserts that the
optimizer does not choose it and that the answer matches serial execution.

This commit was prepared with Claude Code: it reduced the crash to the statement
above, made the gate check every table, and wrote the test.
9b79af290ca4...
9b79af290ca4815c84a4d7594480e7b8235c4713
Rex Johnston
MDEV-39492 Parallel Query: refuse semijoin strategies the worker cannot run

The gate refused semijoin materialization, through bush_children, and LooseScan,
through loosescan_match_tab, but nothing else. FirstMatch and DuplicateWeedout
left the plan looking like an ordinary inner join in the flat table list, and
pwt_worker::worker_join_inner() is a plain nested loop that knows nothing of
either, so it emitted exactly the duplicates they exist to remove.

  SET join_cache_level=0;
  SET optimizer_switch='materialization=on,semijoin=on,firstmatch=on,loosescan=off';
  SELECT * FROM t1 JOIN t2 ON (t2.f4 = t1.f3)
    WHERE ( 8 ) IN (SELECT t3.f1 FROM t3, t4);

answers two rows serially. Run by the workers under a FirstMatch plan it
answered three, one of them repeated and the other lost, and the count varied
with how the chunks fell. It was only reachable after the previous commit stopped
this shape of query crashing.

That the plan, not the parallel execution, was at fault could be ruled out by
keeping parallel_worker_threads at 4, so the cost model still produced the
FirstMatch plan, and adding FOR UPDATE, so the engine declined the parallel scan
and the same plan ran serially. That answers two rows.

Refuse a join tab carrying any semijoin strategy state: sj_strategy,
do_firstmatch, check_weed_out_table, flush_weedout_table, first_weedout_table,
and loosescan_match_tab as before. Testing the strategy rather than the shape of
the join means the two strategies that hide in a flat table list are covered by
the same condition as the two that do not.

parallel_query_excluded gains the query above in its smallest form, asserting
that the optimizer does not choose it and that it answers one row per matching
outer row rather than one per inner match.

This commit was prepared with Claude Code: it found the wrong result while
checking what the previous commit had uncovered, isolated plan from execution,
and wrote the test.
976af1358198...
976af1358198123a96526aed06dcee9bc03d8697
Rex Johnston
MDEV-39492 Parallel Query: the workers' statistics belong to the session

A worker counts its reads, its lock calls and everything else in its own THD, and
~THD puts them straight into the global counters. Nothing ever reached the
session that asked for the work, so SHOW SESSION STATUS was short by whatever the
workers did: after a parallel scan of a 1000-row table Handler_read_rnd_next
reported around 500, the rows this thread read out of the materialised result,
rather than the 1502 the same query reports serially. Nine tests in the main
suite noticed, in Handler_read_%, COLUMN_DECOMPRESSIONS and
Optimizer_join_prefixes_check_calls.

Each worker now copies its status counters into its pwt_worker just before its
THD is destroyed, and quiesce_workers() adds them to the session's own after
joining every worker, so only one thread ever touches either side and no locking
is involved. The worker then clears its own counters, which stops ~THD adding the
same numbers to the global counters that the session will pass on later.

Only the counters move. Memory accounting stays with the worker's THD, because
more of that THD's memory is freed after the snapshot is taken and ~THD has to
reconcile all of it with the global counters. Clearing with the
clear_for_flush_status offset leaves those fields alone, and the snapshot drops
its copies of them. Suppressing ~THD's accounting entirely instead, which is the
obvious way to avoid counting the same numbers twice, loses that reconciliation:
the server then reports an internal memory accounting error of a couple of
hundred thousand bytes at shutdown.

Session and global counters were not equally wrong. Global was right all along,
because ~THD was adding to it, so this changes which session the work is
attributed to and not the total.

parallel_query_worker_side compares Handler_read_rnd_next for the same query run
serially and in parallel, and requires the parallel run to report at least as
much. Written as a comparison rather than an exact figure because a chunked scan
ends at each chunk, so it reads a few more than a single scan does. Without the
handover the parallel run reports about a third of the serial figure and the test
fails.

This commit was prepared with Claude Code.
89ac59d6844c...
89ac59d6844ce2167862a149943fcc56326aab99
Rex Johnston
MDEV-39492 Parallel Query: refuse an expression copy that is not independent

A worker repoints the Item_field leaves of its copy of the WHERE condition, the
ref values and the select list at its own tables. That is only safe if the copy
owns those leaves. Several item classes implement deep_copy() as a shallow copy
while still holding child items, every Item_cache and Item_outer_ref among them,
so their copy keeps pointing at the original's children, and Item_cache::walk()
does visit those children. Rebinding one moved the manager's own Item_field onto
worker one's table, then worker two moved it again, so every worker and the
manager ended up sharing a field bound to some other thread's table. The
in_use assertion in Field::val_int() caught it in a debug build,

  SELECT STRAIGHT_JOIN count(*) FROM t1 JOIN t2 JOIN t3
    WHERE t1.f1 IN (SELECT f1 FROM t4) AND t2.f1 IN (SELECT f1 FROM t5);

crashing after semijoin conversion left a cached reference in the condition of a
select the workers ran. A release build reads another thread's record buffer,
and the manager's own plan is left rebound to a table that is closed when the
workers are reaped.

The gate now requires more than clonability, it requires the copy to share no
Item_field object with the item it came from. Testing it there costs nothing,
the gate already deep-copies every candidate item and throws the copy away, and
a query that fails the test declines to serial execution at optimize time.
pwt_clone_rebind() asserts the same property before it rebinds anything, so a
copy that slips through is caught at the point of damage rather than by whatever
reads a foreign table later.

Checking for a shared leaf rather than enumerating the classes that shallow-copy
means the check also covers the classes not yet met. There are more of them than
Item_cache: Item_outer_ref and Item_copy_string hold child items and copy
shallowly too.

The three tests that hit this with parallel_worker_threads forced on --
opt_hints_join_order, innodb_mrr_cpk and subselect_innodb -- no longer crash.
parallel_query_clone is the new test: the semijoin query above, plus a
correlated reference in a select list and one in a condition, each answered
serially and again with workers enabled. Without this commit the first of them
trips the assertion in pwt_clone_rebind(), and without that assertion too it
reaches the original in_use crash.

This commit was prepared with Claude Code: it traced the assertion from the core
file to the shallow deep_copy, wrote the shared-leaf check and the test.
70589ab1e739...
70589ab1e739702c444770dd03ee326bb224ae75
Rex Johnston
MDEV-39492 Parallel Query: let the manager evaluate over the row it was sent

The manager could send the rows its workers produced but not evaluate anything
against them, which is why the gate refused HAVING even though end_send() applies
join->having and the manager is where end_send() now runs. HAVING binds to the
select list through Item_ref, and an Item_ref resolves through the query's
reference array, which still pointed at items reading base tables -- tables the
manager never positioned on. So the reference array is what had to move, not
HAVING.

That is a swap the server already makes, for the same reason, in the temp-table
path: materialise a row, then point HAVING and ORDER BY at the materialised
fields rather than at the tables. sql_select.cc says so where it does it -- "the
actual switching to the temporary tables fields for HAVING and ORDER BY is done
in do_select() by calling set_items_ref_array(items1)". The manager's
result_table is a materialised row layout, so it does the same: a slice of the
reference array holding one Item_field per field of result_table, installed for
the length of the drain and restored after, from the copy
init_items_ref_array() keeps.

The slice is built by hand rather than by change_to_use_tmp_fields(), which walks
each item's result_field -- and result_table is deliberately built from clones of
the select list so that the query's own items are never bound to a tmp field, so
those are the clones' fields and not the ones to send from. What the slice needs
is only that slot i is select-list item i and that field i of result_table is
where a worker projects item i, which is true by construction: manager_send_list
is already one Item_field per field, in order, and the slot numbering matches as
long as all_fields carries no hidden items. The gate now requires that rather
than assuming it -- a hidden item would have no field to point at -- which costs
nothing today because a HAVING that names a column outside the select list is
refused by name resolution before it ever reaches here (ER_BAD_FIELD_ERROR).
That last point is also what makes the whole thing safe: every column reference
in HAVING is necessarily a reference to a select-list item, so there is nothing
left in it that would read a base table directly.

HAVING was tried the other way first, cloned per worker like the conditions are,
and it does not work: deep_copy() duplicates an Item_ref, which then still points
into the manager's array, and "SELECT a+b AS s ... HAVING s > 5985" answered zero
rows where five were right. That case is in the test for exactly that reason.
Refusing what could not be cloned refused every HAVING worth running, so there
was no safe subset to ship.

parallel_query_having compares fingerprints, not counts: which rows HAVING selects
is determined even though their order is not. Seven shapes -- a select-list
column, an alias, a column the predicate does not mention, a conjunction, an empty
result, a HAVING over a row the worker assembled from two tables, and a select list
holding a constant -- all identical to serial, all confirmed to have run in the
workers.

The forced-worker sweep is unchanged at its known failures, and main and innodb
pass at 2105.

Worth saying what this is for beyond HAVING: aggregates, GROUP BY and ORDER BY all
need the manager evaluating expressions against the shipped row, which is the same
slice. This is the piece they were waiting on.

This commit was prepared with Claude Code: it found that HAVING cannot be cloned
into a worker, identified the temp-table path's reference-array swap as the
mechanism the manager wanted, established by measurement that all_fields carries
no hidden items for these shapes so the slot numbering holds, and wrote the test
around the alias case that the cloning attempt got wrong.
664e53af808a...
664e53af808a651af7386a2456838fdedfa560f3
Rex Johnston
MDEV-39492 Parallel Query: cost by the leaf page count, not the root fanout

pscan_chunk_count_estimate() answered the root page's fanout, because that was
all a scan could be divided into. Ctx::split() now divides a coarse range
further, so the ceiling is no longer where the division starts but where it can
no longer go: a chunk cannot be smaller than a leaf page, there being nothing
under a leaf page to divide by. The estimate is therefore the clustered index's
leaf page count, which is also the right answer for a tree too shallow to be
split, where the ranges are the root's records and the root's records are the
leaf pages.

The old answer under-stated the ceiling on exactly the tables the split was
added for. A table of six thousand rows whose root holds two records was costed
as though two workers were the most it could occupy, so the optimizer stopped
believing in a third; the table has ninety-two leaf pages and the split reaches
them. Under-stating is the safe direction to be wrong in, which is why it was
left this way in the commit that added the estimate, but it is no longer
accurate, and it would have kept the optimizer from choosing plans the engine
can now execute several times faster.

The statistics are still what they are. InnoDB reports a thousand-row table as
one leaf page where its data length says four, so a table that small gets no
discount at all and is costed as the serial scan it very nearly is. That is the
same conservatism as before and it costs nothing real, the setup term dominating
anything that small either way.

The per-row factor stays at 1.16. It was measured before chunks were split and
re-measuring now gives a figure below 1, but only in the better of two regimes
the same query alternates between: at six workers the scan-only case runs at
either 19 or 31 milliseconds with nothing in between, and which one it picks
varies between runs of a few queries. Setting a constant from the good mode
would be claiming a per-row saving that is not reliably there, so the comment
now says which end of the range the number is rather than presenting it as a
measurement of the current code. The bimodality is worth its own investigation:
the fast mode implies a per-worker throughput above the serial reader's, which
would be the InnoDB prefetcher engaging, and it is a factor of 1.6.

parallel_query_worker_count now asserts on t4, whose root holds two records over
ninety-two leaf pages, that eight workers are costed below six. Six is what the
root fanout could pay for once the setup term is included, so improving past it
is the ceiling having risen. Reverting the estimate to the root fanout flips
that assertion. The t3 case, which used to carry the proof that a discount is
applied at all, now records the opposite and t4 carries the discount proof, so
neither assertion can pass vacuously.

The forced-worker sweep of the main suite is unchanged at 20 failures.

This commit was prepared with Claude Code: it derived the leaf page ceiling,
read the actual statistics out of mysql.innodb_index_stats when two of its own
assertions came out wrong, found that the setup term legitimately makes sixteen
workers dearer than four on a table this size, and settled on the pair of
assertions above after confirming by mutation that they detect the change.
662d800e35a6...
662d800e35a6cd905026620604eb6c04a76511b2
Yuchen Pei
MDEV-15621 [to-squash] Follow some gemini review comments

Also removed table_rows in a SELECT, to avoid an unrelated flaky failure
65c0bf852bd6...
65c0bf852bd6cb5fb8804f933337626f8d44b654
Rex Johnston
MDEV-39492 Parallel Query: filter by the condition the join buffer would apply

An inner table read through a BNL join buffer keeps part of its condition
somewhere a worker never looked. JOIN_TAB::make_scan_filter() copies the
conjuncts that need only that table into cache_select->cond, for the buffer to
apply as it fills, and JOIN_TAB::remove_redundant_bnl_scan_conds() then removes
those same conjuncts from select_cond -- it calls set_cond(NULL) outright when
they were the whole of it. A worker uses no join buffer and cloned select_cond,
so the conjuncts were enforced in neither place.

  SELECT STRAIGHT_JOIN b2.v FROM b1, b2
  WHERE b2.x = b1.k AND b2.v > 15 AND b1.id <= 2;

answered two rows serially and ten with workers, the extra eight being every row
of b2 that b2.v > 15 excludes. The join condition still held -- ten rows, not
forty -- which is what pinned it on the single-table half: make_scan_filter()
extracts by table map, so the two-table equality stays in select_cond and only
the single-table predicate moves.

This is the third time the same shape has bitten. The optimizer moves a
predicate out of select_cond into a structure the worker does not replicate:
handler::pushed_idx_cond, then the semijoin strategies, now the join buffer's
scan filter. So the accessor that was introduced for the first case is now
pwt_table_conds(), reporting both halves, and one clone helper ANDs them. The
gate and both clone sites go through it, so the condition the gate approves
stays the condition a worker evaluates.

Only BNL is affected. The hashed and batched buffers give the table an access
type outside JT_EQ_REF/JT_REF/JT_ALL, which the gate already refuses, so they
never reach a worker -- verified across join_cache_level 2, 4, 6 and 8, where
only level 2 differed. BNL is the default.

The conjunction is built with quick_fix_field() over two already-fixed clones,
which is what remove_redundant_bnl_scan_conds() itself does when it rebuilds a
condition out of fixed conjuncts. A worker only evaluates the item, so the
fix-time caches Item_cond::fix_fields() would rebuild are not read.

parallel_query_join gains the query above, answered both ways, asserting that it
still ran in the workers and that b2 still carries a join buffer, so the case
cannot quietly stop being covered. Reverting the cache half of the accessor
brings the eight rows back.

Found while surveying JOIN_TAB for the executor-split work: the survey wanted to
assert that a cloned tab's join-cache fields are inert, which is what turned up
the fact that they are not.

This commit was prepared with Claude Code: it traced the predicate to
make_scan_filter() and remove_redundant_bnl_scan_conds(), established by
sweeping join_cache_level that only BNL reaches a worker, and wrote the test.
64789bad53e8...
64789bad53e8a94be355c2f38477732662988f42
Rex Johnston
MDEV-39492 Parallel Query: start no more workers than the engine has chunks

parallel_worker_threads is a request, not a division of labour. InnoDB
divides the table at B-tree boundaries: Parallel_reader partitions at the root
page, so the number of chunks is that page's fanout, and create_contexts()
declines to subdivide further while the tree is shallower than
SPLIT_THRESHOLD. A 44MB table of a million rows yields four chunks, and it
yields four whether one worker was asked for or fifty.

Workers past the last chunk were created anyway. Such a worker is handed
HA_ERR_END_OF_FILE the first time it asks for work and exits without reading a
row, having already cost a THD, a table instance opened from the share for
every table in the join, the cloned conditions and select list, and a row
buffer. On that four-chunk table, asking for twelve workers started twelve and
found work for four. The query took the same time as with four, because the
other eight had nothing to contribute.

Ask the engine how many chunks it made and start no more workers than that.
handler::pscan_chunk_count() answers 0 for an engine that cannot say, and
InnoDB answers 0 as well while any chunk is still flagged for splitting: such
a chunk is replaced at run time by a variable number of finer ones, so the
count is not an upper bound yet and must not be used as one. That is what
keeps the large-table case, where the tail chunks do get split, from being
held down to the pre-split count.

Parallel_workers_started counts the workers really started, summed over the
queries Parallel_queries_executed counts, which is what makes the difference
visible from SQL. parallel_query_worker_count reads the two as a ratio,
because both are cumulative and --ps-protocol executes a statement more than
once, so workers-per-query is the figure that holds across protocols. Without
the fix a table of a single leaf page reports eight and sixteen workers where
it now reports one.

This makes nothing faster. It stops paying for threads that cannot be given
anything to read, which matters under concurrency, where they compete for the
same cores as everyone else.

This commit was prepared with Claude Code: it measured the chunk counts and
their row distribution by instrumenting create_contexts() and
pscan_get_next_row(), confirmed the failing forced-worker tests are unchanged
from HEAD, and wrote the test.
646e30ccd94f...
646e30ccd94f1ff974b5cd5dfe251e1f0efff202
Rex Johnston
MDEV-39492 Parallel Query: filter by the condition from before the pushdown

When the optimizer pushes part of a condition into the engine,
push_index_cond() leaves tab->select_cond holding only the remainder and keeps
the original in tab->pre_idx_push_select_cond. The pushed half lives on from
there in handler::pushed_idx_cond, which belongs to the handler it was pushed
into -- the manager's. A worker reads through its own handler, opened by
open_table_from_share() with nothing pushed into it, and it cloned select_cond,
so the pushed half was enforced in neither place.

  SELECT pk, a, b FROM p1,p2,p3 WHERE b >= d AND pk < c AND b = '0';

answered one row serially and six with workers, `pk < c` having been applied
nowhere, and the unfiltered rows then multiplied against the third table.
Setting index_condition_pushdown=off made the parallel answer correct, which is
what pinned it on the pushdown rather than on the plan the cost model chose.

Clone the pre-pushdown condition where there is one. One accessor,
pwt_table_cond(), is used by the gate and by both clone sites, so the item the
gate approves is always the item a worker evaluates -- the two drifting apart is
what this bug was.

This gives up what the pushdown was for: the engine no longer rejects an index
entry before the row is read, so a worker does more clustered-index work per
match than the serial plan. The alternative was to refuse these plans at the
gate, which would have cost the parallel scan altogether on a common plan shape.
Correct and parallel beats correct and serial here, and pushing a clone onto the
worker's own handler would recover the difference -- that wants the worker to
hold a real JOIN_TAB to hang the key number off, so it belongs with the cloned
JOIN, not before it.

parallel_query_join gains the query above, answered serially and again with
workers, asserting that it still ran in the workers rather than falling back and
that p1 still carries a pushed index condition, so the case cannot quietly stop
being covered. Without the fix the parallel answer is six rows.

range_innodb, which is where this was found with parallel_worker_threads forced
on, now differs only in EXPLAIN output.

This commit was prepared with Claude Code: it traced the predicate to
push_index_cond() moving it out of select_cond, and wrote the test.
5783afb3133d...
5783afb3133dbfdd8ea8b4911203bade273793aa
Rex Johnston
MDEV-39492 Parallel Query: a worker's result table belongs to the worker

create_tmp_table() runs on the manager's thread, so every worker's result table
came out with the manager in TABLE::in_use, and nothing put the worker there
afterwards, unlike the table copies in open_worker_tables(). Field::get_thd()
hands out TABLE::in_use, so when worker_emit_row() projected an item into one of
those fields and the projection raised a warning, the warning was raised on the
manager's THD from the worker's thread.

  CREATE TABLE t1 (a TIME(6));
  INSERT INTO t1 VALUES ('838:59:59.999999');
  SELECT a, a + INTERVAL 2 YEAR FROM t1;

produces ER_DATETIME_FUNCTION_OVERFLOW per row from
Item::save_date_in_field(), which passes field->get_thd() down to the function
that raises it. Three things followed. The worker's own error handler never saw
the condition, because it is installed on the worker's THD and the condition was
raised on the manager's, so instead of being relayed through the message queue it
was stored directly in the manager's diagnostics area. That store came from a
worker thread while the other workers ran, unsynchronised, on a structure the
manager also uses. And the memory for it was charged to the worker's THD, since
thread-specific allocations are accounted to the running thread, while the block
itself belonged to the manager's Warning_info, so the worker's
status_var.local_memory_used was still 2040 at destruction and
destroy_background_thd() tripped the not-freed-memory assertion in ~THD.

Put the worker in TABLE::in_use once the table is built, next to where its other
tables get the same treatment. The warning is then raised on the worker's THD,
PWT_error_handler relays it, and finalize_parallel_workers() surfaces it on the
manager after the join, which is the path it was always meant to take.

parallel_query_worker_side runs the query above serially and again with workers,
and records both. The rows and the three warnings match, so the relay is checked
for what the user sees rather than only for the absence of a leak. Reverting the
one line makes the test abort in ~THD.

type_time_hires now passes with parallel_worker_threads forced on.
type_temporal_innodb still fails there, on Item_cache_time inside
Item_func_min_max: a shallow-copied cache shared between workers, the same family
as the copy that is not independent refused in 11481453439, but reached through a
cache rather than a field, which that commit's shared-leaf test does not see.

This commit was prepared with Claude Code: it traced the leaked block to the
diagnostics area with safemalloc's report, found the manager's THD reaching the
worker through Field::get_thd(), and wrote the test.
4ce052e49b1f...
4ce052e49b1ff19e19279514a50092b7ca755b39
Yuchen Pei
MDEV-15621 [refactor] Partitioning cleanup

change p_column_list_val::fixed to a bool
remove redundant end label in partition_info::fix_column_value_functions
4b2f81b90e4b...
4b2f81b90e4bd681850e51ddbc3a190ebdeeeeec
Rex Johnston
MDEV-39492 Parallel Query: give ANALYZE the numbers the workers produced

ANALYZE reads a JOIN_TAB's counters from the tracker the optimizer left on the
Explain object, and the engine's counters from the handler it recorded there.
Both belong to the manager, and the manager never runs the driving table's read
loop, so a parallel query reported the table as untouched: r_loops 0, no r_rows,
no r_filtered, no r_engine_stats. For a feature whose whole purpose is to make a
scan faster, the tool for seeing where a scan spends its time said nothing about
it.

Each worker now counts what it did to each of its tables in the same terms
sub_select() and evaluate_join_record() use -- rows read, rows that passed the
table's condition, and one scan per probe of an inner table -- and copies the
engine's counters out of the tables while they are still open. The manager adds
all of it to the trackers and handlers ANALYZE reads, in quiesce_workers() after
every worker has been joined, so this thread is the only one touching either
side and no locking is needed. ha_handler_stats::add() already existed for the
partitioning case and does the engine half.

The driving table reports one scan, not one per worker: the chunks are one scan
of the table between them, which is what the serial plan reports and what keeps
the rows-per-scan figure comparable between the two.

  ANALYZE SELECT a FROM t1 WHERE a % 7 = 0;

now answers r_rows 5000.00 and r_filtered 14.28 whether it runs serially or in
the workers, differing only in the access type.

Not included: r_table_time_ms and r_other_time_ms, which come from the elapsed
time trackers rather than from counters, and which the workers measure in
parallel. Summing them would report more time against the table than the query
itself took, and reporting one worker's share would understate the work. That
needs a decision about what ANALYZE should mean for a parallel scan, so it is
left out rather than guessed at, and the two fields stay absent.

parallel_query_worker_side compares the tabular ANALYZE for the same query run
both ways, with the row estimate masked because it is an InnoDB approximation.
Without the handover the parallel run reports r_rows and r_filtered as NULL.

This commit was prepared with Claude Code.
4864bcac465f...
4864bcac465f81ce67602f7ea330b6c5e67d074a
Rex Johnston
MDEV-39492 Parallel Query: workers evaluate with the session's variables

A worker runs in a background THD, which starts from the global variable values,
and init_parallel_workers() copied only the identity of the session: its
security context, database and command. So the workers evaluated the session's
own expressions with somebody else's settings, and the query quietly meant
something different there. With the session in one time zone and the server in
another,

  SET TIME_ZONE = "+03:00";
  SELECT HOUR(ts) FROM t1;

answered 3 in this thread and 12 in a worker, and the same difference inside a
WHERE dropped every row a condition on a TIMESTAMP column should have kept, which
is how function_defaults_innodb failed with parallel_worker_threads forced on.

Hand each worker the variables an expression reads while it is evaluated:
time_zone, sql_mode, default_week_format and old_behavior. One at a time rather
than as a whole struct, because system_variables owns per-THD allocations, the
dynamic variables and the session tracker among them, and it also carries
option_bits, which would tell a worker it is inside the session's
multi-statement transaction and change how it commits.

Which variables those are was settled by testing rather than by listing what
looked relevant. Variables read while an expression is *built* need no handover,
because the copies are built and fixed on the user's thread: DAYNAME(),
MONTHNAME() and DATE_FORMAT('%W') keep the lc_time_names they were fixed with,
and a division keeps the scale div_precincrement gave it, so neither variable is
copied. max_allowed_packet cannot be set per session, so a worker already has the
session's value. That leaves the four above, three of them because a test showed
each one changing an answer, and old_behavior because the date and time
conversions read it as they run, the same way sql_mode is read.

parallel_query_session_vars checks the five expressions in both places, projected
and in a WHERE. Without the handover the projection answers 12, 2 and 9 where it
should answer 3, 10 and 10, and the WHERE returns no rows where it should return
three.

This commit was prepared with Claude Code: it reduced the failure to HOUR() over
a TIMESTAMP, established which variables are read at evaluation time and which at
fix time, and wrote the test.
471293662791...
471293662791da20bed38a18837d5d4290f19186
Rex Johnston
MDEV-39492 Parallel Query: the manager sends rows through end_send()

The other half of the executor split, and the last of the two hand-written
executors. manager_collect_and_send() drained a worker's batch and called
select_result::send_data_with_check() itself, keeping its own count of rows sent,
rows accepted and duplicates -- a second copy of end_send() with none of what
end_send() also does.

It now stands on the real one. end_send() is called with a null JOIN_TAB, which
is how the server itself calls it for a plan whose tables were all const: it then
takes the list to send from JOIN::fields, so that is pointed at the Item_fields
over the record layout the workers ship, for the length of the drain and no
longer. The row accounting, HAVING, and the LIMIT checks including WITH TIES are
the server's from here on.

end_send() loses its static. Its siblings end_send_group() and end_write_group()
are already declared in sql_select.h, and sub_select() already is too -- which is
what a worker was given to run in the commit before this one -- so this is the
odd one of the family rather than a new kind of exposure.

What it is for. The gate refuses LIMIT, HAVING, GROUP BY, ORDER BY and
aggregates, and every one of those is an operator that stands where end_send()
stands or just above it. With the manager's loop ending in the server's own
operator, each becomes a question of which operator to end in and what the
workers must produce for it, rather than another few dozen lines here. The
NESTED_LOOP_QUERY_LIMIT return is wired to the stop flag that the batch handoff
protocol has always had and never used, so a manager that has enough rows now
stops its producers instead of draining them.

LIMIT was checked by opening the gate temporarily: SELECT ... LIMIT 5 over a
20000-row table in four workers sends exactly five rows and terminates, and
LIMIT 999999 sends all twenty thousand. The gate is left closed, because LIMIT
without ORDER BY returns whichever rows finish first rather than the ones a
serial scan would reach, and while that is a legal answer to an unordered query
it is a different answer, so it wants a decision rather than a commit.

No test of its own: the rows the manager sends are the same rows, and the suite
is what says so. main and innodb pass at 2103, the parallel tests pass under
--ps-protocol, and the forced-worker sweep is at its known eighteen.

This commit was prepared with Claude Code: it established that end_send()
accepts a null JOIN_TAB and reads JOIN::fields in that case, which is what let
the manager reuse it without inventing a plan for itself, and verified the LIMIT
path against a temporarily open gate rather than leaving the branch unexercised.
42778193c794...
42778193c794ce779cfd6cb3b92b67cfa1102e8f
Rex Johnston
MDEV-39492 Parallel Query: a worker runs the server's nested loop

The worker's hand-written join goes, and sub_select() takes its place. Third and
last step of the executor split on the worker side; the manager still collects
and sends.

worker_join_inner() was a nested loop that understood inner equi-joins and
nothing else, which is why can_run_query_in_workers() is a list of refusals: an
outer join, a semijoin strategy, a join buffer, LIMIT or an aggregate would each
have had to be written into it a second time. What replaces it is the loop the
serial plan runs. Each of the worker's JOIN_TABs points at the next through
next_select, the last one at pwt_end_send() where a serial plan would have
end_send(), and worker_run_query() drives it the way do_select() does -- once to
produce rows, once more to signal end of records. Every capability the gate
currently refuses becomes a question about what the manager does with the rows,
not about what the worker can join.

Two pieces of plumbing were needed. The driving table reads through the engine's
chunk reader rather than a table scan, so it gets a record source of its own;
every other table keeps the one make_join_readinfo() chose, because those
functions act on the JOIN_TAB they are handed and so already read this worker's
table through this worker's ref. And the executor reaches the worker through
function pointers whose signatures carry no worker, so a thread-local holds it
for the length of the join, the way mysqld.cc holds THR_THD.

Three things had to be got right that the copy in "give a worker real JOIN_TABs"
had left plausible but wrong, all found by the suite:

  - READ_RECORD mixes plan and per-scan state. make_join_readinfo() sets the
    row-fetching function and unlock_row at optimize time -- join_read_next_same
    for a ref table, join_no_more_records for eq_ref -- while the buffers are
    filled when the scan starts. Clearing the whole struct left null pointers for
    sub_select() to call, so the plan half is now restored by hand.

  - The worker's JOIN must describe the worker's array. JOIN_TAB::pfs_batch_update()
    finds the innermost table by join_tab + table_count - 1, and with the
    manager's table_count, which counts the const tables it resolved before the
    join, that lands past the end. table_count and top_join_tab_count are the
    worker's table count and const_tables is zero, and cached_pfs_batch_update is
    recomputed per tab rather than inherited, because sub_select() asserts it
    against the live answer.

  - TABLE::reginfo.join_tab is read at execution time, not just by the optimizer:
    join_read_next_same() finds the tab it is reading for through it. A worker's
    table copy has to point at the worker's tab. That is the third field of TABLE
    to need this after map and in_use, so the standing rule holds -- assume
    anything the optimizer sets on a TABLE is missing on a copy until checked.

read_first_record() means read the first record, which the pscan source at first
did not, leaving evaluate_join_record() to run once on a buffer nothing had
filled: one extra row per worker, and NULL, so a count changed while a checksum
over the same rows did not. Worth remembering as a shape -- an off-by-one in row
count that no fingerprint catches.

The trackers ANALYZE reads now come from the executor itself, which counts
r_scans, r_rows and r_rows_after_where into JOIN_TAB::tracker -- already this
worker's. So the handover in "give ANALYZE the numbers the workers produced"
keeps working unchanged, except that sub_select() counts one scan of the driving
table per worker where the report wants one between them, so that one is no
longer summed.

No test of its own: this is a refactor, and its test is that the suite still
passes. parallel_query_join and parallel_query_worker_side between them drive
eq_ref, ref with fan-out, a three-table chain, a non-indexed inner table, a
pushed index condition and a join buffer's scan filter, and all of it now goes
through sub_select() and evaluate_join_record(). main and innodb pass in full at
2103 tests, the parallel tests pass under --ps-protocol, and the forced-worker
sweep is at its known failures.

Not what was expected: the warning that reports "at row 0" was thought to fall
out of this, since sub_select() calls reset_current_row_for_warning(). It does
not. Warnings a worker raises already relayed with the same text as serial for
every class a SELECT can produce, so that defect is about a warning type reached
another way and stays open.

This commit was prepared with Claude Code: it established which JOIN_TAB and
READ_RECORD state is plan and which is per-scan, wrote the record source, the
terminal and the chaining, and diagnosed each of the four failures above from the
suite rather than from reading.
373090adfb56...
373090adfb569463f6b741f8edd647176e795693
Rex Johnston
MDEV-39492 Parallel Query: count the queries the workers ran

Nothing reported whether a query had actually been executed in parallel. The
optimizer trace records which table the optimizer picked, but the engine can
still decline afterwards and the query run serially, so the trace answers a
different question, as parallel_query_join's assertion about CREATE ... SELECT
used to claim wrongly. The tests that compare a parallel result against a serial
one had to infer execution from a side effect instead, that the manager's
Handler_read_rnd_next stays low because the workers do the reading. That
inference is about to stop holding, the workers' statistics belong in the session
and the next commit puts them there.

Add Parallel_queries_executed, a session and global status counter, incremented
in run_worker_side_join() once the workers are running, which is after the engine
has had its chance to decline. include/parallel_query_fingerprint.inc now asserts
on it: the serial run must show none and the parallel run must show some. That is
the question the tests were always asking.

This commit was prepared with Claude Code.
2e7abfe0858b...
2e7abfe0858b57a62b9a901233c2d3d6fd416c3d
Rex Johnston
MDEV-39492 Parallel Query: GROUP BY

The manager hands each drained row to the plan's terminal function, and for a
GROUP BY that is sub_select_postjoin_aggr(): it writes the row into the server's
aggregation temp table keyed on the group, and at end of records reads the table
back and drives every stage after it. So the grouping, the aggregates, HAVING
over the groups and an ORDER BY of the groups are the server's own work, done on
the manager, and this commit adds none of them. What it does is stop refusing
them.

The gate no longer refuses group_list, and no longer refuses need_tmp. The shapes
a temp table is otherwise built for -- DISTINCT, ORDER BY, window functions, a
procedure -- are each refused on their own terms, which says what is excluded
instead of excluding a superset of it. need_tmp said little there in any case: the
gate runs from make_join_readinfo(), before make_aggr_tables_info() plans the
aggregation, so it still held its pre-planning value.

What a chunked scan cannot promise is row order, and two of the server's group
terminals need it. end_send_group and end_write_group find a new group by
comparing each row's group values with the previous row's, so given unordered rows
they would emit a group per run of equal neighbours. The optimizer picks one of
them when its plan delivers the rows already grouped, which EXPLAIN shows as
"Using filesort" without "Using temporary". end_update and end_unique_update look
each group up in the temp table by key and do not care what order the rows arrive
in, and that is the plan for a GROUP BY no index resolves -- the case worth
parallelising.

Which terminal was chosen is not known when the gate runs, for the same reason
need_tmp was not. So pwt_plan_needs_group_order() asks in run_worker_side_join(),
after the aggregation has been planned, and declines the parallel scan the way the
engine can when it will not do a consistent read. AGGR_OP gets a getter for
write_func so the question can be asked.

With no GROUP BY, end_send_group is the terminal and is safe: there is one
implicit group, group_fields is empty and no group change is ever reported. That
is the aggregate case, which continues to run in the workers.

An ORDER BY that reaches execution unrefused is sound only when there is an
aggregation temp table for the sort to be applied to, because then the order is
established on the manager after every row has arrived and the order they arrived
in does not matter. Asserted rather than assumed, since the gate cannot see such
an ORDER BY either.

Measured shapes are in the new test, including the two halves of the order
decision: a GROUP BY that gets the keyed temp table runs in the workers, and
COUNT(DISTINCT) per group and WITH ROLLUP get the sorted plan and are declined.
GROUP BY the primary key is refused earlier and for an unrelated reason -- the
plan reads the clustered index in order, and a driving table that is not a full
scan is not one the gate accepts.

This commit was prepared with Claude Code: it traced the terminal-function chain
to find that one call reaches every post-scan stage, established from
set_postjoin_aggr_write_func() which terminals depend on row order, wrote the
gate relaxation and the execution-time check, and measured each shape in the test
against its serial answer -- correcting two explanations it had first written,
about indexes forcing the ordered plan, that EXPLAIN then disproved.
1dce7286861d...
1dce7286861dd0d0db11481e3872b5d6c13e2612
Rex Johnston
MDEV-39492 Parallel Query: cost a parallel scan by what it can actually do

scale_cost_for_parallel_scan() divided the row and copy cost by
parallel_worker_threads, with nothing else in the arithmetic. Three things were
missing and all three pointed the same way, so the optimizer was most
optimistic exactly where parallelism helps least.

The divisor was the request rather than what the table can be divided into. The
engine partitions at the root page, so the chunk count is that page's fanout,
and a worker beyond the last chunk gets nothing to read. With
parallel_worker_threads=50 on a table of a million rows in 44MB, the optimizer
believed the scan was fifty times cheaper. It is four chunks, so a little over
twice as cheap. An error of that size is enough to prefer a parallel full scan
over a perfectly good index, which is what the forced-worker runs of
greedy_optimizer, partition_explicit_prune, type_temporal_innodb, vector and
xtradb_mrr were showing.

Reading a row through a worker also costs more than reading it serially, some
1.16 times for a scan whose rows are cheap to evaluate, because every row is
copied into a batch buffer, handed over under a mutex and read again by the
manager. And each worker has to be built: a THD, a table instance per table in
the join opened from the share, the cloned items and a row buffer, some 22
microseconds. Both are measured on a release build.

So: clamp the divisor to handler::pscan_chunk_count_estimate(), multiply by the
per-row factor, and add the setup cost. The estimate comes from the clustered
index statistics rather than from a page read, since it is wanted at optimize
time: for a tree of three levels the root's fanout is every non-leaf page but
the root, and for a tree of two levels the root's records are the leaf pages.
It needs the statistics to have been gathered, and answers 1 without them, so
an un-analyzed table is costed as the serial scan it may well end up being.
That is the safe direction to be wrong in, because it never claims parallelism
the table cannot supply.

init_parallel_workers() now declines a table the engine cannot divide at all.
One chunk means one worker reading the whole table by itself, which is the
serial scan plus a copy, a handover and a re-read for every row of it. There is
nothing to parallelise and the serial path does it for less.

parallel_query_join's index-condition case needed two changes as a consequence,
and they are worth spelling out because the case had been passing for the wrong
reason. Its p2 held two rows in a single leaf page, so the engine cannot divide
it and the query now runs serially; p2 grows to more than one page. Its plan
also depended on the old cost model: a two-row table was only ever the driving
table because dividing its scan cost by four made it look cheaper than reading
p1 by its index. With the cost fixed the optimizer puts p1 first, where pk < c
cannot be pushed into an index at all, so the pushed-condition situation the
test exists to cover disappeared from the plan. STRAIGHT_JOIN now pins the
order the case needs. Reverting pwt_table_cond() under the new test answers
three thousand rows where one is correct, so the coverage is intact and rather
sharper than the six rows it used to produce.

The forced-worker sweep of the main suite goes from 25 failures to 20, the five
above, and adds none.

This commit was prepared with Claude Code: it measured the per-row factor and
the setup cost, derived the chunk-count estimate and checked it against
instrumented chunk counts on two tables, found that the index-condition case
had been relying on the mis-costing, and confirmed by mutation that both halves
of this change are what the tests detect.
1d2ac09ffbab...
1d2ac09ffbabb847c4f37500d2eab3b02ffddb33
Rex Johnston
MDEV-39492 Parallel Query: give a worker real JOIN_TABs

No functional change. This is the first step of replacing the worker's private
nested loop with the executor's, and it replaces the structure that loop reads.

pwt_jointab described a table in five fields: its TABLE copy, the access type, a
ref, a condition and the sorted flag. sub_select() and evaluate_join_record()
want a JOIN_TAB, so every capability added to the private loop would have to be
added again when the split moves to the real one. A worker now holds an array of
JOIN_TAB, n_tables of them, indexed the way pwt_manager::mgr_tabs is so that [0]
is the parallel-scanned driving table -- the indexing sub_select() will be
started from. The driving table's condition moves onto join_tabs[0].select_cond
with every other table's, so worker_cond goes away.

The tabs are copied from the manager's and then rebound, rather than built from
nothing, so that a field this code does not know about holds the value the
optimizer chose instead of a zero that would look deliberate. Overwritten: the
TABLE, the condition, the ref, the tracker. Cleared: the places a condition can
hide (pre_idx_push_select_cond, cache_select) now that select_cond carries all of
it, and the join buffer, which a worker does not use. Nulled deliberately: the
JOIN, next_select, read_first_record and read_record, which the next two steps
fill in -- reaching them before then is a crash rather than a read of the
manager's execution state.

That last distinction is the point of pwt_assert_tab_inert(). A copy keeps the
manager's pointer in anything the copy does not overwrite, which is the trap
TABLE::map and TABLE::in_use were, and the fields that matter here are ones the
gate is supposed to have excluded rather than ones this code sets. So each is
asserted inert on the manager's tab before the copy: the outer-join chain, the
semijoin strategies and their weedout and firstmatch tables, emb_sj_nest, the
rowid filter, range access, split derived materialization, the DISTINCT
shortcut, HAVING, and the access type. Relax a gate without teaching the copy
about the field it lets through and a debug build stops here instead of
executing with another thread's state.

Writing those assertions is also what found the BNL scan filter fixed in "filter
by the condition the join buffer would apply": asking whether cache_select was
inert turned out to have the answer no, and a wrong result behind it. They are a
survey as much as a guard, so they are worth running before the code they
protect. Verified live on a debug build -- none of them fires across the main
suite with parallel_worker_threads forced on, 1418 tests, and a deliberately
false one aborts the server, so the negative result is a measurement rather than
an assumption.

main and innodb pass in full, 2103 tests, and the parallel tests pass under
--ps-protocol. The forced-worker sweep is at its 18 known failures, all EXPLAIN
and cost output.

This commit was prepared with Claude Code: it established which JOIN_TAB and
JOIN fields the real executor actually reads (4 of JOIN, some 29 of JOIN_TAB, of
which the gate makes most inert), wrote the copy and the assertions, and checked
the assertions execute rather than trusting that they had.
11dfcaf2c8a8...
11dfcaf2c8a870438ace2e8ce46f6316e685891e
Yuchen Pei
MDEV-15621 Auto add RANGE COLUMNS partitions by interval

Allow auto partitioning by interval in PARTITION BY RANGE COLUMNS

PARTITION BY RANGE COLUMNS (col_name)
INTERVAL interval [AUTO]
(
  PARTITION partition_name VALUES LESS THAN (value)
  [, PARTITION partition_name VALUES LESS THAN (value) ... ]
)

where

- col_name is the name of one column of type DATE or DATETIME or
  TIMESTAMP

- at least one partition is supplied, and the highest partition cannot
  have MAXVALUE range

- INTERVAL interval is a positive time interval. it can be mariadb
  format or oracle NUMTODSINTERVAL/NUMTOYMINTERVAL format. Like
  versioning, the smallest unit is second, i.e. no subsecond like
  microsecond.

- DATE column cannot have interval with values less than a day

- Subpartitions are allowed, but restricted to existing subpartition
  types, i.e. [LINEAR] (KEY|HASH)

When performing one of the following DML statements on such a table,
it will first add partitions by the specified interval until the
partition covers the current time:

- INSERT
- INSERT ... SELECT
- LOAD
- UPDATE
- REPLACE
- REPLACE ... SELECT

Partition addition will not cause an implicit commit like DDL normally
does.

The partitions are named pN.

Otherwise the table behaves exactly the same as a normal RANGE COLUMNS
partitioned table.

Note that TIMESTAMP is not allowed as a type for PARTITION BY RANGE
COLUMNS otherwise.
03a84c6b19ca...
03a84c6b19cadf22fa60d72eea96c61ad026b233
Rex Johnston
MDEV-39492 Parallel Query: worker tables inherit the manager's column bitmaps

A worker's table copies were marked with use_all_columns(), which points both
read_set and write_set at the share's all_set. Correct, since it is a superset
of what the worker evaluates, but coarser than it needs to be in two ways.

InnoDB builds its row fetch template from read_set, so with every column marked
a worker converts every column of every row it scans to MySQL format, not just
the ones its cloned condition and select list reference -- work paid on the hot
path that parallel query exists to shorten, and it grows with the width of the
table rather than with the query. And an all-columns write_set on a table the
worker only ever reads gives up an assertion: a store into a source field no
longer trips marked_for_write().

Copy the manager table's read_set and write_set into the copy's own
def_read_set/def_write_set instead, and point the copy at those. The optimizer
has already marked exactly the columns this query reads, and
open_worker_tables() runs after it has finished, so those bitmaps are final; the
copy owns its bitmaps, so nothing is shared with the manager, and
column_bitmaps_set() signals the engine to rebuild its template.

This keeps the property the marking was added for: every column a cloned item
could touch is in the read_set before any cloning happens, because the optimizer
marked precisely that set.

Two cases in parallel_query_worker_side cover the fidelity of the copy, both
through the serial-vs-parallel fingerprint: a table with eighteen columns where
the query references two, and a virtual column whose base column is read but
never projected. Clearing a single bit of the copied read_set makes both of them
fail (Field_long::val_int() asserts on marked_for_read()), as does
parallel_query_join.

Index-only reads are unaffected by this: they need ha_start_keyread(), which the
worker-side path never issues, for any read_set.

This commit was prepared with Claude Code: it identified the cost of the blanket
marking, made the copy, and added the two coverage cases.
BuildBot (0.8.9) working for the MariaDB project.
Page built: Tue 04 Aug 2026 06:47:40 (UTC)