Console View

Categories: connectors experimental galera main

Legend:

Passed

Failed

Warnings

Failed Again

Running

Exception

Offline

No data

connectors

experimental

galera

main

fd8241568429...

fd82415684290c435bbf69817f9c54883d18dfb8

Teemu Ollakka

teemu.ollakka@mariadb.com

Merge remote-tracking branch 'origin/MGL-337' into 4.ee-MGL-337

f4800db37042...

f4800db370425aadf050346c62ac8c9aaead60c8

Rex Johnston

rex.johnston@mariadb.com

MDEV-39492 PQ: charge a parallel plan for every row the workers ship

The cost of a parallel plan was the workers' share of the join plus the
setup, and nothing for the result: the manager receives every row the
workers ship, by itself, and a plan whose output is a large share of
what it reads handed it as many rows as the join produced at no cost.
A cross join of 1000 x 2000 rows was costed cheaper in the workers and
ran there 15 times slower than serially, the time going into writing
two million rows into the workers' containers and draining them.

The join-order search now adds the cost of shipping to a parallel plan
when the plan is complete, which is the first point its row count is
known: at the leaf of best_extension_by_limited_search() and at the end
of cost_straight_join(), as parallel_ship_cost(). A partial plan carries
no estimate of it, which is a lower bound, so the pruning by cost above
the leaf is unchanged.

What is shipped depends on the query's shape, decided once per
optimization by parallel_estimate_shipped_groups() into
JOIN::parallel_groups. A query that does not aggregate ships every
joined row. One that aggregates over everything it reads ships a partial
per worker, which is charged nothing. A grouped one ships a partial per
group per worker, and the number of groups is estimated as the product
of the GROUP BY columns' distinct values from the engine-independent
statistics, capped by the joined rows; without statistics the estimate
is one group, the optimistic side, which leaves such a query costed as
before rather than charged for every joined row.

A row costs optimizer_parallel_row_cost, a new session variable, while
the worker's container is in memory: 0.03 us by default, measured on a
1,000,000-row scan shipping two INT columns. A container is a temporary
table that starts as a HEAP table of at most min(tmp_memory_table_size,
max_heap_table_size) bytes and is converted to Aria when that is
exceeded, after which every row goes through the page cache: the same
scan shipping a VARCHAR(30) as well took 630 ms in the workers against
65 ms serially, 0.6 us a row. So the rows past what the containers hold
in memory, the busiest worker holding rows / parallel_divisor of them
and each container one HEAP table's worth, cost DISK_TEMPTABLE_LOOKUP_COST
more. The row width is the plan's tables' used field lengths at their
declared length, which is how HEAP stores a VARCHAR, not capped at the
engine's mean row length as the join buffer's estimate is. Set to 0 the
variable switches the charge off, the disk tier included.

On the scratch server, with four workers: the two-INT-column scan of a
million rows stays divided (43 ms against 61), the VARCHAR one is read
serially (93 ms, where the divided reading took 630), and the TPC-H
plans are unchanged. The estimate is the optimizer's: a filter it
cannot see through, such as b % 10 = 0, is costed as shipping every row.

Tests. main.parallel_query_costing: a join that ships every joined row
is divided at the default charge and serial at a large one, and a
grouped query at the same charge stays divided, with the trace's
parallel_rows_shipped for each. main.parallel_query_join, _sort and
_spill switch the charge off beside the setup cost, as what they ship
and spill is what they test, not what it costs. The variable is added
to the sysvars, optimizer_costs, opt_context_replay and mysqld--help
results.

This commit was prepared with Claude Code (Fable 5.1), which measured
the per-row shipping cost in memory and after a spill and found the
spill to be the ten-fold difference, wrote the estimate and the cost,
adapted the tests, rebuilt the worktree and ran the parallel_query tests
in both protocols and the main suite.

e82be40559b6...

e82be40559b6c2604af51d88e1ffcfbb63d6186f

Teemu Ollakka

teemu.ollakka@mariadb.com

Merge remote-tracking branch 'origin/4.x' into 4.ee-MDEV-38920-tests

e17a98e27ced...

e17a98e27ced3114fef9875dd4d92ef849a97760

Rex Johnston

rex.johnston@mariadb.com

MDEV-39492 PQ: key lookups under a divided driver are credited with
the square root of the division

TPC-DS Q68's grouped derived table, with eight workers, was costed
cheaper with store_sales divided among them and date_dim, store,
customer_address and household_demographics looked up by eq_ref under
it -- 395 against 514 for the serial plan that starts from the 72
matching dates -- and ran in 0.70 s where the serial plan runs in
0.24 s.

The join-order search divided every position under a parallel driver
by the driver's parallel_divisor, as if W workers did W times the work
of one. A scan does: each worker reads its own pages at the serial
rate, store_sales at 0.14 us a row per worker against 0.12 serially.
Key lookups do not. The 2,880,404 eq_ref probes of date_dim took
0.35 us each serially and 1.6 us each per worker with eight workers
probing the same index at once, and the plan as a whole ran 1.5 times
faster with two workers, 2.1 with four and 2.3 with eight.

position_cost_in_plan() now divides a ref, eq_ref or ref_or_null
position by the square root of the divisor. That fits the two- and
four-worker measurements and is still generous at eight; it is also
enough to make Q68's divided plan lose at eight workers, and the query
runs the serial plan in 0.32 s. A BNL or hashed join buffer scans the
inner table and divides like a scan, so it is left at the full
division.

What else it moves was checked: TPC-DS Q29 and Q66 keep their divided
plans at four and eight workers, and TPC-H Q1, Q3, Q5, Q10 and Q12 keep
theirs at four with their times unchanged (Q7 and Q11 serial as
before). The dimension lookups under a divided fact table are few next
to the rows scanned, and the discount on them does not reach the
decision.

main.parallel_query_costing: the grouped case's trace lists one complete
parallel plan fewer, the search having pruned the other sooner.

This commit was prepared with Claude Code (Fable 5.1), which measured
the forced plan's scaling by worker count and the per-probe cost from
ANALYZE, wrote the rule, and re-ran the TPC-DS and TPC-H queries, the
parallel_query tests in both protocols and the main suite.

e02e83bd6de8...

e02e83bd6de8cfd2670ea9e6ebab391ac6917a4e

Rex Johnston

rex.johnston@mariadb.com

MDEV-39492 PQ: spread the driving rows over no more workers than the pieces they fill

The rows that survive the driving table's WHERE were spread, in the
costing, over as many workers as the engine could cut the table into.
Where in the table they lie was not asked, and for the tables an
analytic query filters most often it is not spread at all: the 366
rows of date_dim for one year are four of its 671 pages, next to each
other in the primary key, and the 122 returned dates of TPC-DS Q29 are
two. The partitioner hands out a chunk per page, so those rows reach
four workers of eight, or two, and the join under them runs at the
pace of the busiest while the costing credited an eighth of it to each
worker.

parallel_scan_cost() now takes the surviving rows to lie together:
R rows of a T-row table of P pieces fill ceil(R * P / T) of them, and
the rows reach no more workers than that. For rows spread evenly
through the table this is pessimistic, and it only bites when the
survivors are fewer than N pieces' worth; for rows that do lie
together it is what the workers will find, and a plan chosen on a
division the workers cannot make costs more than a division not made.
The busiest worker's share then follows as before, R / ceil(R / W).

On TPC-DS Q66 the divisor for date_dim drops from 8 to 4, which is
what the workers achieve, and the plan is still chosen at cost 114
against the serial plan's 168. It still loses, 457 ms against 315,
because the worker holding the fourth quarter does half the work:
sales peak before the holidays, and how much of the join each driving
row carries is not something the statistics describe. Q29's divided
plan is unchanged at divisor 2 and runs as before, and the TPC-H plans
and times are as the costing commit measured them.

The cap moved a second kind of plan into the wrong pruning family. A
large fact table of which one percent survives, scanned in parallel
with a dimension looked up per surviving row, now has a divisor of 1
-- its 200 rows fill one page -- and the pruning heuristic compared it
with the serial prefixes, where the five-row dimension beats it on
rows and on cost and drops it, although its scan alone, divided four
ways, repays the setup several times over. The parallel family is
therefore for a parallel candidate with something to gain by the
division, either in the join under it (parallel_divisor > 1) or in the
scan itself (the new POSITION::parallel_scan_pays, the divided reading
costing less than the serial one); a candidate with neither is one of
the serial prefixes but for the setup, as before.

main.parallel_query_costing's 2000-row dimension becomes a 20000-row
one whose first 2000 rows match, four pages that four workers can
divide, and the sections it supports read the same.

This commit was prepared with Claude Code (Fable 5.1), which measured
the per-worker row counts of Q66 and Q29 against the divisor the
costing had assumed, implemented the cap, adapted the test, and ran
the parallel-query tests in both protocols and the main suite.

df5121eb15bb...

df5121eb15bba68490cbdd0d47457bf4405e69a3

Andrei Elkin

andrei.elkin@pp.inet.fi

MDEV-42004 STOP SLAVE waits for slave_net_timeout when master is unreachable

STOP SLAVE could take up to slave_net_timeout to stop the slave IO thread
when the master accepts the connection but does not answer, e.g. when its
host is frozen or its packets are dropped.
The IO thread was blocked in a reconnect
try_to_reconnect() -> safe_reconnect() -> connect_to_master() ->
mysql_reconnect() -> mysql_real_connect()
waiting for the greeting of the master, and nothing could wake it up in the
base of this patch, 11.4 at c5272f132d3. The stall is not specific to
semi-sync: an asynchronous slave waits the same, and a semi-sync one waits
in addition for rpl_semi_sync_slave_kill_conn_timeout at the IO thread exit,
which this commit does not change.

The wait was interruptible before MDEV-32567 removed the pthread_kill() of
the slave thread in terminate_slave_thread() (commit 013fc02a23e) in order to
rely on the THD::awake_no_mutex() -> vio_shutdown(active_vio) mechanism.
This bug exposes that it does not cover slave reconnect, as the slave's IO
thread does not bind its THD::active_vio to a new socket vio at the time of
the reconnect attempt.

This commit makes that "tentative" vio reachable for awake_no_mutex().
For that mysql_real_connect() now publishes the vio to the slave thread
right after it is created.
See the new hook slave_io_thread_attach_vio() in the TCP branch (the Unix one
is irrelevant for slave) that pairs an existing slave_io_thread_detach_vio().
So now in the patch when STOP SLAVE shuts the vio down, which wakes up
the poll().
The detach hook is called when the bound to THD::active_vio is about
to be destroyed.

connect_to_master() reports the interrupt to the error log when
log_warnings > 2 and the slave has been stopped after the failed connect:

Slave I/O thread: reconnect to master '[email protected]:19000' was
interrupted by the slave stop: Lost connection to server at 'reading
initial communication packet', system error: 0 ...

Tested with rpl.rpl_stop_slave_unreachable_master.

The test simulates an unresponsive master: it is not shut down but frozen
with SIGSTOP, so the kernel still completes the TCP handshake of a new
connection, while no greeting is sent, and the IO thread gets into its
reconnect. The test waits for that to most probably happen and then
runs STOP SLAVE, which normally (not on an overloaded server) must not wait
for slave_net_timeout.
A shut-down master would not serve for that: its port is closed, so the
connect fails at once with "Connection refused", and the IO thread, which
then sleeps connect_retry in slave_sleep() (it checks io_slave_killed()),
stops promptly without this fix.
The test is not run on Windows, as it needs SIGSTOP.

With --let $debug_test= 1 the test prints how long STOP SLAVE took. Measured
this way, it returned after 3.5 s before the fix and 1.0 s after, with
slave_net_timeout=3 and rpl_semi_sync_slave_kill_conn_timeout=1; the same
test with a plain asynchronous slave took 2.5 s and 0.0 s.
For a reliable run on BuildBot the time is not checked. The fix is proven
by the error text in the note above. A reconnect that waits for the greeting
until the timeout is out fails with "waiting for initial communication
packet ... Connection timed out", while the one whose vio is shut down fails
with "reading initial communication packet". The test greps the slave's
error log for the latter with search_pattern_in_file.inc, which reports

FOUND 1 /Slave I/O thread: reconnect to master .* was interrupted by the slave stop: .*reading initial communication packet/ in mysqld.2.err

da65599d35ad...

da65599d35adcaf6deaf416187eb323487d64a47

Jan Lindström

jan.lindstrom@mariadb.com

Merge branch '4.x-MDEV-42140' into 4.ee-MDEV-42140

d20762cf0d81...

d20762cf0d8104de8a4467a7f4a2c35bbba7bac4

Alexey Yurchenko

alexey.yurchenko@mariadb.com

Merge pull request #1136 from mariadb-corporation/4.ee-MDEV-38920-tests

MDEV-38920 add gcomm unit tests for the EVS install timeout defect

c0cae835a355...

c0cae835a355dd4de74fe520e5a0e9a93c02d83a

Jan Lindström

jan.lindstrom@mariadb.com

MDEV-42140 fix uncaught exception from allowlist check in IST receiver

galera::ist::Receiver::run() called acceptor_->accept() outside its own
try block. AsioAcceptorReact::accept() throws gu::Exception when the
allowlist check rejects the peer, and run() is invoked straight from
the raw pthread entry point run_receiver_thread(), which has no
exception handler of its own, so the exception reached
std::terminate() and aborted the whole process (SIGABRT) instead of
being reported through ist_end().

In galera.galera_wsrep_schema_detached this happened when a node
rejoined the cluster while mysql.wsrep_allowlist still held a stale
entry: the IST receiver's incoming connection got rejected and crashed
the process instead of failing the state transfer cleanly.

Move the accept() call inside the existing try block so the exception
is caught like any other IST failure. The acceptor must also be closed
on this error path, not only after a successful accept: otherwise a
later finished()/interrupt() reconnects to the still-listening
acceptor and blocks forever waiting for a handshake nobody sends.

Add galera/tests/ist_check.cpp:test_ist_allowlist_rejected_v7, which
reproduces the crash deterministically via a mock allowlist service
that rejects every connection, confirmed to abort with the same
uncaught gu::Exception on pre-fix code and pass on the fix.

Assisted-by: Claude:claude-4.6-sonnet

c075f7399d54...

c075f7399d546096f31352d5107ce4bc64f391b3

Jan Lindström

jan.lindstrom@mariadb.com

Merge branch '4.x-26.4.29' into 4.ee-26.4.29

ba0f575b0deb...

ba0f575b0deb8581dee5b69dace580900ff54d36

Jan Lindström

jan.lindstrom@mariadb.com

MDEV-42000 Interrupt IST receiver's SST wait on shutdown

ist::Receiver::run() waits for ready() to be called once SST/IST
negotiation completes. If the server starts closing before that
happens, nothing signals the wait, and the receiver thread, and
ReplicatorSMM::close() with it, block forever.

Add an interrupted_ flag the wait loop also checks, and a cancel_wait()
method that sets it and signals the condition variable safely from any
thread. Call it from start_closing() and from finished(). Add
test_ist_interrupted_before_ready_v7 as a regression test.

Assisted-by: Claude:claude-4.6-sonnet

b9a81a76343c...

b9a81a76343c2e11c51195058a9cdfae601851ba

Rex Johnston

rex.johnston@mariadb.com

MDEV-39492 PQ: a GROUP BY column the WHERE binds to a constant is a group column

TPC-DS Q66 ran three times slower with eight workers than without, and
the workers never ran. Each UNION branch groups by the warehouse
columns and d_year under d_year = 2000. The gate asked before the plan
(pwt_query_shape_allows_workers) admitted the branch, so the search
costed date_dim as a parallel driver and chose it, at cost 26 against
the serial plan's. The gate asked after the plan refused the same
branch: "a select list item reads a column outside an aggregate that
GROUP BY does not name". The plan then ran serially in the shape
chosen for the workers, date_dim first with the fact table looked up
per date, 1015 ms against 321.

The two gates disagreed because they read different group lists. The
first sees the GROUP BY as written, with d_year in it. By the time the
second asks, remove_const() has dropped d_year from join->group_list
and from the aggregation table's key as having one value everywhere,
and the select list's d_year is no longer named by either. The column
is a group column all the same: bound to a constant by the WHERE, it
has the one value in every group, which is what the check exists to
establish.

Pwt_group_checker now accepts a column that the join's multiple
equalities bind to a constant, looked up through the COND_EQUAL both
gates are given, whether or not the group list names it. The two gates
agree again, and the branch runs divided. A query cannot be refused on
this ground after the plan if it was admitted before it, which is the
property the pre-plan gate was introduced to provide: a parallel plan
the workers will not run must never be chosen.

main.parallel_query_why gains the shape: a GROUP BY column bound to a
constant, read by the select list, with no refusal and the driving
table divided.

This commit was prepared with Claude Code (Fable 5.1), which traced the
Q66 refusal to the trimmed group list, implemented the check through
the multiple equalities, wrote the test case and ran the parallel-query
tests in both protocols and the main suite.

a9833fec498d...

a9833fec498d9c1c12bbbc49c74fad5d0e11965e

Vladislav Vaintroub

vvaintroub@gmail.com

MDEV-42004 STOP SLAVE waits for slave_net_timeout when the primary is
unreachable

STOP SLAVE could take up to slave_net_timeout to stop the slave IO
thread when the primary accepts the connection but does not answer,
e.g. when its host is frozen or its packets are dropped. The IO thread
is blocked in mysql_real_connect(), waiting for the server greeting,
and THD::awake() could not interrupt it: until the connect succeeds,
the vio is not THD::active_vio.

Let the client tell the owner of the connection about its vio. A new
MYSQL_OPT_VIO_CALLBACK (and MYSQL_OPT_VIO_CALLBACK_ARG, its first
argument) is called with the new vio as soon as it is created, and with
NULL right before it is deleted. It is kept in options.extension, so
mysql_reconnect() carries it over to its temporary MYSQL object.

The slave IO thread sets it up in connect_to_master(). The callback
makes the vio THD::active_vio, so STOP SLAVE shuts the socket down with
vio_shutdown(), which wakes up the blocked wait. If STOP SLAVE has
already set abort_slave, the vio is shut down right away. This works on
all platforms, no signals are needed.

The test does not need SIGSTOP and runs on Windows too. The "primary"
is a socket that listens and never accepts or answers. It fails without
the fix (STOP SLAVE waits for most of slave_net_timeout), and passes in
about a second with it.

The idea of the test, to freeze the primary and check that STOP SLAVE
does not wait for slave_net_timeout, is reused from Andrei Elkin's
original patch for this bug.

Co-authored-by: Andrei Elkin <[email protected]>
Assisted-by: Claude:claude-sonnet-5-5

a2c5329b9139...

a2c5329b9139b4b1c318d861688a1fe6ae83eb91

Rex Johnston

rex.johnston@mariadb.com

MDEV-39492 PQ: each worker runs a join buffer of its own

A plan that read a table through a join buffer was refused by the gate
(101a4ccc724), and the join-order search costed the tables under a
parallel driver without one so as not to choose such a plan. That kept
the workers out of every plan where a buffer is the right join method:
a cross join, a small dimension scanned under the divided driver
(TPC-DS Q66's ship_mode, which the gate refused on that ground), any
join without a usable index. Before 101a4ccc724 the worker stripped the buffer off its copy of
the tab and joined a row at a time, scanning the inner table once per
driving row instead of once per buffer-full, which is where the order of
magnitude went on

SELECT * FROM s1 JOIN s2 ON (1) -- 1000 x 2000 rows

Now a worker runs a buffer of its own. The saving a buffer gives is fewer
scans of the inner table, and a buffer per worker gets the same saving: a
worker with a quarter of the driving rows fills it a quarter as often, so
the inner table is scanned about as often in total as serially, at most
one extra partial-buffer scan per worker, and those scans run
concurrently. There is nothing to share between the workers.

setup_worker_jointabs() builds, for a tab the manager reads through a
BNL or BNLH cache, a new JOIN_CACHE of the same kind over the worker's
own JOIN and tabs, chained to the previous tab's the way the manager's
is, and calls init() on it. init() walks the worker's tabs up to this one
and takes the fields of the worker's table copies, and it sizes and
allocates the buffer through the worker's THD, which carries the
session's join_buffer_size, so each worker gets a buffer of the session's
size. The tab before it is pointed at sub_select_cache(), which fills
the buffer and flushes it when full and on the end-of-records call the
worker's driver already makes. The buffer is freed with the worker's
tables (pwt_worker::close_tables()).

A buffer evaluates two conditions the nested loop does not, and both
needed a home on the worker's tab. The scan filter in cache_select is
applied to each row of the inner table as the buffer scans it
(JOIN_TAB_SCAN::next()), and remove_redundant_bnl_scan_conds() has taken
its conjuncts out of select_cond, so the worker's tab gets an SQL_SELECT
holding a clone of it, rebound to the worker's tables. The remainder is
evaluated by JOIN_CACHE::check_match() through select->cond rather than
select_cond, so a cached tab gets a second SQL_SELECT over the cloned
select_cond; neither carries a quick. The gate checks the scan filter is
worker-safe whole: it is applied before any prefix row is matched, so
the manager cannot take a conjunct of it the way pwt_split_table_cond()
takes one of a table condition.

A hashed buffer (BNLH) keys the buffered rows by the tab's ref, which
for a hash join is keyed by a KEY create_hj_key_for_table() built for
the manager's table, with key parts naming the manager's fields.
pwt_clone_hj_key() makes the same descriptor over the worker's fields,
clone_table_ref() takes the KEY rather than deriving it from the table,
and the gate and the inert assertions admit JT_HASH.

BKA and BKAH are not run: they read the inner table through multi-range
read on the handler. The gate refuses them, and so that the costing never
chooses a plan the gate refuses, check_join_cache_usage_for_tables()
caps a plan chosen for the workers at join_cache_level 4, where a level
that serially means BKA means BNLH. best_access_path() prices a ref
access the same with or without a BKA buffer, so the plan runs in the
shape it was costed in either way.

The join-order search costs the tables under a parallel driver with join
buffers and hash joins (parallel_chain in best_access_path() now
withholds only the rowid filter).

The worker's hand-built JOIN gets the manager's tables_list, read only:
next_linear_tab() ends its walk at exec_join_tab_cnt(), which is zero
for a JOIN without one, and JOIN_CACHE::calc_record_fields() walked past
the end of the worker's array.

ANALYZE: the worker gets its own three buffer counters, summed into the
manager's tab trackers by quiesce_workers() like the table counters.
Exec_time_tracker gains add() and reset_counts() for that.

Measured on the cross join above with four workers: ANALYZE shows s2
scanned twice, once per worker that got rows, where the stripped worker
scanned it a thousand times. The join is no longer the cost of that
query in the workers; what remains is every joined row being written to
the worker's sink and drained by the manager, which the costing does not
price and which no buffer can help. TPC-H SF1 plans and times (Q3, Q5,
Q10, Q12) are as before this commit.

Tests. main.parallel_query_join: the BNL case now runs in the workers,
plus a BNLH case, the level-6 case (BKA serially, BNLH in the workers)
and the cross join above. main.parallel_query_why: a buffered inner
table is no longer a reason. main.parallel_query_costing gains a small
table scanned under the divided driver, which takes its BNL buffer in
the serial and the parallel plan alike and runs in the workers.

This commit was prepared with Claude Code (Fable 5.1), which traced what
a JOIN_CACHE reads from its JOIN and tabs, wrote the worker-side cache
construction, condition and hash-key cloning, the gate and costing
changes and the tracker merge, updated the tests, rebuilt the worktree,
ran the parallel_query tests in both protocols and the main suite, and
timed the cross join and TPC-H on a scratch server.

a11068f0e09a...

a11068f0e09ab328a20d374e8128a0c87c27d5d5

Jan Lindström

jan.lindstrom@mariadb.com

Merge branch '4.x-MDEV-42139' into 4.ee-MDEV-42139

a0640b0094ef...

a0640b0094ef568b57f1b47016ad7c019311f73a

Jan Lindström

jan.lindstrom@mariadb.com

Merge remote-tracking branch 'upstream/4.ee' into galera-26.4.29

9fbc3638203c...

9fbc3638203ca36adccc3852e991fdb99d981655

Rex Johnston

rex.johnston@mariadb.com

MDEV-42165 Selectivity of a multiple equality with an earlier table is never applied

During optimization, when a table is joined onto the tables already in the
plan, table_multi_eq_cond_selectivity() is meant to account for the
multiple equalities tying its columns to theirs that the chosen access
method does not already use: for each such equality, the largest
avg_frequency/rows over its fields on the earlier tables, multiplied
together across equalities.

It never did. The accumulator starts at 1.0 and is only ever raised:

double eq_fld_sel= 1.0;
...
if (curr_eq_fld_sel < 1.0)
set_if_bigger(eq_fld_sel, curr_eq_fld_sel);

Every candidate is below 1.0, so eq_fld_sel stays 1.0 and the function
returns 1.0 for every table in every plan. This has been so since the
function was added (fc1c8ffdadf, the MWL#253 pilot patch).

The effect is visible whenever a table is joined on a column without an
index, so that the join runs through a join buffer: the rows the join
condition rejects are counted as surviving, and every table joined after
it is costed for that many loops. In the test, one of 20 dimension rows
survives its filter and one twentieth of the fact rows joins it, but the
fact table is estimated at filtered 100 and the lookup table under it at
2000 loops where 100 happen. TPC-DS Q93 shows the same shape at scale:
store_returns joined to reason on an unindexed sr_reason_sk, 287514
store_sales lookups estimated where 3710 happen, and a serial plan costed
at 360 that runs in 43 ms.

Make the accumulator do what the comment describes: the largest selectivity
among the fields of the earlier tables, 1.0 only when there is none. The
fact table is then estimated at filtered 5 and the lookup table at 100
loops.

Two things the no-op had been hiding had to change with it.

A column without statistics has an average frequency of 0, which the
accumulator would have taken as "no row matches" and the plan as a
zero-row join. Such a field says nothing and is skipped.

table_after_join_selectivity() multiplied the factor into pos->records_out,
the smallest row count any access method considered for the table had
estimated. That estimate often comes from a ref access on the very
equality the factor accounts for -- a scan whose row count was capped by
the estimate of a ref on the join column, or a ref on one key capped by a
ref on another -- so multiplying counted the equality twice: supplier
after nation in the dbt3 Q9 plan came out at 0.44% where 4% is right. The
factor is now applied to the access method's own estimate (records_init
with the ref discounts for a ref access, records_read for a scan) and the
smaller of that and pos->records_out is kept. This also makes the function
idempotent, which optimize_wo_join_buffering() relies on when it re-costs
a semi-join range from positions that already carry the factor.

Eleven result files change: filtered values where an equality with an
earlier table now counts, and a few join orders in tests on tiny tables
where the estimates moved.

This commit was prepared with Claude Code (Fable 5.1), which found the
no-op while tracing why the parallel plan for Q93 was preferred, confirmed
it with the test here, found the two consequences above from the main
suite's result changes, and wrote the fix, the result updates and this
message.

97318c74b0dd...

97318c74b0dd190101c387ddf995ace6841d4880

Aleksey Midenkov

midenok@gmail.com

MDEV-41849 Killed query reports ER_NOT_FORM_FILE for a table with virtual column expressions

Since MDEV-37938 the lexer stops on kill: it sends the kill message and
returns END_OF_INPUT. parse_sql() fails only if the grammar rejects the
text read so far.

When a query is killed while a table is opened, the parse of a vcol
expression or partitioning clause from the .frm fails, and
open_table_from_share() reports ER_NOT_FORM_FILE after the kill error,
as if the .frm were corrupted. The grammar also adds ER_EMPTY_QUERY or
ER_PARSE_ERROR. A killed parallel slave worker shows these errors in
Last_SQL_Error. Table discovery replaces the kill error with
ER_SQL_DISCOVER_ERROR. The event scheduler logs an error during
compilation of the event. A kill while trigger bodies are parsed marks
the triggers as broken, and DML on the table fails until FLUSH TABLES.

If the kill comes after a part that is valid on its own (a in a + 1),
the parse succeeds and the cut-short text is used. This applies to any
text parsed by parse_sql().

The fix makes parse_sql() fail when the lexer stopped on kill and
reports only the kill error: the grammar adds no errors,
open_table_from_share() does not report ER_NOT_FORM_FILE, discovery
keeps the kill error, the event scheduler logs only the kill error, and
triggers are not marked as broken.

Lex_input_stream gets ignore_kill, stopped_on_kill and
fail_if_stopped_on_kill(), used in this order:

1. The caller sets ignore_kill before the parse.
2. The lexer checks ignore_kill. If it is not set and the thread is
killed with an error, the lexer raises the kill error, stops and
sets stopped_on_kill.
3. After the parse, fail_if_stopped_on_kill() reads stopped_on_kill
and fails the parse.

ignore_kill is set when triggers are loaded for their names only
(DROP TABLE, RENAME TABLE, I_S.TRIGGERS): they need every trigger name,
e.g. for the .TRN files, and a parse stopped by the kill loses the name.

A CTE referenced twice is parsed again by THD::sql_parser() inside the
outer parse. That parser takes ignore_kill from the outer one and passes
stopped_on_kill back to it, so the outer parse also fails on the kill.

ABORT_QUERY and KILL_BAD_DATA have no kill error, so the lexer does not
stop on them and reads the whole text, as before MDEV-37938.

Table_triggers_list::check_n_load() deletes the triggers loaded so far
on every error, which also fixes their leak on out-of-memory errors.

8dddf649cdc0...

8dddf649cdc0de97ffaf64343e2d2f4cb3281e1f

Teemu Ollakka

teemu.ollakka@mariadb.com

MDEV-38920 add gcomm unit tests for the EVS install timeout defect

Cover the install timeout ordering defect and the configuration
sanitizing which fixes it. Simulated time makes both tests
deterministic, and a run takes a fraction of a second.

test_mdev_38920_install_timer_expired (check_pc.cpp) builds a primary
component of five nodes with install_timeout below suspect_timeout.
Messages from node 5 to node 1 are then dropped for 60 seconds, after
which the link is restored. That asymmetry is what
asymmetry_elimination() exists to resolve, and an install timeout
below the suspect timeout disables it. A fully silent node does not
reproduce the defect, because check_inactive() evicts a suspected node
which has no join message at once. The fault is therefore injected as
one directional message loss rather than as a process freeze.

The verdict is that no node may log "install timer expired". The other
assertions record the expected outcome, but neither discriminates the
fix on its own, because a provider which does not sanitize the install
timeout still recovers from a fault of this length.

test_mdev_38920_config_warn (check_evs2.cpp) asserts that the
constructor clamps both install_timeout and inactive_check_period,
warns about each, and leaves a conforming configuration alone.

Both verdicts rest on what the provider logged, so add
gcomm::LogCapture to check_trace. It collects log messages while
started, and raises the log level to info when lower, so that a test
which asserts that a message was absent cannot pass merely because the
message was filtered out.

create_dummy_node() gains an optional install timeout. It defaults to
the inactive timeout, as before.

864758e24e42...

864758e24e420cac8edfb077e00f5b145d1bfe57

Teemu Ollakka

teemu.ollakka@mariadb.com

MGL-337 Bounds-check GCS state, UUID, vote and action messages

gcs_state_msg_read() parsed a state message received from the network
without checking its length in release builds. The fixed header was
read without a length check, name and inc_addr were measured with
strlen(), and the version-dependent fields were guarded only by
assert(). A truncated or unterminated state message from a cluster
member caused an out-of-bounds read and could crash the node.

Track the read offset and check the message length before each field.
Find the ends of name and inc_addr with memchr() within the buffer.
Form field pointers only within the message. Messages of version 5 and
later always contain the v5 fields, also when gcs protocol is below 2,
so require them. Return NULL for a malformed message. The caller
already logs and ignores a message which cannot be parsed.

Other GCS message handlers had the same missing bounds checks:

- gcs_group_handle_uuid_msg() checked the message size only with
assert() and read a full UUID from a shorter message. Drop a state
UUID message of a wrong size.
- gcs_group_handle_vote_msg() logged the optional vote message text as
a C string, without a NUL terminator check within the message, and
computed a pointer past the code message for shorter messages. Build
the text only for failed votes and limit it to the message size.
- gcs_act_proto_read() read the protocol version byte before the
message length check. Check the length first, and set proto_ver to
-1 for a message shorter than the header.

Add gcs_state_msg_test_truncated, which places truncated and
unterminated messages before a PROT_NONE page so that any read past
the end of the message faults. The buffer is 8-byte aligned, as
gcs_state_msg_read() requires. Run it with gcs protocol 1 and later.
Move GuardedBuf to gcs_test_utils and add a test for each handler.
Extend the vote message tests to check the recorded vote, the logged
text, a vote without text, and the size check before gcs protocol 1.

Assisted-by: Claude:claude-opus-5-5

7d2aee2bf069...

7d2aee2bf069e4de61947c0af9af4d687511cc8a

Jan Lindström

jan.lindstrom@mariadb.com

Bump Galera version to 26.4.29

76d951e3a37c...

76d951e3a37c455b61780dbd0767268cca7ee383

Alexey Yurchenko

alexey.yurchenko@mariadb.com

Merge pull request #1135 from mariadb-corporation/MDEV-38920-tests

MDEV-38920 add gcomm unit tests for the EVS install timeout defect

75b767433fa2...

75b767433fa2cb08a12bc63a2b50d8f1dd3c5ee9

Alessandro Vetere

iminelink@gmail.com

MDEV-21725 Log page reorganization as the ranges by which records moved

btr_page_reorganize_low() logged the payload as one WRITE from the first
to the last changed byte. When the reorganization removes a gap near the
start of the page, it moves all the later records, and this WRITE covers
almost the whole page.

btr_page_reorganize_log_payload(): Log the payload with MEMMOVE for the
runs of records that only moved, and WRITE for the rest. The function
first copies the runs that moved towards the end of the page, in
descending order of destination. Then it copies the runs that moved
towards the start of the page, in ascending order of destination. In
each order, no copy changes the source of a later copy. If a copy of the
first order changed a part of a source, the function does not copy that
part. Instead, it logs with WRITE the bytes at the destination of that
part that differ from the old page. The function does not copy a run
that is shorter than 8 bytes. It also does not copy the shortest runs
that moved towards the end of the page when the working memory is full.
It logs the changed bytes of these runs with WRITE. The log format does
not change.

btr_page_reorganize_low(): Reserve a second scratch frame for the
working memory of btr_page_reorganize_log_payload().

page_copy_rec_list_end_no_locks(): Add the output parameter extra.

In all versions since the 10.5 redo log format, the recovery parser has
an error. For a same_page record that follows a MEMSET or MEMMOVE
record, it calculates an offset that is too low. It can then reject a
valid MEMMOVE whose source is below its destination as "Malformed log
record". Older servers and backup tools must still recover the log that
the new payload logging writes.

mtr_t::memmove(): Write the page identifier when the parser would
calculate a source below 8 (FIL_PAGE_PREV), which it rejects.
mtr_t::m_last_lag is the difference between the correct offset and the
offset that the parser calculates.

mtr_t::memset(), mtr_t::log_write(): Update mtr_t::m_last_lag.

recv_sys_t::parse_tail(): Add a FIXME for the parser error.

No new test: innodb.alter_copy and innodb.instant_alter_crash recover
page reorganizations. Without the mtr_t::memmove() change,
innodb.alter_copy, innodb.autoinc_persist, innodb.innodb-alter-tempfile,
innodb_fts.crash_recovery and innodb_zip.wl5522_debug_zip fail in
recovery.

innodb.index_lock_upgrade,on with 4k pages (Debug) writes 2,815,439
bytes of redo log instead of 3,293,565 (shutdown LSN minus startup LSN).

Assisted-by: Claude:claude-opus-5-5

7589f40db8c4...

7589f40db8c4251e04a231385337fcc80b22cb0b

Vladislav Vaintroub

vvaintroub@gmail.com

69a0ae27e1d6...

69a0ae27e1d68b7e0b73c85af8277e152bd458af

Thirunarayanan Balathandayuthapani

thiru@mariadb.com

MDEV-39061 mariadb-backup compatible wrapper for BACKUP SERVER

This adds a shell script that lets users keep using their existing
mariadb-backup commands while the real work is done by the new
server-side BACKUP SERVER command. The goal is "drop-in": users
should not have to change their backup scripts.

mariadb-backup-server.sh : Understands the usual mariadb-backup
modes and translates each one.

mbstream-server.sh : lets streamed backups be unpacked by
pipelines that expect the mbstream CLI

mariadb-backup-pipe.sh is the stream sink that
BACKUP SERVER itself runs.

All three are documented in README.md.

--backup
========
Connects with the mariadb client and runs
"BACKUP SERVER TO '<dir>'".

Connection options (--user, --host, --port, --socket,
--defaults-file, ssl, ...) are passed through to the client.

--parallel=N becomes "<N+1> CONCURRENT": mariadb-backup
runs a dedicated log_copying_thread() besides its N data-copy
threads, while CONCURRENT counts every BACKUP SERVER worker.
The result is clamped to the 1..256 the parser accepts,
and to a minimum of 2.

After the backup it writes backup-prepare.cnf into the backup
directory, recording what --prepare needs later:
- where mariadbd lives
- InnoDB parameters (page size, data file path, undo tablespaces,
checksum algorithm, log file size)
- If the server is encrypted then how to reload the
encryption key plugin (the file_key_management variables),
so an encrypted backup can be prepared without extra input.
- every active provider_* compression plugin, with plugin-dir.

The providers matter because --prepare runs "mariadbd --bootstrap
--defaults-file=backup-prepare.cnf", which suppresses the
/etc/my.cnf.d snippet that normally enables a provider. Recovery has
to decompress a PAGE_COMPRESSED page before it can apply redo to it,
so without the provider it aborts with "Unable to apply log to
corrupted page". mariadb-backup carries the same plugins through
backup-my.cnf (encryption_plugin.cc QUERY_PLUGIN collects the
ENCRYPTION plugin and the provider_* DAEMON plugins together), so
this is parity with it rather than a new capability.

--backup --stream
=================
Runs "BACKUP SERVER WITH 'pipe'".
The server only accepts a bare command name, which it resolves
n the PATH of the mariadbd process after prepending mariadb-backup-,
so uses the installed mariadb-backup-pipe instead.

mariadb-backup-pipe writes the tar into
.mariadb-backup-pipe-<stream>.tar, a named pipe the wrapper
created in the server @@datadir and is already draining to
its own stdout.

The backup therefore reaches the consumer as it is produced and
never lands on local disk. The wrapper appends backup-prepare.cnf
as a final tar afterwards; the server's tar carries no
end-of-archive marker, so that trailing archive supplies the
only one and the whole stream extracts with a plain "tar -x".
The trailing archive is written with "tar -b 1" so that nothing
follows its end-of-archive marker. At the default blocking factor tar
pads the archive to 10240 bytes; the reader stops at the marker and
exits with the padding still unwritten, which on a small pipe is a
short write. tar treats a short write as fatal, so a backup that had
in fact completed was reported as failed.

--parallel is ignored here, with a warning.

Four properties follow from how BACKUP SERVER streams,
all differing from mariadb-backup:
- local: the stream command runs inside the server,
so the wrapper must share its filesystem;
- fifo lives in @@datadir, because mariadb-backup-pipe resolves it
relative to the working directory of mariadbd, so --stream needs
write access there;
- tar only: any --stream=<format> yields tar;
- single-threaded: one worker, so no parallel read either.
--target-dir is optional in stream mode;

mbstream-server.sh maps the mbstream CLI onto a plain
"tar -x"/"tar -c", so existing "mbstream -x"/"-c" pipelines
keep working on the wrapper's stream.

mbstream-only flags (-p/--parallel, ...) are accepted and
ignored; any other unknown option is rejected.

Environment overrides:
MARIADB (client),
MARIADBD (the --prepare bootstrap server) and
TAR (the tar implementation, e.g. TAR=bsdtar) can each be overridden.

To run the bootstrap under rr, put it in MARIADBD and
let rr's own _RR_TRACE_DIR choose the trace location, e.g.
_RR_TRACE_DIR=/dev/shm/rr MARIADBD='rr record mariadbd'

--prepare
=========
Starts "mariadbd --bootstrap" on the backup directory using
backup-prepare.cnf as its defaults file, replays the archived redo
log between the start and target LSN read from backup.cnf,
then builds a fresh ib_logfile0 so a normal server can start
on the directory.

mariadbd is taken from the path recorded in backup-prepare.cnf
if that binary exists, otherwise by searching
/libexec, /sbin, /bin and the configured install directories.

PATH is not searched; set MARIADBD to point elsewhere.
User --defaults-file/-extra-file and encryption options are
layered onto the bootstrap.

--copy-back / --move-back
=========================
Copy or move a prepared backup into the datadir. The datadir
is created if missing, a non-empty datadir is refused unless
--force-non-empty-directories is given, and a chown
reminder is printed.

If --aria-log-dir-path is given, the Aria logs (aria_log_control,
aria_log.*) are relocated into that directory.

Packaging
=========
The wrapper is not installed by default and never replaces the
real mariadb-backup / mbstream binaries.
1. cmake -DWITH_MARIABACKUP_WRAPPER=ON (default OFF) controls it.
2. When ON, the scripts install as /usr/bin/mariadb-backup-server,
/usr/bin/mbstream-server and /usr/bin/mariadb-backup-pipe, tagged
COMPONENT Backup so they ship in the mariadb-backup package.
mariadb-backup-pipe must end up in the PATH of the mariadbd
process, not merely in the PATH of whoever runs the wrapper.
mariadb-backup-pipe is generated unconditionally, because the
mariabackup test suite needs it on the server's PATH even when the
wrapper is not installed; only its install rule is gated.
3. RPM: nothing extra to do. the component handles it.
4. DEB: not wired. debian/rules uses --fail-missing and does not
enable the option, so the -server binaries are not listed.
To ship via DEB, make a paired change: add
-DWITH_MARIABACKUP_WRAPPER=ON in debian/rules and list all three of
usr/bin/mariadb-backup-server, usr/bin/mbstream-server and
usr/bin/mariadb-backup-pipe in debian/mariadb-backup.install together.
5. The real mariadb-backup/mbstream binaries and the
mariabackup symlink are left untouched; opt in via an alias or a
symlink early in PATH.

Limitations (not supported yet)
===============================
1) Incremental backup & prepare (--incremental-basedir,
--incremental-dir, --apply-log-only)
2) --rollback-xa
3) Partial backup (--databases, --tables, --tables-file)
4) Output compression and encryption (--compress, --encrypt)
5) --export is accepted but only warns and runs a plain recovery
6) --extra-lsndir is ignored
7) --parallel is ignored with --stream
8) Windows: POSIX sh only, not installed on Windows

Behaviour differences from native mariadb-backup
================================================
- The wrapper needs the mariadb client on PATH for --backup;
--prepare needs mariadbd recorded in backup-prepare.cnf, in a
standard install directory, or named by MARIADBD
- BACKUP SERVER refuses an already-existing target directory
- BACKUP SERVER does copy the data file as raw pages without
checksum validation, so a corrupted table is not detected
at backup time
- --prepare only works on a wrapper-made backup: it
needs backup-prepare.cnf
- --stream is tar, not xbstream, local-only and single-threaded,
and needs write access to @@datadir for the fifo

Tests
=====
include/have_mariabackup_wrapper.inc redirects $XTRABACKUP to
mariadb-backup-server.sh and $XBSTREAM to mbstream-server.sh,
skipping when a wrapper or the mariadb client is unavailable.

include/have_mariabackup_combination.inc runs a test under both the
[CLIENT] mariadb-backup binary and the [SERVER] wrapper.

suite/mariabackup/suite.pm prepends the directory holding
mariadb-backup-pipe to PATH. BACKUP SERVER resolves the command in the
PATH of the mariadbd process, which mtr starts from the suite's
environment, so it has to be set there rather than with a --let at
--exec time. have_mariabackup_wrapper.inc skips the [SERVER]
combination when the helper is not on PATH.

658c121dee7a...

658c121dee7a92973f1936052425a6bc5480dc2f

Alexey Yurchenko

alexey.yurchenko@mariadb.com

Merge pull request #1147 from mariadb-corporation/4.ee-MDEV-42140

4.ee mdev 42140

5eeb7ec0e802...

5eeb7ec0e802ea7855d27e80eaeec0bee4064f02

Aleksey Midenkov

midenok@gmail.com

5bdf4ce7ed88...

5bdf4ce7ed88c21e3e400ef054d0ee9262359730

Jan Lindström

jan.lindstrom@mariadb.com

MDEV-42139 fix spurious wakeup in gu_debug_sync_wait()

gu_debug_sync_wait() blocked on a single, unlooped condition variable
wait. pthread_cond_wait() may return without any matching signal (a
spurious wakeup), and nothing here re-checked the real predicate, so a
spurious wakeup was treated the same as a real gu_debug_sync_signal()
call. Every other condition variable wait in galerautils (gu_to.c,
gu_fifo.c) already loops on its predicate; this was the one place that
did not.

In galera_3nodes.galera_join_with_cc_B this let a waiter on the
"process_primary_configuration" sync point disappear from
wsrep_debug_sync_waiters between a wait_condition.inc poll and the
test's own follow-up SELECT, producing an empty result instead of the
point name.

Add a per-waiter 'signalled' flag and re-check it in a loop around the
wait call. Add galerautils/tests/gu_debug_sync_test.cpp, which
reproduces the defect deterministically via a test-only hook that
injects a simulated spurious wakeup, and wire GALERA_GU_DBUG_ON through
to galerautilsxx so the test can build.

Assisted-by: Claude:claude-4.6-sonnet

592779a0b277...

592779a0b27716f7b799a6fe506cadb8c5019d3c

Jan Lindström

jan.lindstrom@mariadb.com

MDEV-42000 Interrupt IST receiver's SST wait on shutdown

ist::Receiver::run() waits for ready() to be called once SST/IST
negotiation completes. If the server starts closing before that
happens, nothing signals the wait, and the receiver thread, and
ReplicatorSMM::close() with it, block forever.

Add an interrupted_ flag the wait loop also checks, and an interrupt_wait()
method that sets it and signals the condition variable safely from any
thread. Call it from start_closing() and from finished(). Add
test_ist_interrupted_before_ready_v7 as a regression test.

Assisted-by: Claude:claude-4.6-sonnet

54d860e94e9b...

54d860e94e9bef188df8de6eab50119217654304

Oleksandr Byelkin

sanja@mariadb.com

fix main.mdev_31342_kill with view and cursor protocol

51bed9174c44...

51bed9174c44d478cb3811b58f60362662f45743

Rex Johnston

rex.johnston@mariadb.com

MDEV-39492 PQ: choose the divided driving table as a plan, not by its scan alone

TPC-DS Q29 ran at twice its serial time with eight workers. Serially the
optimizer starts at date_dim d2, whose WHERE keeps 122 of 73049 rows,
and reaches store_returns through the returned-date index: 16700
lookups into store_sales. With workers it started at store_returns
instead, divided its 287514 rows eight ways, and applied the d2 filter
only after the store_sales lookup, so each worker did 36000 of those
lookups. Dividing d2 and keeping the serial order under it costs 24
against the chosen plan's 58 and the serial plan's 61, and the search
never saw it.

best_access_path() priced the driving table's scan both ways and kept
the cheaper reading on the spot. For d2 the parallel reading is the
dearer: one eighth of a 12.5 scan plus 2 ms of setup for each of eight
tables comes to 17.6, so the serial reading was kept and the plan with
d2 divided was never extended. What that comparison leaves out is the
join under the driving table, which the division cuts eight ways too
and which is where the cost is. The scan of a table like d2 can never
repay the setup on its own; the join under it repays it many times
over.

The search now offers the driving position every candidate twice. After
get_costs_for_tables() has costed each table with its scan read
serially, add_parallel_driver_candidates() costs it again with
JOIN::parallel_driver_costing set, under which best_access_path()
prices a full or range scan as one worker's share plus the setup
(parallel_scan_cost() no longer judges the reading), and appends the
result as a further candidate when the parallel scan is what
best_access_path() chose. Both readings are extended, and the complete
plans compete on cost. optimize_straight_join() costs the fixed order
twice in the same way and keeps the cheaper. The sort space in
make_join_statistics() grows by two POSITIONs per table for the extra
candidates.

Two more things had to change for the search to be honest about it.

The pruning heuristic keeps parallel prefixes apart from serial ones so
that a divided big table is not dropped next to a tiny driver. With
every table now offered as a parallel candidate, the tiny driver's
parallel reading landed in the parallel family too -- one row, next to
no cost -- and dropped every prefix that does spread its rows, which
is what the family exists to protect. A candidate whose divisor is 1
spreads nothing over the workers and is compared with the serial
prefixes, where it loses to its serial twin by the setup it is charged.

And the divisor itself credited a table with a division the workers
cannot make. With the local comparison gone, TPC-H Q5 chose NATION as
its driving table: 25 rows, 5 surviving, costed as spreading the whole
join under it over four workers at cost 230 against the ORDERS-first
plan's 297. NATION is a single InnoDB page, which the partitioner
cannot cut, so one worker ran the whole join at serial speed. The
handler now says how many pieces a scan of its table can be cut into
(parallel_scan_pieces(): for InnoDB the leaf pages of the clustered
index, which bounds the chunks the partitioner can make), scaled by
the share of the table a range reads, and the rows that survive the
driving table's WHERE are spread over no more workers than that. They
are also spread as the busiest worker sees them: R rows over W workers
put ceil(R/W) on one of them, so the divisor is R / ceil(R/W), which
makes five rows over four workers a division by 2.5 and one row a
division by 1. The scan's own cost is still divided by N: a scan the
engine cannot cut is at most a couple of pages, worth nothing next to
the setup, and dividing it is what lets the parallel-query tests, whose
tables are that small, run the workers with the setup cost at 0.

Measured on TPC-DS SF1 with eight workers, Q29: serial 34-46 ms, before
78-90 ms with store_returns divided, after 33-35 ms with d2 divided.
The two workers that receive the 122 qualifying dates, contiguous in
the primary key, do most of the join, so the gain over serial is
small; the loss is gone. Q93 and Q32 stay serial, as they should. On
TPC-H SF1 with four workers the plans of Q1, Q3, Q5, Q7, Q10, Q11 and
Q12 and their times are as the costing commit measured them, Q5 at
165 ms with ORDERS divided.

main.parallel_query_costing gains a 2000-row dimension with 200 rows
matching and 100 fact rows under each: its scan alone cannot repay the
default setup, and it drives the join divided all the same, with and
without STRAIGHT_JOIN.

This commit was prepared with Claude Code (Fable 5.1), which found the
local comparison in the optimizer trace of Q29, implemented the second
candidate, the divisor-keyed pruning family, the pieces estimate and
the busiest-worker divisor, found the Q5 regression and the pruning
regression by measuring TPC-H and re-running the costing test, wrote
the new test section, and ran the parallel-query tests in both
protocols and the main suite.

3e696bb53463...

3e696bb53463ec36d7dc9986f8273c173d97686e

Rex Johnston

rex.johnston@mariadb.com

MDEV-39492 PQ: a plan chosen for the workers must reach them, through
the split-materialized reoptimization and a reduced GROUP BY

TPC-DS Q68 with eight workers chose, for its grouped derived table, the
plan with store_sales divided among the workers and every other table
looked up by eq_ref, and then ran that plan serially: a full scan of
2,880,404 rows with four lookups each, 1.7 s against 0.3 s for the
serial plan that starts from the 72 matching dates. Two gate defects,
each enough on its own, turned the chosen parallel plan into a serial
one in the shape the workers would have run.

The first is in the post-plan check that every select-list column
outside an aggregate is a group column. The query groups by
ss_ticket_number, ss_customer_sk, ss_addr_sk and ca_city, and selects
ca_city. The plan reads customer_address by eq_ref on ss_addr_sk, a
grouped column, so remove_const() drops ca_city from the plan's GROUP
BY (eq_ref_table(): every row of a group reads the one same
customer_address row), and the aggregation table is keyed by what is
left. The check, asked with that key, found ca_city unnamed and
refused. And the pre-plan form of the same check, which is asked again
by the full gate, read the query's GROUP BY from join->group_list,
which remove_const() had reduced in place -- relinking the select's
own ORDER nodes, so select_lex->group_list was reduced with it -- and
refused for the same reason.

Pwt_group_checker now also accepts a column of a table the plan reads
by eq_ref through columns that are themselves grouped, constant or
determined the same way, which is remove_const()'s own reasoning and
leaves the column with one value per group; it is asked only of a
finished plan, with the plan to read. And pwt_query_shape_allows_workers()
copies the query's GROUP BY the first time it is asked, before the plan
(choose_plan()), into JOIN::parallel_query_group, and answers from the
copy afterwards.

The second is in what survives a reoptimization. The outer query joins
the derived table by a key on ss_customer_sk, so opt_split.cc tries
splitting it: it saves the plan, runs choose_plan() again with the outer
equality pushed in, and restores the saved plan when the split loses, as
it does here with 100,000 customers to run it for. choose_plan() sets
JOIN::parallel_plan_chosen for the plan it finds, so the restored plan
-- the divided one -- came back with the flag the losing split had left,
false, and parallel_join_check() declined it as "costed cheaper read
serially". The flag is a property of the plan and now travels with it:
Join_plan_state carries it, save_query_plan() takes it and
restore_query_plan() puts it back.

With both, the derived table runs in the workers: 0.76 s with eight,
Parallel_queries_executed counting it. That is still slower than the
0.30 s serial plan, which the costing values at 514 against the divided
plan's 395. The gap is not in the gate. The forced store_sales-first
plan takes 1.61 s serially, 1.10 s with two workers, 0.78 s with four,
0.70 s with eight and 0.73 s with twelve, and ANALYZE puts the time in
the 2,880,404 eq_ref lookups of date_dim: 0.35 us each serially, 1.6 us
each in a worker when eight of them look up the same 73,049-row table
at once, while the scan of store_sales divides cleanly. The costing
divides lookups by the number of workers as it divides the scan, and
for lookups into a small hot table the engine does not. That is a
separate matter, left for its own change.

Tests. main.parallel_query_why: a GROUP BY column of a table read by
eq_ref through a grouped column, dropped from the plan's key, no longer
a reason. main.parallel_query_costing: a grouped derived table joined by
a key on its group column from an outer query too wide for the split to
win, which runs in the workers.

This commit was prepared with Claude Code (Fable 5.1), which traced the
two refusals on a copy of the TPC-DS data, found the reduced GROUP BY
and the flag lost across the restored plan, wrote the fixes and the
tests, rebuilt the worktree, measured the forced plan's scaling by
worker count, and ran the parallel_query tests in both protocols and
the main suite.

326ef9aaff28...

326ef9aaff2892238dd0c6cf5d0e1064d1338f59

Sergei Petrunia

sergey@mariadb.com

MDEV-42044: I_S.OPTIMIZER_CONTEXT exposes rows, DDL and statistics ...

Optimizer Context has DDLs of used tables and VIEWs, table statistics,
and even captured table rows which include all, even the columns that
the query doesn't read.

The user running the query might not have access to some of this
information. Making it available in the optimizer context would create
an opportunity to bypass the security controls.

There is a similar issue in the Optimizer Trace. It is solved by not
producing optimizer trace if it would expose data that's not available
otherwise.

Piggy-back on the Optimizer Trace security checks and do the same:
disallow saving Optimizer Context if it can contain information that
the caller would not have access to.

In addition to Optimizer Trace's check, add
opt_ctx_can_read_all_columns() and check that when we're about to
dump a full table row into the optimizer context.

2ec2e7d11023...

2ec2e7d1102318353ddf4dcfa01a59dbbb092629

bsrikanth-mariadb

srikanth.bondalapati@mariadb.com

MDEV-42044: Fix Optimizer Context privilege checks

Follow-up to the previous MDEV-42044 commit:

* opt_trace_disable_if_no_security_context_access() returned early when
the optimizer trace was not started, so a context recorded through
DEFINER routines, triggers or views was never marked as restricted.
Mark the recorder before that return.
* Check that the invoker can read every table whose DDL and statistics
are dumped, including tables locked only for stored functions and
triggers. Previously only dumped rows were checked.
* Skip the access checks when built with NO_EMBEDDED_ACCESS_CHECKS.
* Factor the repeated "missing privilege" code in opt_trace.cc into
helpers.

Tests: assert the "Insufficient privileges" output, and add cases for
column-level grants, an unexecuted function, a DEFINER trigger,
optimizer_trace=ON, and a privileged user who still gets a full context.

21bd8f3e1840...

21bd8f3e1840dbdc8033d1dd748205bca0f85359

Aleksey Midenkov

midenok@gmail.com