Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Arcadiy Ivanov
fixup! Limit the memory used by GROUP_CONCAT() with ORDER BY

`main.gconcat_warn` case 5 recorded `Row 88 was cut by group_concat()`
and reads `Row 127` on a 32-bit build, so the test fails on
`x86-debian-12-fulltest` and `x86-debian-12-fulltest-debug`.

The case starves the sort tree until a repack drops rows, and the row
the warning names is how many rows the budget held: the bytes the
repack copies to, divided by `sizeof(TREE_ELEMENT)` plus the size of
an element. `TREE_ELEMENT` is 24 bytes where a pointer is eight and 12
where it is four. The two recorded numbers agree with that. The case
comment gives the repack 2720 bytes, and 2720 over 88 is about 31
bytes a row against 2720 over 127 for about 21, a difference of the
order the smaller `TREE_ELEMENT` accounts for.

The test already says this, and already masks the result it prints for
the same reason:

    The exact result depends on sizeof(TREE_ELEMENT) and the
    reclength, so only check that the result came out short.

The warning is printed beside that result and was not masked. Mask the
row number with `--replace_regex`, as `main.gconcat_distinct_spill`
masks the same warning for the same reason. `SHOW COUNT(*) WARNINGS`
still checks that the group gave exactly one warning, which is what
the case is there to show.

Only case 5 needs it. The other cut warnings in this test come from
`group_concat_max_len` on a handful of rows, and their row numbers do
not depend on the width of a pointer.
Jan Lindström
Merge remote-tracking branch 'origin/MDEV-40281' into 4.ee-MDEV-40281
Rex Johnston
Parallel Query: count a worker in under the lock that counts it out

A query could hang for good, with the manager waiting on COND_data_avail for a
worker that had already finished and gone.

active_workers is decremented in report_worker_final_state() under LOCK_data,
and read in locked__process_manager_wakeup() with LOCK_data held, but
register_worker() was incrementing it under no lock at all. The two do run at
the same time: init_parallel_workers() builds the team one worker at a time and
starts each thread as it creates it, so worker 1 can be counting itself out
while worker 2 is being counted in.

  manager thread                      worker 1
  ------------------------------      ---------------------------------
  new pwt_worker  -> reads 2
                                      thread_func_end()
                                        LOCK_data, reads 2, writes 1
  writes 3

The decrement is lost and the count stands one too high. From then on
claim_next_result() waits for a worker that will never publish a result and
never signal again, because every worker there was has already exited.

Taking LOCK_data around the increment is the whole fix.

It wants a small table and enough load: a table the engine divides into one
chunk leaves the other workers with nothing to scan, so they reach
thread_func_end() almost at once, in the window while the rest of the team is
still being built. main.function_defaults_innodb is such a test, and this
reproduced it within six attempts:

  ./mtr --parallel=16 --mem --force --ps \
        --mysqld=--parallel-worker-threads=4 \
        main.function_defaults_innodb main.function_defaults_innodb \
        main.function_defaults_innodb main.function_defaults_innodb \
        main.function_defaults_innodb main.function_defaults_innodb

It now runs clean twelve times over. No test is added: a lost update needs a
race to be lost, and a test that only sometimes fails is worse than none.

This commit was prepared with Claude Code: it reproduced the hang, read the
manager's bookkeeping out of the stopped server with gdb -- active_workers 1
against four workers, all four with a destroyed THD and the statistics that
only thread_func_end() writes, so all four had decremented -- and reasoned back
from four decrements and a count of one to a lost increment.
Dmitry Shulga
MDEV-40076: ASAN global-buffer-overflow in reconstruct_create_trigger_stmt

Follow-up patch to address review comments found by reviewer
Alexander Barkov
MDEV-39518 Allow prepared statements in stored functions in assignment right hand

Allowing prepared statements in stored functions when
a stored function is used in an assignment right hand.

Both DEFAULT clause of a variable initialization and
the right side of the SET statement are supported:

  CREATE PROCEDURE p1()
  BEGIN
    -- case 1: DEFAULT clause
    DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK

    -- case 2: SP variable assignment statement
    DECLARE spvar2 INT;
    SET spvar2= f1_with_ps(); -- OK
  END;

- Only assignments to SP variables works for now:
  * SET spvar= func_with_ps(); -- OK
  * SET @uvar= func_with_ps(); -- Error

- Only bare function calls are supported for now. Using a function in
  an expression does not make it PS-safe yet:
    SET v= f1()+0;

- The parser now does not reject PS statements in stored functions.
  PS applicability in stored functions is now detected at run time.
  Note, PS statements in triggers are still prohibited by the parser.

- Functions with PS do not acquire MDL locks on tables, and no MDL is
  taken on the routines themselves either. They work like procedures in
  terms of table opening and routine locking: a concurrent DROP FUNCTION
  can complete while such a function is executing.

- Functions with PS are not replicated as a single `SELECT f1()` call.
  They are replicated per-statement, like procedures.

Helper changes:
- Changing the return result for LEX::sp_variable_declarations_init()
  from void to bool to catch errors in the caller properly.

Misc:
- This patch incorporates fixes for the following bugs found during debugging:
  MDEV-39518,MDEV-40224,MDEV-40225,MDEV-40226,MDEV-40227,MDEV-40240,
  MDEV-40285,MDEV-40288,MDEV-40315,MDEV-40318,MDEV-40890,MDEV-40900,
  MDEV-40901,MDEV-40913,MDEV-40914,MDEV-41013,MDEV-41015,MDEV-41019

Assisted-by: Claude - reviews and minor clean-ups
Sergei Petrunia
MDEV-40168: JSON-over-fulltext: let the estimate consult the FTS cache.

fts_estimate_word_docs() probed only the on-disk auxiliary INDEX_[1..6]
table.  Documents inserted but not SYNCed yet are only in the in-memory
FTS cache, so they were missed entirely, and on a table that has never
been SYNCed the auxiliary table is empty and the estimate was simply
"unknown".

Look in the cache as well.  fts_index_cache_t::words is an rb tree of
fts_tokenizer_word_t, and fts_node_t::doc_count already holds the number
of documents in the node's ilist, so this is one rbt_search plus a walk
over a short vector: no ilist decoding and no I/O.

Three details:

  - the cache mutex is taken with trylock.  This runs during
    optimization, where a SYNC holding cache->lock across SQL execution
    would stall the optimizer; dropping the cache contribution is the
    better trade.

  - nodes flagged fts_node_t::synced are skipped.  An in-flight SYNC has
    already written them out, and the estimator reads the B-tree without
    a read view, so it sees those records; counting the node too would
    count its documents twice.

  - index_cache->words is NULL between fts_cache_clear() and
    fts_cache_init().  The query path never observes that because it
    runs after fts_init_index(); the estimate does not run it.

DB_RECORD_NOT_FOUND now means both sources are empty.  The cache on its
own cannot prove a word absent, because it is only complete once
fts_init_index() has run, and an estimate must have no side effects.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Teemu Ollakka
Fix fragile branch name parsing
Teemu Ollakka
Merge remote-tracking branch 'origin/4.x-MDEV-36926' into 4.ee-MDEV-36926
Jan Lindström
Merge remote-tracking branch 'origin/4.x-26.4.27' into 4.ee
Alexander Barkov
MDEV-39518 Allow prepared statements in stored functions in assignment right hand

Allowing prepared statements in stored functions when
a stored function is used in an assignment right hand.

Both DEFAULT clause of a variable initialization and
the right side of the SET statement are supported:

  CREATE PROCEDURE p1()
  BEGIN
    -- case 1: DEFAULT clause
    DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK

    -- case 2: SP variable assignment statement
    DECLARE spvar2 INT;
    SET spvar2= f1_with_ps(); -- OK
  END;

- Only assignments to SP variables works for now:
  * SET spvar= func_with_ps(); -- OK
  * SET @uvar= func_with_ps(); -- Error

- Only bare function calls are supported for now. Using a function in
  an expression does not make it PS-safe yet:
    SET v= f1()+0;

- The parser now does not reject PS statements in stored functions.
  PS applicability in stored functions is now detected at run time.
  Note, PS statements in triggers are still prohibited by the parser.

- Functions with PS do not acquire MDL locks on tables, and no MDL is
  taken on the routines themselves either. They work like procedures in
  terms of table opening and routine locking: a concurrent DROP FUNCTION
  can complete while such a function is executing.

- Functions with PS are not replicated as a single `SELECT f1()` call.
  They are replicated per-statement, like procedures.

Helper changes:
- Changing the return result for LEX::sp_variable_declarations_init()
  from void to bool to catch errors in the caller properly.

Misc:
- This patch incorporates fixes for the following bugs found during debugging:
  MDEV-39518,MDEV-40224,MDEV-40225,MDEV-40226,MDEV-40227,MDEV-40240,
  MDEV-40285,MDEV-40288,MDEV-40315,MDEV-40318,MDEV-40890,MDEV-40900,
  MDEV-40901,MDEV-40913,MDEV-40914,MDEV-41013,MDEV-41015,MDEV-41019

Assisted-by: Claude - reviews and minor clean-ups
Sergei Petrunia
Estimate the number of records an MVI access will read

QUICK_MVI_SELECT carried records=10 and read_time=0.001, numbers picked
low enough that the access always won over a table scan. Ask the engine
instead, through the fulltext_estimate() added by the previous commit.

Mvi_access::estimate_records() estimates one element key at a time and
combines the answers the way the query combines the keys:

- A disjunctive access (JSON_OVERLAPS) reads the rows of every key, so
  the estimates add up.

- A conjunctive access (JSON_CONTAINS) reads the rows that have all of
  the keys, so the rarest key alone bounds the result. We use its
  estimate and drop the other keys from the query: reading the rarest
  key and letting the WHERE clause discard the rest is not worse than
  having the engine intersect the terms. This is the trade-off
  collect_mvi_keys() already makes for the keys it cannot encode - a
  shorter AND matches a superset of the rows, and the JSON predicate
  does the exact filtering.

The engine may be unable to estimate a key: ha_innobase only looks at
the fulltext auxiliary tables, so until the words are flushed out of the
FTS cache the answer is "unknown" for everything. Such a key takes no
part in the choice of the rarest one, and if not a single key could be
estimated the old guess stands and the query is left as it is.

For an OR we cannot do that: we have to read that key and have no idea
what it costs. Give the access a DBL_MAX read_time and do not use it.
That has to be acted on in get_best_mvi_access() rather than left to the
cost comparison, because best_access_path() takes a quick select to be
cheaper than a table scan without checking - true of anything the range
optimizer proposes, but this access does not come from there.

read_time is now the cost of reading the estimated rows plus evaluating
the WHERE clause on them, which is what the join optimizer expects of a
quick select's read_time. It does not account for the fulltext search
that produces the rowids in the first place.

The trace prints the estimate, or says the access is unusable and why.

The two existing tests ran on tables whose words were still in the FTS
cache, so the JSON_OVERLAPS sections would have stopped using the index
entirely. They now flush the cache with OPTIMIZE TABLE under
innodb_optimize_fulltext_only, the way innodb_fts.estimate does. The
trace test gets a table with a skewed distribution ("bbb" in every row,
"aaa" in one) to show the conjunctive access keeping only the rarest key
and still producing the same rows as the same query with IGNORE INDEX.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Thirunarayanan Balathandayuthapani
MDEV-26057 Assertion `!vcol->v_indexes.empty() in trx_undo_log_v_idx

Problem:
========
-  Rollback of an INPLACE ALTER TABLE is executed while holding only a
shared metadata lock on the table, so DML can run concurrently.
rollback_inplace_alter_table() resets dict_col_t::ord_part in a
critical section of its own, after row_merge_drop_indexes() already
removed the aborted indexes from the dictionary cache and emptied
dict_v_col_t::v_indexes. During this time, DML statement can see a
virtual column with ord_part set and an empty v_indexes, which
makes assert failure in trx_undo_report_insert_virtual().

Solution:
========
row_merge_reset_ord_part(): Added a function to reset
dict_col_t::ord_part for the columns that are no longer a field of
any index remaining in the dictionary cache.
For virtual columns the decision is based on dict_v_col_t::v_indexes
being empty, and no element is ever removed from that list.

row_merge_drop_indexes(): Added a call to row_merge_reset_ord_part()
in the branch that removes the indexes from the cache, in the same
dict_sys.latch critical section. That branch is taken only when
MDL_EXCLUSIVE is held or when this is the only handle to the table,
so no concurrent DML can observe the intermediate state.
In the lazy drop branch the indexes and their v_indexes entries
stay in the cache and nothing is reset; that is done later,
when the indexes are dropped while holding MDL_EXCLUSIVE.

check_col_exists_in_indexes(): Removed the only_committed parameter,
which no longer has any caller.

row_quiesce_col_ord_part(): Added a function to get
dict_col_t::ord_part and dict_col_t::max_prefix of a column
from the committed indexes that are
present in the dictionary cache.

row_quiesce_write_table(): Write the row_quiesce_col_ord_part() return
values to the .cfg file instead of the cached dict_col_t fields,
because a rolled back ADD INDEX leaves ord_part set until the
aborted index is removed by a later DDL, and
max_prefix is never reset when an index is dropped, which makes
IMPORT TABLESPACE reject the tablespace with a bogus schema mismatch.
Jan Lindström
MDEV-40281 Tolerate a concurrently closed socket in set_fd_options()

set_fd_options() threw when the socket had already been closed before the
connect or accept completion handler ran. The gu::Exception escaped the
handler and left the gcomm event loop, taking the whole backend down.
Return early instead and let the operation which follows report the error.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Teemu Ollakka
Merge branch '4.x-fix-ghprb-mysql-branch-parsing' into 4.ee-fix-ghprb-mysql-branch-parsing
Arcadiy Ivanov
MDEV-40692 GROUP_CONCAT replays a group when an OFFSET skips every row

Nothing says how many times a statement asks for the result of a group,
and the answer must not depend on it. A `HAVING` clause on the alias is
the shortest statement that asks twice, and it returns a different value
than the same aggregate asked once:

    SELECT GROUP_CONCAT(a ORDER BY a LIMIT 2 OFFSET 4) v FROM t1;
    -> (empty)
    SELECT GROUP_CONCAT(a ORDER BY a LIMIT 2 OFFSET 4) v FROM t1
      HAVING v LIKE '%';
    -> a,b

`val_str()` walks only while `result_finalized` is false, and
`dump_leaf_key()` raises that flag for the first row it writes. A row
that falls inside the offset is skipped by an earlier return, which
decrements the offset counter and leaves the flag alone. The row-limit
arm immediately above it does raise the flag before its own early
return, so two adjacent early returns behave differently.

A walk in which every row was skipped therefore writes nothing and
records nothing. The next caller walks again with the offset already
spent, and the rows skipped the first time are appended to a result
buffer that was handed over once already.

Once the duplicate filter has spilled to disk the second walk is worse
than wrong. `Unique::reset()` documents the contract:

    Clear the tree and the file.
    You must call reset() if you want to reuse Unique after walk().

The first walk flushed the tree and emptied it, so the second flushes an
empty tree, appending a chunk that holds no rows. `merge_walk()` reads
nothing back from it and fails `DBUG_ASSERT(bytes_read)`. A build
without assertions goes on to take keys from that chunk.

Set `result_finalized` where the walk block ends, so that it records
every path that has consumed the filter rather than only the paths that
wrote a row.

On the release branches only the form without `ORDER BY` reaches the
duplicate filter. Since MDEV-21879 the `DISTINCT ... ORDER BY`
combination builds its result the same way, so both forms can reach the
assertion here.
Vladislav Vaintroub
MDEV-40608 build mysqlservices without an embedded CRT requirement

mysqlservices only exposes a thin C API, no CRT state crosses it, so
don't force whatever CRT/config built the server onto a plugin linking
it. Without /Zl, a plugin built in a config with no matching installed
mysqlservices variant (CMake silently substitutes one - verified with
a toy project) gets an ignorable but noisy LNK4098 warning.

Assisted-by: Claude:claude-5-sonnet
Monty
fixup! db1d6d824d96694acaa9e715836e43562a718e15
Jan Lindström
MDEV-40281 : galera.galera_wsrep_new_cluster test failure

Rejoining with an emptied datadir requires a full SST, which on a loaded
machine does not finish within the 60 seconds galera_wait_ready.inc allowed,
aborting the test while the SST was still running. Let the readiness wait
take an optional $galera_wait_ready_timeout and give that restart 300
seconds.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Sergei Petrunia
MDEV-40168: JSON-over-fulltext: add estimates.

Add records_in_range-like estimates for fulltext index
Jan Lindström
Merge remote-tracking branch 'origin/4.x-26.4.28' into 4.ee-26.4-28
Sergei Petrunia
Move the JSON function MVI code into opt_mvi_jsonfuncs.cc

opt_multi_valued_index.cc held two separate concerns: the index side
(how a value is encoded into the index, the access descriptor, the quick
select) and the predicate side (which JSON functions can be computed
from an MVI, which of their arguments holds the indexed expression, and
what to search the index for). Split the second one out.

Moved verbatim:

  get_mvi_index()
  collect_mvi_keys()
  Item_func_json_contains::get_mvi_access() and ::mvi_analyze()
  Item_func_json_overlaps::get_mvi_access() and ::mvi_analyze()
  add_mvi_access()

The only code change is that encode_mvi_key() is no longer static: it is
used both by Item_func_mvi_encode::val_str_ascii(), which stays, and by
collect_mvi_keys(), which moves. It is declared in
opt_multi_valued_index.h now. The other three moved helpers had no
callers outside the moved code and stay static.

Item_func_mvi_encode is not a JSON function and stays put: it is how the
values get into the index in the first place.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Dmitry Shulga
MDEV-40076: ASAN global-buffer-overflow in reconstruct_create_trigger_stmt

Follow-up patch to address review comments found by reviewer
Alexey Yurchenko
Merge branch 'MDEV-38920-ensure-timeouts' into MDEV-38920-ensure-timeouts-4.ee
Monty
Trivial optimziations for group_concat

- Remove some if
- Reorder code
- More code comments

(cherry picked from commit dc6a897961c311f981b150e4207ffc1390a219ef)
Dmitry Shulga
MDEV-40947: ASAN global-buffer-overflow in events_to_string/reconstruct_create_trigger_stmt

Manual set of mysql.event.kind to the value 'DDL' resulted in crash
on running the statement SHOW CREATE TRIGGER.

The crash was caused on accessing the array base_event_names
by index for DDL trigger event, but the array missing this
entry.

To fix the issue, add an entry for the DDL event in
the array base_event_names.
Sergei Petrunia
MDEV-40168: JSON-over-fulltext: move the estimator into fts0est.cc.

Pure code motion, no functional change.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Jan Lindström
Merge remote-tracking branch 'upstream/4.ee' into mariadb-4.x
Rex Johnston
Parallel Query: the gate says why it turned a query away

"Why did this not run in parallel" was a question the reader had to answer
themselves, from a plan, in a debugger, some way from the query that provoked
it. It is now a question the server answers:

  SELECT JSON_EXTRACT(trace, '$**.parallel_scan_declined_because')
    FROM information_schema.optimizer_trace;

  ["grouped: the aggregation table is keyed by a unique constraint rather than
    by the group -- create_tmp_table() does that for a GROUP BY element too
    wide for a key part, and a hash of the row is not something a group can be
    looked up in",
  "the plan needs a temporary table and the workers cannot pre-aggregate into
    it"]

The first entry is the check that objected; the rest is that refusal working
its way back up. Every one of the gate's refusals now goes through
pwt_decline(), which keeps the DBUG_PRINT and adds the reason to the trace.
Sixteen of them had neither before -- a bare "return false" -- and those were
in the two functions a grouped query spends most of its time in.

Three things this needed beyond saying the words.

The gate is asked twice. Once from make_join_readinfo() while the plan is still
being built, and again from parallel_join_check() when it is finished. The
early call runs before make_aggr_tables_info() has built the aggregation table,
so it reported "the terminal is not end_update()" for every grouped query --
an answer about a plan that no longer exists by the time the query runs, and a
worse place to start than no answer at all. A trace flag threaded through the
gate means only the decision that stands speaks. Threaded rather than kept in a
static, because several connections optimize at once.

Which check fires first decides what you are told. Two reasons were being lost
to a coarser test reached earlier: ordered_index_usage now precedes the
access-method test, since "an index supplies the ORDER BY order" says more than
"this is not a scan the engine divides", and both refuse the same plans. And
the duplicated access-method test in table_can_be_parallel_scanned() is gone,
so is_parallel_scan_applicable() owns that question alone and can tell its
several answers apart.

A reason has to name the cause, not the symptom. A GROUP BY on a column too
wide for a key part is refused because create_tmp_table() keys the container by
a unique constraint, which makes the terminal end_unique_update(), which is why
pwt_plan_group_key() finds nothing -- and "the terminal is not end_update()" is
a true statement that sends the reader nowhere. The check that would have named
the width is never even reached. So when there is no group key, this asks
make_aggr_tables_info()'s own condition of the same table it asked it of, and
says which of the two it is.

main.parallel_query_why pins one refusal per shape that can reach it, so a
reason cannot quietly go missing or attach itself to the wrong query.

This commit was prepared with Claude Code, after a report that finding why a
wide GROUP BY column would not run in parallel took an afternoon in gdb. It
counted the refusal points first -- 22 of 38 traced, 16 silent, the one being
hunted among the silent -- and worked from there.
Sergei Golubchik
workaround for https://bugzilla.redhat.com/show_bug.cgi?id=2390105
Sergei Petrunia
Adjust the MVI tests to the estimate that reads the FTS cache

fulltext_estimate() used to see only the on-disk auxiliary table, so the
multi-valued index tests, which insert and immediately EXPLAIN, got
"unknown" for every element key: the conjunctive accesses fell back to a
guess of 10 rows and the disjunctive ones were dropped for want of a
cost. Both tests worked around that with OPTIMIZE TABLE under
innodb_optimize_fulltext_only. The estimate consults the cache now, so
the workaround is gone and the row counts in the plans are real.

Two sections of the trace test were written before the estimate existed
and no longer showed what they said they did:

- "Several element keys" printed one range, not several, because a
  conjunctive access now keeps only the rarest key. It runs on a table
  with a skewed distribution instead ("bbb" in every row, "aaa" in one),
  which makes the choice of key visible rather than a tie, and checks
  the rows against the same query with IGNORE INDEX.

- The JSON_OVERLAPS section is where several ranges are printed now, so
  it says so, along with the estimates adding up.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Sergei Petrunia
Make optimizer trace print "range", not "index_merge" for MVI quick selects.
Hemant Dangi
Merge branch 'MDEV-36621-4.x' into MDEV-36621-4.ee_1

Adjust page_size in top_level_seqno_lock_protects_ist_buffers to
account for 4.ee per-page meta overhead (gcache::Page::meta_size()).
Alexey Yurchenko
Merge branch 'MDEV-38920-ensure-timeouts' into MDEV-38920-ensure-timeouts-4.ee
Arcadiy Ivanov
fixup! MDEV-40802 COUNT(DISTINCT <blob>) fails when its tmp table converts

`heap.count_distinct_blob_convert` asserts that four of its aggregates
converted their in-memory temporary table to the on-disk engine. On a
32-bit build three of those assertions read `OFF`: the table never
filled up, so nothing was converted and those cases covered nothing.

The test filled a 64 KB in-memory table with 2000 rows, 1000 of them
distinct. On a 64-bit build the table overflows somewhere between 1500
and 2000 rows, so 2000 clears the limit by only a few hundred. A record
holding a blob carries a pointer to the value, and where a pointer is
four bytes rather than eight the record is narrower, so the same 2000
rows no longer reach the limit.

Use 8000 rows, 4000 distinct, several times what the wider build needs.

Only the count is chosen that way. The value widths and the memory
limit stay exactly where they were measured, because they are what
makes the table run out of record slots rather than out of blob space.
That is the case where the row left over at the overflow is a duplicate
of one already copied, and handling that duplicate is what the fix is
about; a limit or a width that overflows on a blob value converts the
table while covering nothing. Rows added past the overflow cannot move
it - it happens once the table is full, whatever follows - so the
64-bit build converts on the same row as before.

Verified by backing the fix out. With `ignore_last_dupp_key_error`
returned to 0 the test fails with `ER_DUP_UNIQUE` at 8000 rows exactly
as it did at 2000. No recorded `CONVERTED` line changed; only the row
and distinct counts did.
Sergei Petrunia
Add optimizer trace for the multi-valued index access

get_best_mvi_access() picked an Mvi_access and wrapped it in a
QUICK_MVI_SELECT without recording anything, so there was no way to see
which index was chosen or what it would search that index for: the
rows_estimation trace showed a range_analysis that found nothing, and
then a plan using a key the trace never mentioned.

Print a "multi_value_index_use" object:

  {
    "table": "t1",
    "index": "idx",
    "ranges": ["616161"]
  }

Mvi_access::print_json() fills in the index and the element keys,
following TRP_RANGE::trace_basic_info(): same "index" / "ranges" member
names, so an MVI entry reads like a range scan's.

The keys are printed in their encoded form, which is not readable. That
is what is stored in the index and what we search for, so it is still
the useful thing to print; making it readable can come later. It is
plain ASCII (hex plus the xx/xxxx padding from encode_mvi_key()), so it
needs no JSON escaping.

get_best_mvi_access() runs inside the "rows_estimation" array, so the
named object needs an object of its own around it, the same way
make_join_statistics() and the sel_arg_alloc_limit_hit trace do it.
Without it the writer hits an assertion in
Single_line_formatting_helper::on_add_member().

The new test is a separate file because optimizer trace tests need
not_embedded.inc, and putting that in multi_valued_index.test would skip
the whole feature test on embedded builds. It cross-checks the printed
keys against mvi_encode() over the indexed column, which produces the
tokens the index is actually built from.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Vladislav Vaintroub
MDEV-40608 propagate DBUG_OFF, ENABLED_DEBUG_SYNC and SAFE_MUTEX to external plugins

they affect ABI, but aren't in headers, so must be passed separately
Sergei Petrunia
Make JSON_OVERLAPS sargable for multi-valued indexes

Both argument orders are handled, since JSON_OVERLAPS is symmetric:

  JSON_OVERLAPS(array_indexed_expr, '[foo, bar, ...]')
  JSON_OVERLAPS('[foo, bar, ...]', array_indexed_expr)

JSON_CONTAINS is true when ALL of the elements have a match, JSON_OVERLAPS
when ANY of them does, so the access it produces has conjunctive=false and
build_ft_query() leaves the keys optional instead of prefixing them with
'+'.

The two get_mvi_access() implementations share collect_mvi_keys(), which
is the scan of the JSON literal that used to sit inside
Item_func_json_contains::get_mvi_access().

The two differ in one way beyond the flag. An element that cannot be
encoded for the index (a number against a CHAR array, say) is skipped for
JSON_CONTAINS: dropping a key from an AND makes the index scan less
selective, so it still returns a superset of the rows the predicate
matches and the predicate does the exact filtering afterwards.

That reasoning does not hold for an OR. A row can satisfy the predicate
through the very element we failed to encode, and MVI_ENCODE skips such
elements as well, so that row has no key in the index for the scan to find
it by - dropping the key would lose it. So for a disjunctive access we
give up instead of skipping. With a row {"tags": [123]} in the table,

  select * from t1 where json_overlaps(j->'$.tags','[123,"aaa"]')

must not use the index, and the test checks its result against the same
query with IGNORE INDEX.

Also print "match": "all"/"any" in the optimizer trace. Now that an access
can be either, the printed ranges alone did not say whether a row has to
have all of the keys or just one.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Rex Johnston
Parallel Query: AVG, and an aggregate inside an expression

TPC-H Q1 now runs in the workers. Two things were stopping it, and both are
here.

AVG
---
An average does not compose out of averages, so what a worker has for a group
is two numbers: the sum of its rows and how many there were. Nothing had to be
invented to carry them. Item_sum_avg::create_tmp_field() already packs a sum
and a count into one field, because that is how the serial grouped plan
accumulates an average, and a worker's grouped container was already
maintaining both.

What was missing was the merge. The manager primes its own aggregate with a
worker's partial through direct_add(), and Item_sum_sum::direct_add() carries
only the sum -- Item_sum_avg::count was left untouched, so every partial
counted as one row. direct_add() now takes the count alongside the sum, and
reset_field()/update_field() add it to the field's count instead of the 1 an
ordinary row contributes, in the same shape Item_sum_sum's already had.

Nothing changes for a serial plan: direct_added is false there, and the new
branches are not taken.

An aggregate inside an expression
---------------------------------
SUM(x)+1 was refused, and so was SUM(x)/COUNT(x) -- the obvious way to write an
average by hand. A select list item that is not itself an aggregate is
evaluated once per group from whatever row the terminal is holding, so a field
in it that GROUP BY does not name takes an arbitrary value; that is worth
refusing. But the check walked into the aggregates as well, found x under
SUM(x), and refused a query that was never in doubt.

It now steps over Item_sum subtrees, using with_sum_func() to answer an
aggregate-free subtree wholesale with the walk that was there before, and
decomposing only the node kinds that can hold an aggregate. Anything else
carrying one is refused rather than guessed at: being wrong in the permissive
direction here is wrong results. SUM(x)+x is still correctly refused, because
the bare x is walked and is not a group column.

The shipped row's shape
-----------------------
With more than one AVG the second onward came back wrong, which was not
arithmetic. Item_sum_avg is the one aggregate whose create_tmp_field() differs
between a keyed and an unkeyed temporary table -- it packs the count beside the
sum only for the keyed form -- and the two ends of the transport were built
differently:

  recv, group_container  make_container(..., group)  16-byte {sum, count}
  exec.result (shipped)  make_container(...)          plain 8-byte double

flush_groups() copies a group container record into the shipping container by
reclength, so it wrote past a narrower record and the manager read back a wider
one, drifting by one AVG's worth of bytes at a time. For SUM, COUNT, MIN and
MAX the two widths are identical, which is why this sat there unnoticed. The
shipping container is now built with the same key; a worker ships one row per
group, so there is nothing for the key to collapse.

main.parallel_query_aggregate gains an AVG section -- serial against parallel
over a DECIMAL average, and three AVGs at once, which is the case that exposed
the shape bug -- and AVG leaves its "what is not accepted" list. STDDEV,
BIT_XOR and COUNT(DISTINCT) stay there.

This commit was prepared with Claude Code: it bisected TPC-H Q1 down to the two
refusals, found the container shape mismatch from the pattern that only the
last AVG was wrong, and fixed a null Item passed to VDec that the aggregate
test caught as a crash.
Teemu Ollakka
Merge remote-tracking branch 'origin/4.x-MDEV-36926' into 4.ee-MDEV-36926
Vladislav Vaintroub
MDEV-40608 propagate DBUG_OFF, ENABLED_DEBUG_SYNC and SAFE_MUTEX to external plugins

they affect ABI, but aren't in headers, so must be passed separately