Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Sergei Petrunia
Fix ubsan failure with not a valid value for enum_mysql_show_type

It produced errors like:
  runtime error: load of value 19, which is not a valid value for
  type 'enum enum_mysql_show_type'

In order to get a correct server-side definition of enum_mysql_show_type,
one must include sql_plugin.h (see #define SHOW_always_last there) before
including include/mysql/plugin.h, either directly or indirectly.
bsrikanth-mariadb
Do not dump stats and const rows for read only engines' tables.

Stats for tables from engines such as Archive, S3, PerfSchema, and
Sequence shouldn't be recorded in the context. Similarly, const row
records should also not be stored in the context.

Added few tests for Sequence's engine tables like seq_1_to_5.
Marko Mäkelä
MDEV-40728 Recovery wrongly fails if FILE_CREATE is followed by FILE_RENAME

fil_name_process(): Simplify the logic. If no matching tablespace is
found but file_name_t::create_lsn had been set in response to parsing
a FILE_CREATE record, try to apply FILE_RENAME to deferred_spaces.

deferred_spaces.deferred_dblwr(): Skip newly created tablespaces
to avoid a bogus invocation of fil_space_free().
KhaledR57
MDEV-40495 KEY_OP_SHIFT redo moves data past the page buffer

The KEY_OP_SHIFT branch of _ma_apply_redo_index() took the shift length
straight from the redo record and used it to form a bmove() source and
size, and to update page_length. The only guards were DBUG_ASSERTs, which
are compiled out when DBUG_OFF is set. A corrupt record with a negative
length could therefore move data from outside the page and wrap
page_length.

Turn both asserts into runtime checks. The page offset must be set and
inside the used page, the resulting page length must still fit the page,
and for a negative shift the source must stay inside the used page too.
The conditions are the ones the asserts already tested, so debug builds
keep the same behaviour.

This also fixes MDEV-40496, which covers the page_length side of the same
branch. The first check bounds it.

The test forges the logged shift length with two new debug keywords,
corrupt_shift_down and corrupt_shift_up, then crashes the server so
recovery has to replay the record.
Sergei Golubchik
MDEV-40608 MariaDB-devel is incomplete for plugins

* create and install mariadb-plugin-config.cmake
* deb: move all headers that plugins need to libmariadb-dev,
  together with libmysqlservices.a. At least until we'll
  create mariadb-plugin-dev.Nobody should need huge
  libmariadbd-dev to develop a plugin
* rpm: all in MariaDB-devel already, no changes here
* adjust plugin.cmake to work for external plugins
* move server-internal part to top-level CMakeLists.txt
* remove WITH_WSREP from my_config.h (it upsets external plugins)
* remove double-defined macros from unireg.h (the guard doesn't help
  if unireg.h is included first)

ColumnStore, until fixed, needs a backward-compatibility workaround
bsrikanth-mariadb
MDEV-40387: perfschema.misc fails on replay

Disable the testfile, as we don't capture context for performance schema
tables.
Daniel Black
sql_test: mallinfo2 msan exclusion no longer needed

MSAN interceptor was added in clang-18.1.
Daniel Black
MDEV-17846 Wrong result with grouping select (fix)

Prevent unused variable 'ref_type'  warnings on non-debug builds.
Yuchen Pei
MDEV-40486 Length check for vector fields in CREATE TABLE ... SELECT

The changes of MDEV-39558 2b6529426a7e7c65d286e093d84138be9dcc34a3
added length check assertion in Field_varstring constructors, and
length check in type inference for SELECT set operations, to emit
errors before reaching the assertions.

That change caused an error to turn into an assertion failure in a
separate path, when the length limit violation is not detected before
tripping the assertion. So in this patch we fix it by adding an
earlier length check in that path.

The reason that we place this check inside
Item_func_vec_fromtext::fix_length_and_dec rather than say
`create_field_for_create_select is for consistency:

If

create table t1 as select
vec_fromtext(concat('[',group_concat(1),']')) as c1 from seq_1_to_64;

fails due to length limit violation, then so should

create table t1 (v vector(64) not null);
insert into t1 select vec_fromtext(concat('[',group_concat(1),']'))
from seq_1_to_64;

Also use max_char_length() instead of max_length. This is a more
accurate length of characters. And add handling of empty string edge
case. Added testcases accordingly.

The change that uses max_char_length() causes side effects where
creating a table using a VEC_FROMTEXT(CHAR(1)) would result in a
0-dimensional vector field. This is accurate but 0-dim vector table
fields should not be allowed. So we make cases like this result in
a one-dimensional field.

Also fixed the underflow in (args[0]->max_length - 1) * 2 when the
arg's max length is 0. Previously this underflow would cause

create table t1 select vec_fromtext(NULL)

to fail with ER_TOO_BIG_FIELDLENGTH. Now it will be a VECTOR(1) field
bsrikanth-mariadb
MDEV-39360: set statement optimizer_record_context for query fails

Move the initialization of context recorder, and replay after
run_set_statement_if_requested() is invoked in the
mysql_execute_command() in sql_parse.cc
bsrikanth-mariadb
MDEV-40383:innodb_gis.point_basic fails on replay

There are 2 problems: -
1. The REPLACE statement that is recorded doesn't store the
  value of geometry type field correctly.
2. The table definition that got recorded has fields with non-null constraint,
  and no default value is specified.
  Also, the "REPLACE INTO" statement that gets stored in the context,
  doesn't have any value specified for these non-null fields.

Solution is to: -
1. When using REPLACE INTO statement, store all the non-numeric values in HEX,
  whenever conversion from field's charset to output's charset is lossy.
2. Instead of storing only the column values that were projected in the
  query, store all the non-virtual column values into the recorded
  REPLACE INTO statement.

Implementation details: -
1. Introduce a new method is_charset_conversion_lossless() in filesort.cc,
  to check if the output charset to which field's data is being written to,
  results in a lossless conversion. If so, non-numeric values being witten
  using REPLACE INTO statement are stored in string representation,
  else they are converted to HEX.
2. Modify join_read_const(), and join_read_system() methods in sql_select.cc,
  and opt_sum_query() method in opt_sum.cc the following way: -
    a. Extend the read_set to make sure, we read all the non-virtual column
        using Optimizer_context_recorder::prepare_captured_row_read().
        This method also saves the original read_set.
    b. Read the row.
    c. Dump the row into the context when no error is noticed while
        reading. Irrespective of the error, restore back the read_set state to
        the original using Optimizer_context_recorder::finish_captured_row_read()
Sergei Petrunia
Code cleanup in JSON array-of-object reading, add unit tests.
bsrikanth-mariadb
MDEV-38701: Optimizer Context Replay: merge into 13.1 tree

Optimizer Context Replay feature allows one to record and replay
a query's Optimizer Context. Optimizer Context includes everything
that one needs to replicate how the Query Optimizer processed the query.

It can be replayed on another to host to debug how the Query Optimizer
processed the query, run what-if scenarios, etc.

== Example recording ==

  set optimizer_record_context=1;
  < Run the query of interest. Typically it's EXPLAIN ...>;
  select context [into dumpfile '/tmp/context.sql']
  from information_schema.optimizer_context;

== Example replay ==
  -- On another machine, just source the script
  source context.sql

This will
* Set relevant system variables to match the recording side;
* Create the dataases, tables and views the query needs;
* Load EITS statistics for the tables;
* Provide the optimizer with other context data
* Finally re-run the query. If it was an EXPLAIN, one should get the
  same output as on the recording side.

Approved-by: Sergei Petrunia ([email protected])
bsrikanth-mariadb
MDEV-40384: innodb_gis.geometry fails on replay

The test had innodb_strict_mode turned OFF, when running the test. But,
in the replay, it was enabled, which caused the creation of tables with
KEY_BLOCK_SIZE=16 fail.

Solution is to record the innodb_strict_mode variable in the context, so
that it gets used during the replay.
Oleksandr Byelkin
attempt #2
bsrikanth-mariadb
MDEV-40388: sequence.simple fails on replay

The problem is that, when recording is enabled for the query such as,
explain select * from seq_1_to_10;
it recorded the table context having a DDL definition as: -

CREATE TABLE `seq_1_to_10` (
    ->  `seq` bigint(20) unsigned NOT NULL,
    ->  PRIMARY KEY (`seq`)
    -> ) ENGINE=SEQUENCE DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci;

Now, when that context is replayed, the DDL statement is executed.
But, we cannot create such a table, and instead it errors out saying
ERROR 1050 (42S01): Table 'seq_1_to_10' already exists.

Solution is to use: -
  CREATE TABLE IF NOT EXISTS seq_1_to_10 ...;

=====

Also, there is a different way to use sequences as: -
  Create sequence s1;
  Explain select * from s1;

Here, we should be recording the DDL statement, but no need to store the
stats for it. However, we didn't record the DDL statement earlier.
Moreover, sequence's next value should be the same in the replay environment.

Solution here is to record the DDL for such a sequence as
  CREATE TABLE IF NOT EXISTS s1 ...;
and also set its start value as the recorded environment's previous value using
  SELECT SETVAL(s1, prev_value);
Oleksandr Byelkin
attempt #3
Sergei Petrunia
MDEV-40742: Optimizer Context Replay: wrong table names on MacOS

show_create_table_ex() will use table's alias when lower_case_table_names=2.
Specify that table names must be used like the table is stored on disk.
Daniel Black
MDEV-40243 Fix MEMORY_LEAK_C leaks in mariadb-dump  (rockdb tests)

With memory leaks fixed in the rocksdb.mysqldump/mysqldump2 no
longer need to run with leak detection disabled.

There tests are still disabled as there's no --rockdb arg to
mariadb-dump, but removing so there's no precidence to ignoring
leaks.

ref: 2217477fb8fc82f4921d9d13afcd24b1d86b34d6
Georg Richter
Cleanup for test: drop my_vector table
bsrikanth-mariadb
MDEV-40389: type_test.type_test_int8 fails on replay

plugins are not yet supported in replay mode.

So, disabling tests type_test.type_test_int8, and type_test.type_test_double
to be run in replay-server mode
Daniel Black
RocksDB: compile fix std::replace requires algorithm header

Otherwise it compile fails.

Found in clang-24.
Marko Mäkelä
fixup! 38d0d83910a9d9a8db2921bdce2a101065e9f68e
bsrikanth-mariadb
MDEV-39226: Add multi-table update, delete feature
Alexander Barkov
MDEV-28498 Incorrect information in file: './test/t0.frm' on CREATE TABLE

Applying HEX encoding write writting an ENUM/SET TYPELIB to FRM
if the TYPELIB has 0x00 bytes in the value.

This HEX encoding was earlier used only to write UCS2/UTF16/UTF32 TYPELIBs.

A new flag FIELDFLAG_FRM_HEX_ENCODED_TYPELIB was added to indicate
that the TYPELIB is hex encoded. It's used only inside FRM.
Note, it's mangled with FIELDFLAG_TREAT_BIT_AS_CHAR.
This should not be harmful:
- BIT and ENUM/SET columns are handled by two separate code branches
  when opening an FRM
- The flag is unset immediately after decoding TYPELIB, so the rest
  of the code does not se an unexpected flag combination.
KhaledR57
MDEV-40495 KEY_OP_SHIFT redo moves data past the page buffer

The KEY_OP_SHIFT branch of _ma_apply_redo_index() took the shift length
straight from the redo record and used it to form a bmove() source and
size, and to update page_length. The only guards were DBUG_ASSERTs, which
are compiled out when DBUG_OFF is set. A corrupt record with a negative
length could therefore move data from outside the page and wrap
page_length.

Turn both asserts into runtime checks. The page offset must be set and
inside the used page, the resulting page length must still fit the page,
and for a negative shift the source must stay inside the used page too.
The conditions are the ones the asserts already tested, so debug buildskeep the same behaviour.

This also fixes MDEV-40496, which covers the page_length side of the same
branch. The first check bounds it.

The test forges the logged shift length with corrupt_shift_length
debug, then crashes the server so recovery has to replay the record.
Rex Johnston
MDEV-39492 PQ workers must read the manager's transaction snapshot

A parallel worker runs in its own THD, so it runs in its own transaction and
opened its own read view at its first read. Nothing tied that view to the one
the manager holds: the workers and the manager could each read a different
version of the table, and the chunk boundaries the engine partitioned under
the manager's view did not even belong to the snapshot the workers scanned.
Both directions were visible. Under REPEATABLE READ a worker returned rows
another session had committed after the manager's snapshot was taken, and rows
the manager's own transaction had written but not committed went missing,
because to the worker's view the manager was just another active transaction.

InnoDB implements it by copying the source transaction's read view.
ReadView::clone() installs the source's ReadViewBase state and opens the copy.
It inherits the source's m_creator_trx_id rather than keeping the worker's own,
so rows written by the sharing transaction itself stay visible through the copy
-- that is what changes_visible() uses the creator id for -- and it inherits
the source's isolation level, so a READ UNCOMMITTED source is still read
without consulting a view at all. The source view is read under its m_mutex,
the same protection the purge coordinator takes in ReadView::append_to(), and
from_thd's handlerton data under its LOCK_thd_data.
Marko Mäkelä
MDEV-40728 Recovery wrongly fails if FILE_CREATE is followed by FILE_RENAME

fil_name_process(): Simplify the logic. If no matching tablespace is
found but file_name_t::create_lsn had been set in response to parsing
a FILE_CREATE record, try to apply FILE_RENAME to deferred_spaces.

deferred_spaces.deferred_dblwr(): Skip newly created tablespaces
to avoid a bogus invocation of fil_space_free().

fil_delete_apply(): A wrapper for fil_space_free(). When recovering
a log in innodb_log_archive=ON format, we must apply FILE_DELETE
records in order to avoid a future clash with FILE_CREATE or FILE_RENAME.
Sergei Petrunia
Remove incorrectly added sql/opt_sum.cc.orig
Rex Johnston
MDEV-40012 Parallel Query: execute the join in the worker threads

Each parallel worker now runs the join and the WHERE over its own chunk of
the driving table and ships the base-table columns of every row that
qualified. The manager copies those columns back into the fields they came
from in its own table instances, so that after a row is drained its records
hold what a serial scan would have left there, and then evaluates the select
list against them and sends the row. This replaces the model where workers
shipped raw source records and the manager ran the join.

The transport carries columns rather than projected select-list values
deliberately. Anything the manager does with a row beyond sending it -- a
condition, an aggregate, a temp table -- reads a record, and not all of those
reads go through Items that could be re-pointed at a shipped value:
create_tmp_table() builds Copy_field pairs holding raw Field pointers into
the base tables. Filling the records themselves is what makes every later
stage work without an indirection of its own.

make_join_readinfo()'s gate (can_run_query_in_workers) chooses the
worker-side path for an inner select-project[-join] with a parallel-
scannable driving table: no tmp table (group/distinct/order/window/
buffer), no LIMIT/SQL_CALC_FOUND_ROWS/procedure/aggregate, no outer join
or semijoin, and every non-driving table reached by eq_ref/ref/full
scan. do_select() then runs run_worker_side_join() instead of the nested
loop; anything ineligible runs serially.

Each worker opens a private copy of every non-const table, deep-clones
and field-rebinds the conditions and select list, and rebuilds each ref
(clone_table_ref, mirroring create_ref_for_key). It scans its driving
chunk, runs its own inner nested loop (cp_buffer_from_ref +
ha_index_read_map / ha_index_next_same for ref/eq_ref, rnd scan
otherwise), projects the read columns of each full match into a private
result table and ships the row image; the manager drains, copies the columns
into its own records and sends.

Writing into those records is not something a SELECT's write_set allows and
Field::store() asserts on it, so the drain marks the fields writable for its
duration; and Copy_field captures &table->null_row, so the manager's tables
have their reader flags cleared before it starts.

A killed worker's own ER_QUERY_INTERRUPTED is no longer treated as a
fatal evaluation error (PWT_error_handler guards with !thd->killed) so
kills keep propagating through kill_signal with the correct kill type.

Standalone helpers are attached to the object they operate on
(pwt_worker / pwt_manager members rather than file-static functions) and
the functions that carry real control flow have DBUG_ENTER tracing.

Tests: parallel_query_worker_side (single table) and parallel_query_join
(eq_ref, ref with fan-out, 3-table chain, full-scan inner table) compare
the parallel result set against serial. parallel_query / parallel_query_oom
moved to a plain SELECT (which now runs worker-side) and were re-recorded.

This commit was prepared with Claude Code: it wrote the worker-side join
execution (worker_join_inner / worker_emit_row, the per-worker table,
ref and expression cloning) and the two new tests; the naming, the
member-function layout and the DBUG tracing are the author's cleanup.

run_worker_side_join() returned 1 -- error -- when init_parallel_workers()
reported HA_ERR_UNSUPPORTED, although its own documented contract, and the
comment on that very branch, say -1 means "the engine declined, run the query
serially". No error was raised on the way out, because nothing had gone wrong:
the engine simply refused the scan. do_select() turned that 1 into
NESTED_LOOP_ERROR, so the statement failed with an empty diagnostics area and
Protocol::end_statement() hit its DA_EMPTY assertion. In a release build the
client gets a statement that neither succeeds nor reports an error.

The engine declines this way for any read that is not a consistent read:
pscan_init_coordinator()'s first check refuses the scan when
select_lock_type != LOCK_NONE. So SELECT ... FOR UPDATE,
SELECT ... LOCK IN SHARE MODE, CREATE ... AS SELECT and INSERT ... SELECT over
a parallel-scannable table all crashed a debug server as soon as
parallel_worker_threads was set -- the optimizer picks the table, then
execution has nowhere to go. Return -1 so do_select() takes its serial path,
which is what make_join_readinfo() left in place for exactly this case by
keeping the table's serial read_first_record.

parallel_query_fallback is the new test: FOR UPDATE, LOCK IN SHARE MODE,
CREATE ... AS SELECT and INSERT ... SELECT each return their rows with workers
enabled. All four reach the decline path (the DBUG_PRINT added here fires four
times), and the first of them already crashes the server without this commit.

This also un-breaks parallel_query_join and parallel_query_worker_side, which
have been failing on this branch since they were recorded: both open with a
CREATE ... AS SELECT that took the bad path. Their serial-vs-parallel
comparisons are built from CREATE ... AS SELECT pairs, so now that the fallback
works both sides run serially and those comparisons no longer say anything
about worker-side execution. The assertion in parallel_query_join that claimed
CTAS "runs worker-side" is corrected to state what the trace key actually
reports, which is the optimizer's choice. Restoring a like-for-like comparison
at scale needs a technique that does not write from a SELECT, and is left to a
follow-up.

This commit was prepared with Claude Code: it traced the DA_EMPTY assertion to
the wrong return code with a DBUG trace, reduced the repro from
CREATE ... AS SELECT to SELECT ... FOR UPDATE, and wrote the test.

MDEV-39492 Parallel Query: give worker tables their place in the join

A worker's private table copies came out of open_table_from_share() with
TABLE::map still zero, because nothing assigns it outside the optimizer's
setup_tables(). Item_field::used_tables() reads TABLE::map, so every item
rebound onto a worker copy reported used_tables() == 0 -- it looked like a
constant. pwt_clone_rebind() re-fixes the clone, and Item_cond::fix_fields()
evaluates any argument that can_eval_in_optimize(), so cloning the condition of

  SELECT a, b, c FROM t1 WHERE a % 7 = 0 AND b > 100

evaluated "b > 100" on the manager thread, against a worker record buffer that
no row had been read into. Debug builds assert in Field_long::val_int() on
marked_for_read(); release builds read that unread buffer to compute a
null-rejection cache. Any WHERE with more than one predicate was affected --
that is, most of them.

Copy map and tablenr from the manager's table when the copy is opened, so a
rebound item computes the same used_tables() as the item it was cloned from and
nothing evaluates it at clone time. Mark all columns readable there too, before
any cloning touches those tables, rather than at the start of worker_run_query()
-- the read_set then holds for the whole life of the copy, and the per-table
loop in worker_run_query() goes away.

This was invisible because no test reached it. parallel_query_join and
parallel_query_worker_side compared parallel against serial by building the
same result twice with CREATE ... AS SELECT, and writing from a SELECT is a
locking read, which makes the engine decline the parallel scan: both sides ran
serially, so the multi-predicate WHERE never reached a worker clone.

Those comparisons are rewritten to compare something that really does run in
the workers. A result set that cannot be written to a table cannot be compared
afterwards either, so each query now goes inside a non-mergeable derived table
-- the inner select runs in the workers, the outer aggregate runs in the user
thread -- and is reduced to COUNT(*), SUM(CRC32(...)) and BIT_XOR(CRC32(...)).
That fingerprint is order-independent, which a parallel result needs since its
rows arrive in batch-completion order, and it compares the whole multiset at
full scale without printing it. The shared method lives in
include/parallel_query_fingerprint.inc.

Each comparison also asserts that the workers, not the user thread, did the
scanning: both runs read the same materialized rows, but only the serial one
also scans the driving table, so the serial run's Handler_read_rnd_next must be
the larger of the two. That is what stops these comparisons from silently going
hollow again -- with the parallel half forced back to 0 workers the check flips
from 1 to 0 and the tests fail.

Both tests were checked to be load-bearing by mutation: dropping one worker
result row in 500 moves every fingerprint (713 -> 712, 5000 -> 4990,
4967 -> 4958, 6000 -> 5988).

This commit was prepared with Claude Code: it root-caused the assertion to the
zero table map, wrote the fingerprint comparison and the include, and ran the
mutation checks.

MDEV-39492 Parallel Query: worker tables inherit the manager's column bitmaps

A worker's table copies were marked with use_all_columns(), which points both
read_set and write_set at the share's all_set. Correct, since it is a superset
of what the worker evaluates, but coarser than it needs to be in two ways.

InnoDB builds its row fetch template from read_set, so with every column marked
a worker converts every column of every row it scans to MySQL format, not just
the ones its cloned condition and select list reference -- work paid on the hot
path that parallel query exists to shorten, and it grows with the width of the
table rather than with the query. And an all-columns write_set on a table the
worker only ever reads gives up an assertion: a store into a source field no
longer trips marked_for_write().

Copy the manager table's read_set and write_set into the copy's own
def_read_set/def_write_set instead, and point the copy at those. The optimizer
has already marked exactly the columns this query reads, and
open_worker_tables() runs after it has finished, so those bitmaps are final; the
copy owns its bitmaps, so nothing is shared with the manager, and
column_bitmaps_set() signals the engine to rebuild its template.

This keeps the property the marking was added for: every column a cloned item
could touch is in the read_set before any cloning happens, because the optimizer
marked precisely that set.

Two cases in parallel_query_worker_side cover the fidelity of the copy, both
through the serial-vs-parallel fingerprint: a table with eighteen columns where
the query references two, and a virtual column whose base column is read but
never projected. Clearing a single bit of the copied read_set makes both of them
fail (Field_long::val_int() asserts on marked_for_read()), as does
parallel_query_join.

Index-only reads are unaffected by this: they need ha_start_keyread(), which the
worker-side path never issues, for any read_set.

This commit was prepared with Claude Code: it identified the cost of the blanket
marking, made the copy, and added the two coverage cases.

MDEV-39492 Parallel Query: give constant select-list items a result field

SELECT 42, a FROM t1 crashed the server whenever parallel_worker_threads was
set. create_tmp_table() does not give a constant item a field, which is right
for a query that materialises its result and can evaluate the constant once
outside the table, but the parallel result table is not that. Its layout has to
mirror the select list item for item, because a worker projects item i into
field i and ships the record image, and the manager sends one Item_field per
field to the client. With a constant in the list the table came out one field
short, and worker_emit_row() ran off the end of the field array into a NULL
Field pointer. The manager's send list was equally short, so even without the
crash the client would have been sent the wrong number of columns. Any constant
did it, a literal, a folded expression such as 1+1, or a session constant like
CONNECTION_ID().

Pass TMP_TABLE_ALL_COLUMNS when building the result table, so every item of the
select list gets a field, and assert the layout matches the list afterwards --
the transport is positional, so a mismatch from any other cause has to be
refused rather than walked over.

That exposed a second problem in the same place. create_tmp_table() overwrites
param->func_count with the number of items it actually has to copy, and
make_result_table() was called once per worker plus once for the manager from a
single TMP_TABLE_PARAM counted once by the caller. A constant needs a field but
no copy entry, so the count dropped to zero after the first table and every
later one allocated fewer fields than its layout, tripping the assertion in
Create_tmp_table::finalize(). Each result table now starts from a freshly
counted param, which is what N identical layouts from one param needs in any
case.

Session constants come out right rather than merely not crashing: the clone is
fixed on the manager's thread, so CONNECTION_ID(), USER() and DATABASE() carry
the user session's values, not the worker's. The test asserts that against
values captured outside the query.

parallel_query_worker_side covers a literal, a string literal, a folded
expression, an all-constant select list where no column of the scanned table
reaches the result, and the session constants, plus a serial-vs-parallel
fingerprint over a select list containing a constant so the case is known to run
in the workers. Every one of them crashes the server without this commit.

This commit was prepared with Claude Code: it found the crash while probing which
expressions the gate accepts, traced both causes from the core file, and wrote
the tests.

MDEV-39492 Parallel Query: refuse an expression copy that is not independent

A worker repoints the Item_field leaves of its copy of the WHERE condition, the
ref values and the select list at its own tables. That is only safe if the copy
owns those leaves. Several item classes implement deep_copy() as a shallow copy
while still holding child items, every Item_cache and Item_outer_ref among them,
so their copy keeps pointing at the original's children, and Item_cache::walk()
does visit those children. Rebinding one moved the manager's own Item_field onto
worker one's table, then worker two moved it again, so every worker and the
manager ended up sharing a field bound to some other thread's table. The
in_use assertion in Field::val_int() caught it in a debug build,

  SELECT STRAIGHT_JOIN count(*) FROM t1 JOIN t2 JOIN t3
    WHERE t1.f1 IN (SELECT f1 FROM t4) AND t2.f1 IN (SELECT f1 FROM t5);

crashing after semijoin conversion left a cached reference in the condition of a
select the workers ran. A release build reads another thread's record buffer,
and the manager's own plan is left rebound to a table that is closed when the
workers are reaped.

The gate now requires more than clonability, it requires the copy to share no
Item_field object with the item it came from. Testing it there costs nothing,
the gate already deep-copies every candidate item and throws the copy away, and
a query that fails the test declines to serial execution at optimize time.
pwt_clone_rebind() asserts the same property before it rebinds anything, so a
copy that slips through is caught at the point of damage rather than by whatever
reads a foreign table later.

Checking for a shared leaf rather than enumerating the classes that shallow-copy
means the check also covers the classes not yet met. There are more of them than
Item_cache: Item_outer_ref and Item_copy_string hold child items and copy
shallowly too.

The three tests that hit this with parallel_worker_threads forced on --
opt_hints_join_order, innodb_mrr_cpk and subselect_innodb -- no longer crash.
parallel_query_clone is the new test: the semijoin query above, plus a
correlated reference in a select list and one in a condition, each answered
serially and again with workers enabled. Without this commit the first of them
trips the assertion in pwt_clone_rebind(), and without that assertion too it
reaches the original in_use crash.

This commit was prepared with Claude Code: it traced the assertion from the core
file to the shallow deep_copy, wrote the shared-leaf check and the test.

MDEV-39492 Parallel Query: the workers' statistics belong to the session

A worker counts its reads, its lock calls and everything else in its own THD, and
~THD puts them straight into the global counters. Nothing ever reached the
session that asked for the work, so SHOW SESSION STATUS was short by whatever the
workers did: after a parallel scan of a 1000-row table Handler_read_rnd_next
reported around 500, the rows this thread read out of the materialised result,
rather than the 1502 the same query reports serially. Nine tests in the main
suite noticed, in Handler_read_%, COLUMN_DECOMPRESSIONS and
Optimizer_join_prefixes_check_calls.

Each worker now copies its status counters into its pwt_worker just before its
THD is destroyed, and quiesce_workers() adds them to the session's own after
joining every worker, so only one thread ever touches either side and no locking
is involved. The worker then clears its own counters, which stops ~THD adding the
same numbers to the global counters that the session will pass on later.

Only the counters move. Memory accounting stays with the worker's THD, because
more of that THD's memory is freed after the snapshot is taken and ~THD has to
reconcile all of it with the global counters. Clearing with the
clear_for_flush_status offset leaves those fields alone, and the snapshot drops
its copies of them. Suppressing ~THD's accounting entirely instead, which is the
obvious way to avoid counting the same numbers twice, loses that reconciliation:
the server then reports an internal memory accounting error of a couple of
hundred thousand bytes at shutdown.

Session and global counters were not equally wrong. Global was right all along,
because ~THD was adding to it, so this changes which session the work is
attributed to and not the total.

parallel_query_worker_side compares Handler_read_rnd_next for the same query run
serially and in parallel, and requires the parallel run to report at least as
much. Written as a comparison rather than an exact figure because a chunked scan
ends at each chunk, so it reads a few more than a single scan does. Without the
handover the parallel run reports about a third of the serial figure and the test
fails.

This commit was prepared with Claude Code.

MDEV-39492 Parallel Query: filter by the condition from before the pushdown

When the optimizer pushes part of a condition into the engine,
push_index_cond() leaves tab->select_cond holding only the remainder and keeps
the original in tab->pre_idx_push_select_cond. The pushed half lives on from
there in handler::pushed_idx_cond, which belongs to the handler it was pushed
into -- the manager's. A worker reads through its own handler, opened by
open_table_from_share() with nothing pushed into it, and it cloned select_cond,
so the pushed half was enforced in neither place.

  SELECT pk, a, b FROM p1,p2,p3 WHERE b >= d AND pk < c AND b = '0';

answered one row serially and six with workers, `pk < c` having been applied
nowhere, and the unfiltered rows then multiplied against the third table.
Setting index_condition_pushdown=off made the parallel answer correct, which is
what pinned it on the pushdown rather than on the plan the cost model chose.

Clone the pre-pushdown condition where there is one. One accessor,
pwt_table_cond(), is used by the gate and by both clone sites, so the item the
gate approves is always the item a worker evaluates -- the two drifting apart is
what this bug was.

This gives up what the pushdown was for: the engine no longer rejects an index
entry before the row is read, so a worker does more clustered-index work per
match than the serial plan. The alternative was to refuse these plans at the
gate, which would have cost the parallel scan altogether on a common plan shape.
Correct and parallel beats correct and serial here, and pushing a clone onto the
worker's own handler would recover the difference -- that wants the worker to
hold a real JOIN_TAB to hang the key number off, so it belongs with the cloned
JOIN, not before it.

parallel_query_join gains the query above, answered serially and again with
workers, asserting that it still ran in the workers rather than falling back and
that p1 still carries a pushed index condition, so the case cannot quietly stop
being covered. Without the fix the parallel answer is six rows.

range_innodb, which is where this was found with parallel_worker_threads forced
on, now differs only in EXPLAIN output.

This commit was prepared with Claude Code: it traced the predicate to
push_index_cond() moving it out of select_cond, and wrote the test.

MDEV-39492 Parallel Query: give ANALYZE the numbers the workers produced

ANALYZE reads a JOIN_TAB's counters from the tracker the optimizer left on the
Explain object, and the engine's counters from the handler it recorded there.
Both belong to the manager, and the manager never runs the driving table's read
loop, so a parallel query reported the table as untouched: r_loops 0, no r_rows,
no r_filtered, no r_engine_stats. For a feature whose whole purpose is to make a
scan faster, the tool for seeing where a scan spends its time said nothing about
it.

Each worker now counts what it did to each of its tables in the same terms
sub_select() and evaluate_join_record() use -- rows read, rows that passed the
table's condition, and one scan per probe of an inner table -- and copies the
engine's counters out of the tables while they are still open. The manager adds
all of it to the trackers and handlers ANALYZE reads, in quiesce_workers() after
every worker has been joined, so this thread is the only one touching either
side and no locking is needed. ha_handler_stats::add() already existed for the
partitioning case and does the engine half.

The driving table reports one scan, not one per worker: the chunks are one scan
of the table between them, which is what the serial plan reports and what keeps
the rows-per-scan figure comparable between the two.

  ANALYZE SELECT a FROM t1 WHERE a % 7 = 0;

now answers r_rows 5000.00 and r_filtered 14.28 whether it runs serially or in
the workers, differing only in the access type.

Not included: r_table_time_ms and r_other_time_ms, which come from the elapsed
time trackers rather than from counters, and which the workers measure in
parallel. Summing them would report more time against the table than the query
itself took, and reporting one worker's share would understate the work. That
needs a decision about what ANALYZE should mean for a parallel scan, so it is
left out rather than guessed at, and the two fields stay absent.

parallel_query_worker_side compares the tabular ANALYZE for the same query run
both ways, with the row estimate masked because it is an InnoDB approximation.
Without the handover the parallel run reports r_rows and r_filtered as NULL.

This commit was prepared with Claude Code.

MDEV-39492 Parallel Query: filter by the condition the join buffer would apply

An inner table read through a BNL join buffer keeps part of its condition
somewhere a worker never looked. JOIN_TAB::make_scan_filter() copies the
conjuncts that need only that table into cache_select->cond, for the buffer to
apply as it fills, and JOIN_TAB::remove_redundant_bnl_scan_conds() then removes
those same conjuncts from select_cond -- it calls set_cond(NULL) outright when
they were the whole of it. A worker uses no join buffer and cloned select_cond,
so the conjuncts were enforced in neither place.

  SELECT STRAIGHT_JOIN b2.v FROM b1, b2
  WHERE b2.x = b1.k AND b2.v > 15 AND b1.id <= 2;

answered two rows serially and ten with workers, the extra eight being every row
of b2 that b2.v > 15 excludes. The join condition still held -- ten rows, not
forty -- which is what pinned it on the single-table half: make_scan_filter()
extracts by table map, so the two-table equality stays in select_cond and only
the single-table predicate moves.

This is the third time the same shape has bitten. The optimizer moves a
predicate out of select_cond into a structure the worker does not replicate:
handler::pushed_idx_cond, then the semijoin strategies, now the join buffer's
scan filter. So the accessor that was introduced for the first case is now
pwt_table_conds(), reporting both halves, and one clone helper ANDs them. The
gate and both clone sites go through it, so the condition the gate approves
stays the condition a worker evaluates.

Only BNL is affected. The hashed and batched buffers give the table an access
type outside JT_EQ_REF/JT_REF/JT_ALL, which the gate already refuses, so they
never reach a worker -- verified across join_cache_level 2, 4, 6 and 8, where
only level 2 differed. BNL is the default.

The conjunction is built with quick_fix_field() over two already-fixed clones,
which is what remove_redundant_bnl_scan_conds() itself does when it rebuilds a
condition out of fixed conjuncts. A worker only evaluates the item, so the
fix-time caches Item_cond::fix_fields() would rebuild are not read.

parallel_query_join gains the query above, answered both ways, asserting that it
still ran in the workers and that b2 still carries a join buffer, so the case
cannot quietly stop being covered. Reverting the cache half of the accessor
brings the eight rows back.

Found while surveying JOIN_TAB for the executor-split work: the survey wanted to
assert that a cloned tab's join-cache fields are inert, which is what turned up
the fact that they are not.

This commit was prepared with Claude Code: it traced the predicate to
make_scan_filter() and remove_redundant_bnl_scan_conds(), established by
sweeping join_cache_level that only BNL reaches a worker, and wrote the test.

MDEV-39492 Parallel Query: give a worker real JOIN_TABs

No functional change. This is the first step of replacing the worker's private
nested loop with the executor's, and it replaces the structure that loop reads.

pwt_jointab described a table in five fields: its TABLE copy, the access type, a
ref, a condition and the sorted flag. sub_select() and evaluate_join_record()
want a JOIN_TAB, so every capability added to the private loop would have to be
added again when the split moves to the real one. A worker now holds an array of
JOIN_TAB, n_tables of them, indexed the way pwt_manager::mgr_tabs is so that [0]
is the parallel-scanned driving table -- the indexing sub_select() will be
started from. The driving table's condition moves onto join_tabs[0].select_cond
with every other table's, so worker_cond goes away.

The tabs are copied from the manager's and then rebound, rather than built from
nothing, so that a field this code does not know about holds the value the
optimizer chose instead of a zero that would look deliberate. Overwritten: the
TABLE, the condition, the ref, the tracker. Cleared: the places a condition can
hide (pre_idx_push_select_cond, cache_select) now that select_cond carries all of
it, and the join buffer, which a worker does not use. Nulled deliberately: the
JOIN, next_select, read_first_record and read_record, which the next two steps
fill in -- reaching them before then is a crash rather than a read of the
manager's execution state.

That last distinction is the point of pwt_assert_tab_inert(). A copy keeps the
manager's pointer in anything the copy does not overwrite, which is the trap
TABLE::map and TABLE::in_use were, and the fields that matter here are ones the
gate is supposed to have excluded rather than ones this code sets. So each is
asserted inert on the manager's tab before the copy: the outer-join chain, the
semijoin strategies and their weedout and firstmatch tables, emb_sj_nest, the
rowid filter, range access, split derived materialization, the DISTINCT
shortcut, HAVING, and the access type. Relax a gate without teaching the copy
about the field it lets through and a debug build stops here instead of
executing with another thread's state.

Writing those assertions is also what found the BNL scan filter fixed in "filter
by the condition the join buffer would apply": asking whether cache_select was
inert turned out to have the answer no, and a wrong result behind it. They are a
survey as much as a guard, so they are worth running before the code they
protect. Verified live on a debug build -- none of them fires across the main
suite with parallel_worker_threads forced on, 1418 tests, and a deliberately
false one aborts the server, so the negative result is a measurement rather than
an assumption.

main and innodb pass in full, 2103 tests, and the parallel tests pass under
--ps-protocol. The forced-worker sweep is at its 18 known failures, all EXPLAIN
and cost output.

This commit was prepared with Claude Code: it established which JOIN_TAB and
JOIN fields the real executor actually reads (4 of JOIN, some 29 of JOIN_TAB, of
which the gate makes most inert), wrote the copy and the assertions, and checked
the assertions execute rather than trusting that they had.

MDEV-39492 Parallel Query: give a worker its own JOIN

No functional change: nothing reads the worker's JOIN until its JOIN_TABs are
driven through sub_select(). Second step of replacing the worker's private nested
loop with the executor's.

A worker cannot share the manager's JOIN. sub_select() writes join->return_tab as
it descends -- the level to unwind to, set unconditionally on entry -- and the
guard that reads it compares against JOIN_TAB*, so with a JOIN_TAB array per
worker one worker's backtrack point would be compared against another's tabs:
pointers into different allocations, and a scan that can end early for no
visible reason. The stronger reason is join->thd, read at twelve sites in that
loop for the diagnostics area, the killed flag and the row counters. Sharing the
JOIN would send all of it to the manager's THD from a worker thread, which is the
bug fixed in "a worker's result table belongs to the worker", reintroduced once
per row and for every query shape. Unlike return_tab it cannot be gated away.

JOIN declares its copy constructor and assignment private and unimplemented, so
the worker's is built by JOIN's own constructor and given what it needs by name.
That constructor is JOIN::init(), a field-initialiser that allocates nothing,
plus a shallow copy of fields_list, so it is cheaper as well as better defined
than copying the bytes of a class whose author said not to. It also matches how
Join_plan_state already holds part of a JOIN: the convention here is a named list
of fields, not an object copy.

Only three fields need carrying, which is what makes a named list practical.
Walking sub_select() and evaluate_join_record() for the shapes the gate allows
leaves thd, return_tab and found_records. map2table is read only for
split_derived_to_update; join_tab_execution_startup() reads join only inside its
two semijoin-materialization branches; JOIN_TAB::preread_init() returns before
touching join->thd unless the table is a materialized derived. All three are
already asserted inert per table by pwt_assert_tab_inert().

result is left null deliberately. It is the manager's connection to the client,
and manager_collect_and_send() is the only thing that may send a row, so a worker
that reaches for it crashes rather than writing to a socket two threads share.

pwt_assert_join_inert() is the same idea as pwt_assert_tab_inert() one level up,
and none of its checks fires across the main suite with parallel_worker_threads
forced on. The sizeof(JOIN) tripwire is debug-only on purpose: JOIN carries
dbug_join_tab_array_size under #ifndef DBUG_OFF, so its size is not the same in
the two build types, and pinning both would leave two magic numbers of which only
one is ever checked by whoever changes the class.

This commit was prepared with Claude Code: it established the three-field surface
by walking the executor and the helpers it calls, found that JOIN is
deliberately non-copyable after having earlier reported the opposite, and
confirmed that main.xtradb_mrr appearing in one forced-worker sweep is a
load-dependent warning-count flake rather than a regression -- it passed six of
seven runs in isolation and a second sweep is back at the known eighteen.
Rex Johnston
MDEV-39492 PQ framework for manager and worker threads

Introduces parallel_worker_threads variable to control the number
of worker threads created by a parallel execution query.

2 new files, sql_parallel_workers.h sql_parallel_workers.cc which
contain structures for the creation, management and deletion of
parallel worker threads (pwt_ in the name).  Main management
class created in the stack in JOIN::exec, implemented for the
top level select.

Threads are registed in server_threads, so are visible in
information_schema.processlist and the show processlist command.

We check that a kill query on a parallel worker is passed onto it's
manager and the query is properly aborted, and that a kill connection
is handled properly in parallel_worker.test.
Daniel Black
MDEV-38405 Assertion `tbl->trn == 0' failed in _ma_set_trn_for_table

Aria bulk insert operations disables share->now_transaction meaning
a concurrent open of the stable table will have
trn == &dummy_transaction_object for its MARIA_HA object during opening.

As the _ma_set_trn_for_table is setting the trn, its harmless if the
current trn is the dummy_transaction_object.

Relax the assert to allow for this state.
bsrikanth-mariadb
MDEV-40518: add both drop table and view stmts

The context only stored "DROP TABLE IF EXISTS t1" before adding a
"CREATE TABLE t1" statement. However, there can be a view named t1
already existing in the database. When the context was replayed,
the CREATE statement failed stating t1 already exists.

Solution is to add both "DROP TABLE IF EXISTS t1", and
"DROP VIEW IF EXISTS t1" before adding a
"CREATE TABLE t1" statement into the context.
bsrikanth-mariadb
MDEV-40220: Add server version to optimizer context

Additionaly, include Version_source_revision as well.
These are read only informative sya variables. So, they are included as
comments instead of SET commands.
Oleksandr Byelkin
fix windows
Oleksandr Byelkin
attempt 4
bsrikanth-mariadb
MDEV-40390: compat/oracle.sp-package fails on replay

UDFs are not yet supported in replay mode.

So, disabling test compat/oracle.sp-package
Daniel Black
MDEV-34482 main.events_processlist test fix

As the test result is dependent of SHOW PROCESSLIST output,
adjust the wait condition to ensure the state is in sleeping
rather than "init" or another state.
Sergei Petrunia
MDEV-40740: Optimizer Context Replay: innodb.xa_unlock_unmodified fails assert

Optimizer Context code changed format_and_store_row() to check both
table->read_set and table->write_set (when required). It used to
use one of those depending on the lock level.

But we don't set the table->read_set bit so we can get an assertion
failure when dumping the column value.

This is fairly rare as Optimizer Context now reads all columns, and the
only other user is statements like "DBUG_PRINT("dml", dbug_format_row(..."